Funding: 1.75 ETH (~$450 USD)

Description: Create an implementation of Exchange capable of trading any forex pair on the MetaTrader 4 (MT4) platform.

Requirements:

• The exchange implementation must be completely interoperable with MT4, including spinning up any necessary ZeroMQ infrastructure.
• The exchange must fully implement the Exchange class, including the ability to list balance, net worth, asset valuations, trade history, etc.
• The exchange must be capable of returning observations from OHLCV, indicators, signals, and other data from the MT4 platform.
• The exchange must be capable of executing live trades on the MT4 platform, with either live or demo accounts.
• Must include an example with an agent trading using a demo account.

Funding: 1.5 ETH (~$300 USD)

Description: Create the necessary components to support observing multiple assets from an exchange, applying a feature pipeline to those assets, and making a trade on any of those assets.

Requirements:

• Create the necessary InstrumentExchange components (or update existing components) to support observing/trading multiple assets on both a simulated and live exchange.
• Ensure the exchanges support feeding their observations through a simple feature pipeline.
• Create the necessary ActionStrategy component(s) to support trading multiple assets at a time.
• Create the necessary RewardStrategy component(s) for rewarding multiple concurrent trades.
• Create the necessary unit tests to confirm the new components are working as expected.

TensorTrade tutorial article

Funding: 1.5 ETH (~300 USD)

Description: Create a useful visualization of the TradingEnvironment in the render method.

Requirements:

• Implement a useful visualization of the TradingEnvironment and the underlying the learning agent's interactions with the environment's InstrumentExchange.
• Display the exchange balance, and all exchange observations/agent trades during each episode in a useful manner.
  • E.g. a candlestick chart with highlighted trades
• Display the total P/L, sharpe ratio, largest drawdown %, and any metrics you see as useful.
• The rendered visualization should use a performant, well-designed charting application such as Bokeh or Dash

Use the RLTrader Github and the corresponding Medium article as a reference.

Funding amount: 1.5 ETH (~$300)

Description: Create fully-featured unit tests for each existing component in /tensortrade and the documentation in /docs.

Requirements:

• Each component should be tested via pytest in its corresponding /tests/tensortrade/**/test_*.py file.
• Each use case of each component should be 100% covered via the unit tests.
• Each function parameter and class argument that affects a component's functionality should be covered by the unit tests.
• Each documentation feature and page should be tested via pytest in its corresponding /docs/**/test_*.py file.
• Each test should be passing.
• Any bugs in components should be fixed during this process.
• Note: You are not required to implement unimplemented methods that raise a NotImplementedError().

TensorTrade tutorial article

Funding: $250 USD (GitCoin link pending) Feature: Enable CUDA GPU acceleration in ParallelDQNAgent

As a user I want to use a CUDA GPU device to accelerate the ParallelDQNAgent's calculations. However, this does not currently work due to a bug with CUDA running in multiple processes.

Scenario: Train a parallel agent w/ a GPU GIVEN I have a ParallelDQNAgent (agent) AND I am running tensortrade on a machine w/ 4+ cores and 1+ GPU WHEN I run agent.train(n_steps=1000, n_envs=4) THEN the agent should train 1000 steps within 4 separate environments AND the agent should share its model between each of the 4 environments AND the model should utilize the GPU for all possible tensorflow operations

This feature request is written using Gherkin: https://cucumber.io/docs/gherkin/reference/

For reference, please look at the ParallelDQNAgent trained in: https://github.com/tensortrade-org/tensortrade/blob/master/examples/train_and_evaluate.ipynb

this is something I noticed during training. That balances will remain in locked_balances when buy -> sell is executed.

Here is a simple test to illustrate the issue.

def test_runs_with__buy_sell_error():
    df = pd.read_csv("tests/data/input/coinbase_(BTC,ETH)USD_d.csv").tail(100)
    df = df.rename({"Unnamed: 0": "date"}, axis=1)
    df = df.set_index("date")

    ex1 = Exchange("coinbase", service=execute_order)(Stream("USD-BTC", list(df['BTC:close'])))

    portfolio = Portfolio(USD, [
        Wallet(ex1, 10000 * USD),
        Wallet(ex1, 0 * BTC)
    ])

    coinbase_btc = df.loc[:, [name.startswith("BTC") for name in df.columns]]

    nodes = []
    with Module("coinbase") as coinbase:
        for name in coinbase_btc.columns:
            nodes += [Stream(name, list(coinbase_btc[name]))]

    feed = DataFeed()(coinbase)

    action_scheme = DynamicOrders(
        trade_sizes=[1.0],
        trade_type=TradeType.MARKET
    )
    reward_scheme = SimpleProfit()

    env = TradingEnvironment(
        portfolio=portfolio,
        action_scheme=action_scheme,
        reward_scheme=reward_scheme,
        feed=feed,
        window_size=5
    )

    env.reset()
    holding_action = 1
    # Buy USD/BTC use all available funds
    buy_all_action = 2
    # Sell BTC/USD use all available funds
    sell_all_action = 3

    env.step(buy_all_action)
    assert len(env.broker.executed) == 1

    env.step(sell_all_action)
    assert len(env.broker.executed) == 2
     # balances will stay in locked_balances and not return in to balances

    env.step(buy_all_action)
    # order will get get executed
    # assert len(env.broker.executed) == 3

Prerequisite: This requires a fix to the class DynamicOrders(ActionScheme): [...] price = exchange.quote_price(pair) instead of price = exchange.quote_price(instrument) [...]

Maybe its a issue that the order service needs to be adjusted or the default order server is not working with the action scheme? However, I would expect to the traded funds to back into the balances when the env shows the trades as executed and not in locked balances

This is the requested feature from issue 46 (https://github.com/notadamking/tensortrade/issues/46) It is a variation on DiscreteActionStrategy in which there is a profit target and stop-loss for each trade, which is recorded statefully as a part of the class init, in percentages. Based on the selected profit target and stop-loss, which both default to 1%, if a trade goes above the profit target or below the stop loss (or equals either) the program must use that timestep to trade the oldest amount bought.

Background: TT got the completely new data feed system to feed data into the environment. It's based on the Node object that encapsulates the way how data can be organized into a pipeline.

One Node can feed data to another Node which can go to a third Node that will multiply a value by some other values, that come from different Nodes. The environment just calls feed.next() every step to get the next frame and doesn't know how and where the data come from. The interface above is too generic and flexible that allows easy implementing different data sources, transformations, Talib indicators, etc - the interface remains the same.

For example, I already implemented a Delay node which returns a value with a predefined delay (to get a price from an hour ago, or a day ago) and a TAlib node which calculates indicators based on an input stream.

One important implication of this system is that there is no pre-transformation - you just select the source, define the pipeline and pass it to the environment - all transformations and aggregation will be executed as the environment calls feed.next().

But there is a problem: some node transformations might require some amount of data to be collected before it can return a meaningful result. For example, the calculation talib.SMA(close, timeperiod=30) will require 30 elements before it can return the corresponding SMA. Before that, it will return N/A, which is an invalid value for a neural network.

You can try to ignore it, and for training, it won't likely affect the model too much, but on a live exchange, your model might make incorrect actions based on incorrect SMA.

So here is my PR: the ability to tell the environment to ignore first N frames before it actually starts sending a data frame to an agent and executing actions.

Funding amount: 2 ETH (~$400)

Description: Document each component described in the article, as well as each existing feature in the codebase (Each file in the /tensortrade directory).

• Write a complete description of each component and file in the /tensortrade directory.
• Include example usage of each use case and its possible solutions.
• Briefly define and explain any variables or parameters.
• Documentation must be publishable via Readthedocs.org using the Sphinx framework.
• Documentation should be brief, yet complete. It should be simple and easy to read.

Use the TensorTrade tutorial article as a reference for describing components and defining their usage.

Funding: 0.25 ETH (~$50 USD)

Description: Replace TALibIndicator component in /tensortrade/features/indicators with a component called TaIndicator, that uses the bukosabino/ta package rather than TA-lib.

Requirements:

• Replace the TALibIndicator with a TAIndicator.
• Remove the SimpleMovingAverage indicator, and replace its usage in all examples with the corresponding TAIndicator feature.
• Create the necessary parameters and kwargs to use all indicators in the bukosabino/ta package within a feature pipeline.

bukosabino/ta GitHub TensorTrade tutorial article

Hello, I am testing out PPO with BSH action scheme and position-based returns. I am quite happy with the learned models but it seems to be too overfitted to the train data. In fact, the model is worse for unseen test data. What is your recommendation to avoid overfitting? I am thinking about random starting in the ENV but do not know the right way to implement it? Is there anyone working on it? Thanks

This part of logic is potentially a bug:https://github.com/tensortrade-org/tensortrade/blob/cc6bd19fbb8451c7972a3a08132407f508c4d47b/tensortrade/env/default/actions.py#L261-L266

For a USD/BTC pair, when the side is sell, quantity is measured in BTC. However, or size < self.min_order_pct * portfolio.net_worth \ is comparing BTC against USD. Is this potentially a bug?

What is the purpose of this condition?

Many thanks!

Hey,

When running from tensortrade.env.default I get this error:

'EntryPoints' object has no attribute 'get'

Currently running python 3.7.14 on Linux.

Seems this issue is connected to new release of importlib-metadata 5.0.0 and people mitigate this through downgrading to importlib-metadata==4.13.0.

I can also confirm it worked in my case.

I'm opening a PR to get feedback on adding support for shorting.

I need this feature, and I'd like to share it with the community too.

Details

The implementation intentionally keeps the original behavior in place as much as possible - which is no shorting support . Developers have to be specific in the code to enable it. What this PR adds:

• MarginWallet - a class that should be explicitly used when shorting is needed. Instead of initializing a Wallet instance, developers can initialize a MarginWallet
• NegativeQuantity - I created this class as the current implementation is pretty strict with throwing errors for negative quantities, and I wanted to leave that functionality untouched.

TODO:

• [ ] Add documentation and sample code showing how to setup shorting.

Related Issues

• #317
• #439
• #327

I have implemented the basic ray example found on:

https://www.tensortrade.org/en/latest/tutorials/ray.html

and made a basic change to the reward function. The idea was to reward the price difference since the last transaction instead of the price difference since the last step. When I run this example multiple times from fresh new environments I seem to get incoherent results.

Another slight change from the example is I run tune with tf instead of torch

I am certain somebody here has though/implemented this logic. Anybody see any issue with this?

Here is my new reward function

`

def __init__(self, price: 'Stream'):
    super().__init__()
    self.position = -1
    self.lastWorth = 0
    self.previousLastWorth = 0
    self.lastAction = 0

    price = Stream.sensor(price, lambda p: p.value, dtype="float")
    price = price.rename("price")
    self.priceFeed = DataFeed([price])
    self.priceFeed.compile()

    position = Stream.sensor(self, lambda rs: rs.position, dtype="float")
    previousLastWorth = Stream.sensor(self, lambda rs: rs.previousLastWorth, dtype="float")

    reward = ((price - previousLastWorth) * position).fillna(0).rename("reward")
    self.rewardFeed = DataFeed([reward])
    self.rewardFeed.compile()

def on_action(self, action: int):
    if self.lastWorth == 0:
        self.position = 0

    if abs(action - self.lastAction) > 0:
        self.lastAction = action
        self.position = -1 if action == 0 else 1
        self.previousLastWorth = self.lastWorth
        self.lastWorth = self.priceFeed.next()["price"]
    else:
        self.position = 0


def get_reward(self, portfolio: 'Portfolio'):
    return self.rewardFeed.next()["reward"]

def reset(self):
    self.position = -1
    self.rewardFeed.reset()
    self.priceFeed.reset()
    self.lastAction = 0
    self.lastWorth = 0
    self.previousLastWorth = 0

`

And just in case, here are some versions I am using.

Ubuntu 22.04.1 LTS Python 3.10.4 tensorflow==2.9.1 tensortrade==1.0.3

Furthermore, I am not certain if this is a bug, as I am quite new to RL and python, however neither of the links recommended for more generic questions seem to work:

Please ask your question on StackOverflow or on the TensorTrade Discord #help-desk channel instead of opening a GitHub issue.

from tensortrade.data.cdd import CryptoDataDownload will load agents even if you don't use them because tensortrade/__init__.py have: from . import agents where tensortrade/agents/__init__.py have from .dqn_agent import DQNAgent, DQNTransition etc

Got it with unchecked debugger breakpoint in tensortrade/agents/dqn_agent.py