agentsclimarketplace

Tensortrade rl

Skill mahmoud20138/Tradecraft/plugins/tradecraft/skills/tensortrade-rl

TensorTrade — composable RL framework for building and training reinforcement learning trading agents. Modular TradingEnv with Observer, ActionScheme, RewardScheme, Portfolio, Exchange components. Integrates Ray RLlib for distributed training. PythonFrom its SKILL.md

Install
npx -y skills add mahmoud20138/Tradecraft --skill tensortrade-rl

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 18 stars18 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

4.9 KB, ~1.0k tokens by cl100k_base, as published. Nobody here has run it

tensortrade-rl

USE FOR:

  • "train RL agent to trade"
  • "reinforcement learning for algorithmic trading"
  • "PPO / DQN / A3C trading agent"
  • "custom trading environment (gym-style)"
  • "backtesting with RL agent"
  • "distributed RL training with Ray"
  • "position-based reward function trading" tags: [RL, reinforcement-learning, trading, PPO, DQN, Ray, RLlib, backtesting, gym, TensorTrade, algorithmic] kind: framework category: quant-ml-trading

What Is TensorTrade?

Open-source Python RL trading framework. Builds composable TradingEnv environments to train and evaluate RL agents against market data.

Research finding: PPO on BTC/USD achieves directional prediction at zero commission, but trading frequency → commissions exceed profits in live conditions.


Core Architecture

┌─────────────── TradingEnv ────────────────────┐
│                                               │
│  Observer      → feature engineering          │
│  ↓                                            │
│  Agent         → RL policy (PPO/DQN/etc.)     │
│  ↓                                            │
│  ActionScheme  → BUY / SELL / HOLD orders     │
│  ↓                                            │
│  Exchange      → simulated execution + fees   │
│  ↓                                            │
│  Portfolio     → wallet + position tracking   │
│  ↓                                            │
│  RewardScheme  → position-based returns       │
└───────────────────────────────────────────────┘

Installation

python3.12 -m venv tensortrade-env
source tensortrade-env/bin/activate
pip install -e .
pip install -r examples/requirements.txt

Basic Usage

import tensortrade.env.default as default
from tensortrade.feed.core import Stream, DataFeed
from tensortrade.oms.exchanges import Exchange
from tensortrade.oms.services.execution.simulated import execute_order
from tensortrade.oms.wallets import Wallet, Portfolio

# Define exchange and portfolio
coinbase = Exchange("coinbase", service=execute_order)(
    Stream.source(price_data, dtype="float").rename("USD-BTC")
)
portfolio = Portfolio(USD, [Wallet(coinbase, 10000 * USD), Wallet(coinbase, 0 * BTC)])

# Build environment
env = default.create(
    portfolio=portfolio,
    action_scheme="managed-risk",     # BUY/SELL/HOLD with risk sizing
    reward_scheme="risk-adjusted",    # Sharpe-ratio based reward
    feed=DataFeed([price_stream, volume_stream, rsi_stream]),
    window_size=20,
)

Training with Ray RLlib

from ray import tune
from ray.rllib.algorithms.ppo import PPOConfig

config = (
    PPOConfig()
    .environment(env_class, env_config={"window_size": 20})
    .training(lr=1e-4, gamma=0.99, train_batch_size=4000)
    .rollouts(num_rollout_workers=4)
)

tune.run("PPO", config=config.to_dict(), stop={"episodes_total": 500})

Hyperparameter Optimization (Optuna)

# examples/training/train_optuna.py
# Searches: lr, gamma, clip_param, window_size, reward_scheme

Action Schemes

SchemeDescription
simpleBUY / SELL / HOLD discrete actions
managed-riskPosition sizing + stop-loss built-in
bshBinary: BUY or SELL only

Reward Schemes

SchemeDescription
simpleRaw P&L per step
risk-adjustedSharpe ratio-based
position-basedReturn based on current position

Key Research Insights

FindingDetail
Directional accuracyPPO learns BTC direction at 0 commission
Commission sensitivityEven small fees (0.1%) significantly hurt returns
Training regimeNeeds 200+ episodes to converge on noisy data
Overfitting riskWalk-forward validation critical

Pre-built Examples

# Simple training run
python examples/training/train_simple.py

# Distributed training (Ray)
python examples/training/train_ray.py

# Hyperparameter search
python examples/training/train_optuna.py

# Use pre-tuned config
python examples/training/train_best.py

What ships with it

Read from the repository

Just SKILL.md. No reference files, no scripts.

Keep looking

Skills are one crate of 610,677. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.