Gradient-Boosted Scalping on
ES Futures

Abstract

We present a hybrid machine learning system for intraday trading of E-mini S&P 500 futures (ES), deployed in live production on CME via Rithmic execution. The system combines three independently-trained models -- an LSTM for tick-level pattern recognition (5,537 parameters, 64.5% accuracy), a LightGBM classifier on 54 engineered features (69.5% win rate solo, 2x ensemble vote weight), and a bar-level ONNX LSTM for 1-minute time-series prediction (53K parameters, highest solo P&L at +$4,494) -- into a weighted ensemble that requires minimum agreement before any signal reaches execution.

The system processes 54 tick-level features and 11 bar-level technical indicators through a 10-stage decision pipeline with 30+ sequential entry gates, achieving a 92.6% signal rejection rate. Only signals surviving all gates reach the exchange.

Development follows an ablation-first methodology: every model and rule-based strategy is validated through systematic ablation studies on historical data before deployment. Three rule-based strategies (Momentum, Mean Revert, STAT_ARB) were added and subsequently removed after ablation proved they degraded ML ensemble performance. The resulting ML-only configuration achieves 65.7% win rate, 1.80 profit factor, and 16.25x return-to-max-drawdown ratio across 447 simulated trades.

Data & Feature Engineering

The system ingests tick-level market data from CME E-mini S&P 500 futures (ES) via Rithmic, captured by a Sierra Chart DLL (T29-LocalBridge_v39.cpp) that writes two CSV streams in real time. The Python bridge reads these files in a 20ms polling loop.

The training dataset consists of ~200K raw ticks collected from live ES H6 futures trading, supplemented by a 614K tick backfill (ticks_RGW.csv). All three models were retrained on February 26, 2026 using the 200K tick dataset with temporal train/validation splits (80/20) and no future-peeking.

The feature engine (features.py) computes 54 tick-level features on every incoming tick, organized into the following categories:

All tick features are normalized to a [-10, 10] range using rolling statistics. Additionally, an 11-feature bar-level pipeline aggregates 1-minute OHLCV bars and computes standard technical indicators used exclusively by the ONNX model:

The PSO (Oscillator Power) indicator is computed from a 500-tick stochastic oscillator smoothed by EMA-25, producing a value in [-1.0, +1.0]. PSO drives the position sizing tier system and provides momentum alignment context to the ensemble. See full pipeline diagram →

Model Architectures

The ensemble comprises three independently-trained models, each designed to capture different temporal scales and feature representations. All models produce directional signals (BUY/SELL) with confidence scores. No single model can trigger a trade alone.

Training methodology: All models use temporal train/validation splits (80/20) with strict temporal ordering -- no shuffling, no future information leakage. F1 and ONNX use class-balanced sampling to handle the inherent imbalance between directional labels. F2 LightGBM uses scale_pos_weight for class balance. All three models were retrained on February 26, 2026 on 200K ticks, producing measurable accuracy improvements: F1 64.5% (was 55.5%), F2 54.6% (was 52.9%), ONNX 54.3% (was 53.8%).

Ensemble & Decision Pipeline

The three models vote independently on every tick. Votes are weighted: F1 contributes 1 vote, F2 contributes 2 votes (reflecting its superior per-trade quality), and ONNX contributes 1 vote, for a total of 4 votes. The ensemble computes:

AI direction override: In earlier iterations, a mean-revert pipeline determined trade direction and models voted on confidence. The current system lets ML models directly determine trade side through ensemble consensus. This eliminated directional conflicts between rule-based signals and model predictions.

The full decision pipeline consists of 10 stages. A signal must survive all stages to become an order on the exchange:

See interactive pipeline diagram with full stage detail →

Risk Management

The risk framework operates at three levels: per-trade brackets, per-session budgets, and per-day global limits. The gate stack at Stage 5 is deliberately aggressive -- it is cheaper to miss a good trade than to take a bad one. Each gate was added to fix a specific failure mode observed in live or simulated trading.

30+ Sequential Entry Gates (92.6% rejection rate on backtest, Feb 24):

Session Budgets:

PSO-driven Kelly sizing determines position size through a 9-layer soft multiplier stack. The PSO alignment value determines the sizing tier: TURBO (pso >= 0.5, 2x multiplier), BOOST (pso >= 0.25, 1.5x), NORMAL (1x), CONTRA (pso <= -0.5, 0.5x). After 3+ consecutive losses, sizing is forced to 1 lot regardless of Kelly or PSO. Maximum position: 10 lots.

Bracket management: Every entry receives an exchange-managed OCO bracket (hard stop + profit target). R-multiple trailing activates at R=0.6 (breakeven), R=1.0 (5t trail RTH / 4t ETH), R=1.5 (tight 3t/2t trail), and R=4.0 (take profit). Minimum 30-second hold time before any stop evaluation prevents the instant-stop toxicity identified in deep analysis. See full risk page →

Ablation Studies

All strategic decisions are validated through systematic ablation. The methodology: monkey-patch the ensemble configuration, replay the backtester on the same historical dataset, and compare 9+ configurations on identical data. This eliminates market condition variance and isolates the contribution of each component.

Central finding: Every rule-based strategy tested degraded ML ensemble performance. Three strategies were added and subsequently removed after ablation:

ML vs. Rule-Based Ablation (644 trades):

Model Solo & Pair Performance:

Statistical Results

Performance metrics from the ablation study (ML-only configuration) and deep analysis conducted on 148K ticks of historical data. All results are from simulated trading on historical data; past results do not guarantee future performance.

Headline metrics (ML-only, 447 trades):

Deep analysis findings (148K ticks, February 21, 2026):

Model comparison (solo performance):

Architecture

The system is designed around a broker-agnostic abstraction layer. Two abstract base classes -- DataService (tick/DOM data) and TradeService (orders/fills) -- define the interface. Broker-specific implementations are swappable providers. All strategy logic, risk management, and ML inference lives in shared core that never changes when swapping providers.

Current deployment: Parallels Windows 11 VM (ARM64) running Sierra Chart with a C++ DLL (T29-LocalBridge_v39.cpp) that bridges tick data and order flow to a Python bridge via CSV files. The bridge runs a 20ms polling loop, processing ~50 ticks/second during active markets.

State persistence & crash recovery: The system persists state to disk on every trade event. On restart, it reads the last known position from Sierra's fill log and reconciles with the DLL's heartbeat file. A 10-second startup lockout prevents trading on stale state.

Scaling path:

See full architecture page with file map and service diagram →

Current Version

The system follows a strict ablation-validated release process. Every model and strategy component is tested against historical data before deployment. Components that degrade performance are removed immediately -- three rule-based strategies have been killed by this process to date.

Past results are from simulated trading on historical data and do not guarantee future performance. The system is deployed in live production ($50K evaluation). All metrics should be interpreted in the context of a limited dataset and evolving market conditions.