The future is withheld, not dimmed.
Replay a real trading session to a model as if it were arriving live — minute bars and news, in order, on a clock you control. Change the prompt, change the model, run the same day again. The variable under test is your instructions, not your parameters.
The one rule
An order is never filled using the bar the agent saw when it decided.
The agent reads the 14:31 bar, decides, and sends an order. That order first fills against 14:32. Filling at the 14:31 close — which is sitting right there, and which a naive engine takes — hands back a price the agent already knew, and every strategy looks brilliant.
The same defect, in a different costume, is an out-of-sample window that quietly contains the decision window. We shipped that bug once in our own research harness. It is the reason this engine has a sequence gate and a test that fails if a fill ever lands on the decision bar.
| Order | Fills at |
|---|---|
| market | next delivered bar's open, adjusted by slippage — a buy at 1 bp pays open × 1.0001, never the bare open |
| buy limit | only if next bar's low ≤ limit — at min(open, limit) |
| sell limit | only if next bar's high ≥ limit — at max(open, limit) |
| stop | arms when the range touches it, fills as market on the bar after |
Three surfaces, one engine
Alpaca-shaped, so an existing client mostly just works.
The feed uses Alpaca's auth handshake, its subscribe exchange, and its exact bar schema (T:"b" with S,o,h,l,c,v,n,vw,t), batched in arrays. An Alpaca SDK connects, authenticates, subscribes and decodes bars unmodified.
It is not byte-identical, and we had that claim audited out. After auth we send one frame Alpaca never sends, T:"replay", carrying the session_id the order API requires — so a trading client does need that one addition. Only bars and news are delivered; trades, quotes and dailyBars are accepted and echoed but never produce data. n is always 0 and vw is always the close, because we store neither — so VWAP strategies are degenerate here and v/n divides by zero.
MCP
Drive a session from your own agent. It pulls, so it never drops an event while thinking — and a slow model doesn't quietly get an easier stream.
WebSocket
Bars and news pushed on a clock you set. Realtime, 1200×, or drained instantly.
REST + fills
Orders in over REST, fills out on a trade_updates socket.
# market data — one URL from a real broker wss://api.backtrade.dev/v2/sip?date=2026-04-30&symbols=SPY&speed=60&blind=1 # order in POST https://api.backtrade.dev/v2/orders?session_id=<id> {"symbol":"SPY","qty":50,"side":"buy","type":"market"} # fill out (numbers are strings here, exactly as Alpaca sends them) {"stream":"trade_updates","data":{"event":"fill","price":"711.79","qty":"50",...}}
The part nobody else will tell you
If the model memorised the day, it isn't trading. It's reciting.
This is the failure that makes most LLM backtests meaningless, and it is invisible in the results — a model recalling a session produces a beautiful equity curve. So every date carries a leakage verdict, and any model without a measured memory cliff is reported contaminated. An unmeasured model is not a clean model.
Exactly one model has been measured here — claude-opus-5, from a date-conditioned answer-rate cliff over 2,368 answers, bounded at 2026-03-01. Everything else returns contaminated, and that is the honest state, not a placeholder. An earlier version of this table listed four models as though all four had been probed; an adversarial audit caught it, and caught that the one real entry was set early enough to certify a month it had itself measured at a 36.7% answer rate.
Blinding is the second line: prices become a base-100 index and the subject ticker leaves the headline, so a session is harder to recognise.
What this cannot model
- Queue position. A limit that the bar's range merely touches might never have filled in reality. We fill it. This flatters limit strategies.
- Market impact. Your order is assumed not to move the price. The share of a minute's volume you took is reported, and the scorecard flags itself when you leave that regime.
- Blinding is partial. Co-mentioned tickers are not redacted, so a determined model could still recognise a date from the news text.
- One shared key, no user isolation. Sessions are bound to the key that made them, and ids are random — but a single key still owns everything it creates. Per-user separation needs real accounts, which do not exist yet.
- Coverage is narrow. A typical SPY session is ~401 bars and ~10 news items. 2,112 symbols have some 1-minute data in the window, but only 69 reach 90% of SPY's bar count.
- Speed is a latency multiplier. The clock runs faster; your round trip does not shrink with it. A fast replay silently models a very slow trader.
Treat every result as an upper bound on what the same logic would have earned.
Honest status
Early. Working. Not finished.
This page exists so the state is legible rather than implied.
| Piece | State |
|---|