How to build an AI trading strategy: a twelve-step protocol
Nine of these twelve steps happen before any money is at risk, and that ratio is the point. The modelling is the short part; the long parts are defining what failure looks like, obtaining data that does not contain the future, modelling costs honestly, validating against the search that produced the result, and building a stop that works when the strategy does not. This page sets out the sequence and what each step protects against. It is a methodology guide, not advice to trade.
Published 27 August 2026 · Updated 28 August 2026 · AI Trading Book Editorial · Reading time about 14 minutes
- Decide what failure looks like first. Without an exit criterion, every result is a success in retrospect.
- Beat the baseline or stop. Buy-and-hold and a simple rule, same data, same costs.
- Never shuffle time-series data, and open the final holdout exactly once.
- Build the kill switch before deployment, independent of the strategy code, and test it.
- Scale by rule, not by mood. Increase size only when a stage completes as planned.
The sequence at a glance
The protocol below is deliberately ordered so that each step protects against a specific, documented failure. Reordering it — particularly moving cost modelling or the split decision later — is how backtests become misleading.
Phase 1 — Design
Step 1. Define the objective and the failure condition
Write down the market, instrument, holding period, capital committed, maximum drawdown you will accept, and the specific condition under which you will stop. This is the step most often skipped, and skipping it makes everything afterwards unfalsifiable: without a pre-set failure condition, any outcome can be rationalised as within expectations.
The drawdown figure deserves particular care because it is the number you will be tempted to revise while losing money. Set it while calm, write it down, and treat it as binding.
Step 2. Confirm your regulatory position
Trading your own account is generally not a regulated activity in the UK, Australia or New Zealand. Doing it for other people generally is — as advice, as portfolio management, or in New Zealand potentially as a Discretionary Investment Management Service. Pooling money from several people, managing a relative's account, or selling signals all move you toward the perimeter.
Resolve this before writing code, not after. The three regulation pages on this site cover the UK, Australia and New Zealand; where your position is unclear, the answer is qualified legal advice rather than an educated guess.
Step 3. Establish a baseline
Measure buy-and-hold and one simple rule — a moving-average crossover will do — on the same data, over the same period, with the same costs. This gives you the number your strategy must beat to be worth anything.
Most strategies fail here and the failure is informative. A system that returns 9% while the index returned 13% has not found an edge; it has found a more expensive way to hold the market. Our evidence page covers the fund-level version of the same comparison.
Step 4. Write the research protocol before touching the data
Fix the hypothesis in one sentence, the universe, the split dates, the cost assumptions and the criterion for success — all before running anything. Then open a trial log and record every attempt as you make it, including the ones you abandon after thirty seconds.
The trial count is what makes statistical correction possible later, and it cannot be reconstructed honestly after the fact. Everyone undercounts a search they have already finished.
Phase 2 — Data and testing
Step 5. Obtain point-in-time data
Three properties matter more than data volume. The universe must include delisted and acquired instruments, or the sample silently excludes every failure. Prices must be adjusted for corporate actions. And any fundamental or reported data must be dated by publication, not by the period it describes — using a figure on its period-end date acts on information the market did not yet have.
Step 6. Split chronologically
Train on an earlier period, validate on the period immediately after, and reserve a final block that you will open exactly once. Never shuffle. Shuffling time-ordered data places observations from after the prediction point into training, which is the mechanism behind reported accuracies above 90% — covered on our accuracy page.
Step 7. Model costs, then double them
Costs decide more strategies than models do. Include everything below, then re-run with trading costs doubled. If the result does not survive, what you have is a cost-model artefact.
| Cost | Applies to | Effect of omitting it |
|---|---|---|
| Spread | Every trade | Largest single omission at high turnover |
| Commission | Every trade, per broker schedule | Understates cost of small positions disproportionately |
| Slippage | Difference between decision and fill price | Backtest fills at prices that were not available |
| Overnight financing | Leveraged positions held overnight | Makes long-held leveraged positions look free |
| Borrow cost | Short positions | Short strategies appear viable when they are not |
| Currency conversion | Cross-currency instruments | Silent drag on every trade |
| Market impact | Positions large relative to liquidity | Strategy works at test size, fails at real size |
| Data and infrastructure | The system as a whole | Fixed costs consume small accounts entirely |
Step 8. Walk-forward validation
Roll the training and testing windows forward across the history so every evaluation happens on data later than everything the model saw. This approximates operating the strategy through changing conditions, and it prevents the single-split shortcut of tuning until one test period looks acceptable.
Report the walk-forward periods separately rather than as an average. A strategy that made everything in one window and nothing in the others has told you something an average would hide.
Phase 3 — Correction
Step 9. Correct for the search
Deflate the Sharpe ratio using your trial count, track-record length and return distribution, and compute the Probability of Backtest Overfitting on your selection procedure. Both tools come from Bailey and López de Prado and exist precisely because the best of many results looks good regardless of whether anything is there.
Harvey, Liu and Zhu's recommendation of a t-statistic above 3.0 rather than the conventional 2.0 applies here. The full treatment is on our backtest overfitting page.
Step 10. Paper trade for a predefined period
Run the system live without capital, for a duration fixed before you start. The point is not the simulated profit — it is the divergence. Comparing live signal generation against backtest expectations exposes data feed differences, timestamp assumptions, latency, and fill quality that a simulation cannot show.
If live signals differ materially from backtest signals on the same days, stop and find out why. That divergence will not shrink when money is involved.
Phase 4 — Deployment
Step 11. Build the kill switch and the limits first
Risk controls are built before deployment, not added afterwards, and the kill switch must operate independently of the strategy code — because the circumstances that require it usually involve that code behaving unexpectedly. This is not merely good practice: under UK-onshored RTS 6 article 12, firms engaged in algorithmic trading are required to maintain the ability to cancel unexecuted orders immediately, and ASIC's CP 386 proposes an equivalent immediate-suspension control for Australia.
| Control | What it limits | Set before deployment |
|---|---|---|
| Maximum position size | Exposure to any single instrument | As an absolute figure, not a percentage of a moving balance |
| Maximum total exposure | Aggregate risk across positions | Including correlated positions treated as one |
| Daily loss limit | Damage from a bad day or a bug | Automatic halt on breach, not an alert |
| Order rate cap | Runaway loops sending orders | Orders per minute, hard-capped |
| Duplicate order detection | Repeated submission of the same intent | Independent of the strategy logic |
| Kill switch | Everything, at once | Manual, tested, works if the strategy process is unresponsive |
| Stale data halt | Trading on a frozen feed | Stop if data is older than a set threshold |
| Reconciliation | Divergence between assumed and actual positions | Checked against the broker, not internal state |
Test the kill switch before you need it. An untested stop is an assumption, and the Knight Capital episode of 1 August 2012 — roughly USD 440 million lost in about 45 minutes across more than four million executions — is the canonical demonstration of what happens when a deployment problem outruns the ability to stop it.
Step 12. Deploy capital in stages
Start at a fraction of intended size. Hold each stage for a period fixed in advance, compare live results against expectations, and increase only if the stage completed as planned. Increasing because recent results were good is the most common route from a working system to a large loss, because it concentrates size exactly when a strategy has been luckiest.
Where a language model fits
A large language model is useful for research, code generation, document summarisation and interpreting unstructured text. It is not a validated forecasting system, and it should not hold unrestricted trading authority.
The defensible architecture separates the two: the model proposes, and deterministic risk logic disposes. Every order passes through independent validation against position limits, exposure caps and sanity checks before it reaches the broker, and that validation does not consult the model. This keeps the useful capability while removing the failure mode where a confident, wrong output goes straight to market — and it is also what makes the system auditable, since the constraints are readable code rather than model behaviour.
Retraining and retirement
Decide the retraining cadence in advance and document it. Retraining after a drawdown is a common instinct and a bad one: it fits the model to the most recent noise and destroys any claim that live results were out-of-sample. Whatever cadence you choose, your validation cadence should match it — a model retrained monthly is not meaningfully validated by an annual review, which is one of the practical tensions in applying RTS 6 to adaptive systems.
Retirement criteria belong in the same document as the deployment criteria. Persistent divergence between live and backtest results, a drawdown past the step-one limit, or the disappearance of the market condition the strategy relied on are all reasons to stop. Defining them beforehand means the decision is made once, calmly, rather than repeatedly while losing money.
Common mistakes, in order of cost
| Mistake | Why it is costly | Prevented by |
|---|---|---|
| Iterating against the test period | Destroys the only out-of-sample evidence you had | Step 6 — open the holdout once |
| No baseline | Market return mistaken for strategy skill | Step 3 |
| Omitted or optimistic costs | Turns unprofitable strategies profitable on paper | Step 7 |
| Shuffled data split | Produces accuracy that cannot exist live | Step 6 |
| Unrecorded trial count | Makes statistical correction impossible | Step 4 |
| Deploying at full size immediately | Concentrates the cost of every unknown into one event | Step 12 |
| Risk controls added after launch | They are needed exactly when there is no time to build them | Step 11 |
| Withdrawal-enabled API keys | A compromise becomes a total loss rather than a trading loss | See API key security Knight Capital, 2012 |
| Raising size after a good run | Adds capital exactly when results have been most favourable | Step 12 |
A realistic expectation
Most strategies that reach step 9 do not survive it, and that is the protocol working rather than failing. The evidence across this site points the same way: AI hedge fund indices have underperformed passive benchmarks over fifteen years, a review of 27 machine-learning equity experiments found no conclusive evidence of returns at scale, and careful directional accuracy sits a few points above a well-chosen baseline.
The realistic reason to build one of these systems is process rather than edge — removing discretionary error, enforcing position limits mechanically, and being able to state exactly why a trade happened. Those are real benefits and they do not require beating the market.
What we could not establish
- A success rate for retail strategies reaching live deployment. No published study tracking retail strategies from backtest to live outcome was identified. Data not found.
- A recommended minimum capital figure. This depends on instrument, broker fee schedule and position sizing; no authoritative threshold exists. Data not found.
- A standard paper-trading duration. No regulator or standards body specifies one. Data not found.
Key takeaways
- Write the failure condition before the strategy. Everything downstream depends on having a standard to fail against.
- The baseline decides whether you have anything. Beating it after costs is the whole test.
- Costs and the split method decide more outcomes than model choice does.
- Correct for your search. Trial count, deflation, PBO — or your result is uninterpretable.
- Build and test the stop before you need it. Knight Capital lost roughly USD 440 million in about 45 minutes.
- Scale on a rule set in advance, never in response to a good run.
Frequently asked questions
Do I need to know how to code to build an AI trading strategy?
Not necessarily, but you need to understand what the system does. No-code platforms remove the programming barrier without removing the methodological one. If you cannot explain why your strategy should work and how it is validated, the interface has not solved your problem.
What is the first step?
Writing down what you are trying to achieve and what would count as failure — market, instrument, holding period, capital, maximum acceptable drawdown and the exit criterion. Skipping this makes every later decision unfalsifiable, because there is no standard to fail against.
Why establish a baseline before building anything?
Because without it you cannot tell whether a result is good. Measure buy-and-hold and a simple moving-average rule on the same data with the same costs. A strategy that does not beat the trivial alternative after costs is not worth deploying, however sophisticated the model.
How much historical data do I need?
Enough to cover multiple market regimes — rising, falling, volatile and quiet — rather than a fixed number of years. A strategy tested only on a trending period has been tested on one regime, and the relevant question is what happens when that regime ends.
What costs should a backtest include?
Spread, commission, slippage between decision and fill, overnight financing on leveraged positions, borrowing costs for shorts, currency conversion, data subscriptions and market impact. Then test whether the result survives doubling the trading costs.
How long should I paper trade?
For a period fixed before you start, long enough to include a range of conditions. The value is not the profit and loss but the divergence: comparing live signals against backtest expectations exposes data feed differences, timing assumptions and fill quality that simulation hides.
What is a kill switch and why must it be separate?
A mechanism that halts trading and cancels open orders immediately. It must work independently of the strategy code, because the situations requiring it often involve that code malfunctioning. Under UK-onshored RTS 6 article 12, firms engaged in algorithmic trading are required to have this capability.
How much capital should I start with?
A fraction small enough that losing all of it changes nothing about your finances. There is no universal minimum, but fixed trading costs consume a larger share of a small account, so the practical floor is where costs stop dominating expected returns.
When should I increase position size?
Only after a stage has run for a predefined period and live results match backtest expectations within a tolerance you set in advance. Increasing size because recent results were good is the most common way a working system becomes a large loss.
What are the signs a strategy should be retired?
Live results diverging persistently from the backtest, a drawdown beyond the limit you set at step one, or the market condition the strategy relied on ceasing to exist. Define these before deployment; deciding afterwards means deciding while losing money.
Should an LLM be allowed to place trades directly?
Generally no. A safer architecture uses a language model for research, code generation or interpretation, with deterministic risk controls independently validating every order before it reaches the broker. A language model is not a validated forecasting system by default.
How often should a model be retrained?
On a documented schedule decided in advance, not in reaction to losses. Retraining after a drawdown fits the model to the most recent noise. Whatever cadence you choose, your testing cadence should match it — a model that retrains monthly is not validated by an annual review.
What is the most common mistake?
Iterating against the test period. Each time you look at held-out data and adjust in response, that data joins the training set. After several rounds there is no out-of-sample evidence left, only a result that has been fitted to everything available.
How long does this whole process take?
Longer than the modelling. Data preparation, cost modelling, validation, paper trading and staged deployment dominate the timeline, and nine of the twelve steps here happen before any capital is at risk. A strategy taken from idea to full size in a week has skipped the parts that matter.
Compiled by AI Trading Book Editorial. The protocol synthesises validation methods from the peer-reviewed literature cited below with the operational controls required under UK-onshored RTS 6 and proposed in ASIC CP 386. The nine-of-twelve figure is our own count of this protocol, labelled as such rather than presented as an industry statistic. This is a methodology guide; it is not a recommendation to trade. Published 27 August 2026; last updated 28 August 2026. Corrections are logged on the corrections page.
Sources
- UK-onshored MiFID II RTS 6 — article 6 conformance testing, article 9 annual self-assessment, article 10 stress testing, article 12 kill functionality.
- ASIC — CP 386, 27 August 2025 — proposed controls enabling immediate suspension of a Trading Algorithm.
- Harvey, Liu and Zhu, Review of Financial Studies 29(1), January 2016 — recommended t-statistic threshold above 3.0.
- Bailey and López de Prado — Deflated Sharpe Ratio, 2014; Probability of Backtest Overfitting.
- Bailey, Borwein, López de Prado and Zhu, Notices of the AMS, 2014 — on the reliability of published backtests.
- Buczynski, Cuzzolin and Sahakian, International Journal of Data Science and Analytics 11(3), April 2021 — review of 27 machine-learning equity experiments.
- Knight Capital, 1 August 2012 — approximately USD 440 million lost in about 45 minutes; 4,026,087 executions across 154 securities; SEC penalty of USD 12 million under the Market Access Rule.
Informational research only. Nothing on this page is personal financial, legal, tax or investment advice, or a recommendation to trade any instrument or to build any system. Automated trading can lose money rapidly, and following this protocol does not make a strategy profitable.