The eight checks, explained
· 6 min read
Every strategy built in Mithos goes through the same eight checks, with no setting to turn them off. Here is what each one is looking for.
Why there is no off switch
The absence of a toggle matters more than the list itself. A check you can disable is a check that gets disabled on the day it says something you didn't want to hear — usually the day it would have been worth most.
So the same eight run on every strategy, in the same order, whether you are on a free account or a paid one. Here is what each is looking for.
- Walk-forward validation — the strategy is rebuilt and tested across successive held-back windows, checking that the result survives forward in time on data it did not see during fitting.
- Multiple-testing penalty — the bar rises with the number of configurations tried, so searching until one result looks good does not count as finding an edge.
- Cost floor — a declared minimum trading cost is applied before the result is judged, so an apparent edge that disappears after realistic costs cannot pass.
- Concentration — return dispersion is checked to make sure the result is not being carried by one or two lucky trades.
- Sample — there must be enough out-of-sample trades to support a reliable statistical conclusion; otherwise the verdict stays undecided.
- Placebo — the strategy is compared against random entries with the same risk, costs, and exits to check that its entry signal is actually doing the work.
- Data source — the evidence is checked for genuine instrument data; a result reached only on stand-in data is marked as proxy and cannot be armed.
- Worst-session veto — a catastrophic single session can veto the strategy even when its average result passes the other checks.
Why they run together
Individually, none of these checks is exotic; several are standard practice that simply gets skipped under time pressure. Their value is in running as a set, on every strategy, in the same order, so results stay comparable and no single flattering number can carry an idea on its own.
A strategy can survive seven of them and still be rejected by the eighth. That is the intended behaviour: the harness reports what the evidence supports, not the best case you could assemble from it.
Where to read more
How it works shows the checks in the context of the full build → backtest → rigor → verdict sequence, and what a rejected strategy actually tells you covers how to read the outcome.
Put your own idea through it
A free account includes building, backtesting and three rigor checks. No card, and no broker needed for research.
Start free