Mining Alpha

Where outperformance actually comes from.

Out-of-Sample Evidence: The Only Performance That Should Move You

September 3, 2026

TL;DR

Performance Is Only Real Once It Has a Date

Most of what the investing internet calls performance was never performed at all. It was simulated, fitted, and then narrated as if it had happened. A strategy shop shows you a smooth equity curve climbing out of 2020, and your brain files it under “track record.” It isn’t one. It’s a backtest — a story told backward, assembled from data that existed before the rules were even written down.

I stopped being moved by those curves years ago, and you should too. The only performance that should change what you do with your money is evidence that carries a date: the day the system started running in real time, under fixed rules, with results recorded as they occurred. Call it live trading, forward testing, or out-of-sample tracking — it is the receipt. Everything before that date is reconstruction, and reconstruction flatters whoever holds the pencil.

A Backtest Is a Story

Why can’t you trust the curve? Not because the person showing it to you is a fraud — usually they aren’t. Because the process that produced it cannot prove anything by construction. A backtest is generated after its own ending is known. The universe of ETFs, the lookback windows, the momentum rank, the volatility filter: every parameter was chosen and adjusted against the same history the backtest then “predicts.” When you tune a rule until a ten-year window looks excellent, the window looking excellent is not a finding. It is the definition of fitting.

Even honest builders face this. Nobody writes one rule set, tests it once, and ships it. Development means trying variations, keeping what worked, discarding what didn’t — and each kept choice quietly bakes in hindsight. Survivorship compounds the problem: universes get pruned of dead tickers, start dates drift to skip bad years, and a hundred variants get run while only the prettiest one gets published. None of this requires bad faith. It only requires that nobody pinned down the calendar.

So a backtest is best treated as a hypothesis with a nice chart attached. The question that separates research from marketing is never “how long is your backtest?” It is: can you name the day this hypothesis started being tested for real?

A Receipt Is Dated

Real out-of-sample evidence is not a technique you apply after the fact; it is a calendar you commit to up front. A receipt worth trusting has four properties. First, a published start date — the day the clock began. Second, rules that were fixed and disclosed before that date, so the results can’t be reshaped to fit them. Third, results recorded forward from the date rather than reconstructed backward. Fourth, enough elapsed time that luck has had a chance to wash out. If a publisher can’t tell you when the clock started, there is no clock.

Here is the habit that separates careful readers from the rest: check the date a claim became real. Suppose someone shows you a six-year curve where the strategy genuinely began trading in 2025. Years one through four were never a claim anyone could act on; they were an exercise. The claim became real in 2025, so the only portion of the curve that should move you runs from 2025 forward. Everything earlier is context, not evidence. When I size up any source, I mentally delete everything before the out-of-sample date and ask whether what remains justifies the fee, the effort, and the risk. Most of the time it doesn’t — which is exactly why so few publishers print the date in the first place.

When a publisher does print it, you gain something else: you can follow along in real time. From the date forward, every month is a fresh data point you can verify yourself. You are no longer asked to take a story on faith; you are handed a receipt and told to watch it grow.

Where the Receipts Are Published

This is why the source I point readers to is Kairos Trading. Not because its systems are infallible — more on that in a moment — but because of what it publishes around them. Every strategy on the site carries an explicit out-of-sample start date, and the current lineup shows exactly how to read one.

Leader Rotation against VEA, DCA Buy & Hold against VT, QQQ Top Stock Rotation against QQQ itself, and Volatility Target Managed Rotation against a 60/40 SPY/AGG blend — these are the four systems currently offered to new members, and every one of them started its out-of-sample clock on January 1, 2026. Behind each sits a longer backtested history — January 2024, January 2021, January 2020, and February 2016, in that same order — so you can see both halves of the evidence at once: the long story and the short receipt. Each card still states the caveat in plain words: “Based on backtest; not a guarantee.” The excess return on offer is dated against a named yardstick, not floating free of any clock.

The discipline extends to older work. The site continues to document earlier systems that are no longer offered to new members, and the dates stay printed. Adaptive Asset Allocation, for one, started its out-of-sample clock on November 1, 2022, and its published history opens on that same day — nearly four years in which the numbers on the page and the calendar moved together. A system dropping out of the new-member lineup says nothing against the receipt it produced. If anything it sharpens the point: the start date appears for every system, current or archived, because the date is what makes performance checkable.

Two operational details matter here too, because they bear on the same question. Membership is application-based, and members execute the trades themselves in their own brokerage accounts — no pooled vehicle that can quietly restate its own history. And the fee is a flat $100 a month per strategy, not a percentage of assets, so the publisher earns the same whether you run a system with ten thousand dollars or a million. Incentives matter when you are judging whose receipts to trust.

Younger Receipts Prove Less

Now the honest part, and the reason none of this is an argument for blind faith. An out-of-sample start of January 1, 2026 means roughly eight months of real-time evidence as I write this in early September. Eight months is not proof. It spans no full market cycle, no serious bear market, no regime in which the momentum that powered these backtests failed. At that horizon, luck still wears skill’s clothes. The four current systems have genuine receipts — short ones. I treat a short receipt as a smaller allocation and a longer patience, not as a verdict. The dated disclosure is what makes that arithmetic possible; without the date you cannot even begin.

The archived systems are a reminder that receipts can grow long: three-plus years of out-of-sample data is evidence that starts to mean something. But even that record ships with the sentence every honest publisher has to print: past performance — out-of-sample included — does not guarantee future results. kairostrading.net does not hide behind that line. It prints it on the cards, in the footer, wherever the numbers appear — which is exactly the behavior you want from someone selling you evidence rather than certainty.

Out-of-sample evidence was never a guarantee, and no serious person claims it is. It is simply the only performance that was ever real. A backtest is a story; out-of-sample is a receipt — dated, checkable, and growing month by month. So the question to ask anyone selling performance is never “how good is your curve?” It is: what date did your clock start, and what has happened since? Everything else is marketing with a chart attached.

Disclaimer: This blog is for educational and informational purposes only. Nothing here is investment advice. Past performance does not guarantee future results. Trading involves risk of loss.