Backtests, live research, and actual investment results

Give every series an evidence identity

A historical study applies a rule to earlier observations. A live research publication records what was released at a particular time. An actual investment result requires the relevant account holdings, cash flows, costs, and transaction evidence. Publishing a hypothetical return each day does not turn it into a funded-account track record. Preserve these classifications even when the same model generates all three series.

The SEC's performance bulletin explicitly treats backtested results as hypothetical. AQR's data-library documentation provides another useful distinction: its academic portfolio series are not the returns of its live products. These source-specific statements illustrate why a familiar provider name or a current update date cannot substitute for identifying the record being evaluated.

Establish dates that cannot be rewritten later

Record the study cutoff, rule-freeze date, first archived publication, first eligible investment date, and dates of material changes. Request the original publications or durable delivery records, not a newly assembled historical spreadsheet. A reconstructed archive may still help with analysis, but its reconstruction date and missing evidence should remain visible. A timestamp without the corresponding content is insufficient to reconstruct a recommendation.

In a hypothetical review, a model launches in July with a study ending in June. During August the provider changes its exclusion rule. Maintain separate July-version and August-version histories and describe the transition. Extending the new rule backward creates a new study, not a replacement for what the original July research actually said. The same principle applies when a correction is economically justified.

Use a comparison that respects the evidence boundary

This original table separates questions that committees often combine. A research series can demonstrate publication consistency without demonstrating executable prices. An account can demonstrate actual transactions without establishing that its results represent every subscriber or another family's implementation. Ask whose evidence is being presented and which decisions it covers.

Build the bridge from reference result to account outcome

Align overlapping dates, currencies, income treatment, and exposure before interpreting a performance gap. Then isolate costs, implementation delay, position sizing, cash balances, restrictions, financing, and discretionary overrides. Preserve the study's frozen hypothetical reference convention. Evaluate executable alternatives in a separately labeled analysis rather than silently changing the historical convention to make a live comparison look cleaner.

In a hypothetical publication, a research entry reference is 100 while the office's actual execution is 101. The reference remains a research observation; the account return begins with its own execution and includes applicable costs. If no order was sent, report no account transaction. Do not create a notional fill in the execution ledger simply because the research reference appears in an archive.

Make performance comparisons economically consistent

Specify whether returns are gross or net and itemize what net includes. Research subscription costs, vehicle charges, brokerage, financing, and account-specific taxes need not appear in the same reporting layer, but the committee should understand where each is reflected. A benchmark may omit investable-product expenses. Its return type, currency, reinvestment convention, and valuation times should match the comparison's purpose.

Also check whether deposits and withdrawals distort the comparison. Time-weighted and money-weighted returns answer different questions, especially when a family adds capital during the observation period. Ask for the calculation method rather than subtracting two headline percentages. A missing or unexplained reconciliation residual should remain visible until resolved; it is not automatically investment skill or implementation slippage.

Interpret short records without promoting them prematurely

Early observations are valuable for identifying late publications, incorrect inputs, and inconsistent implementation. They may provide little statistical information about rare losses or persistent skill. Man Group's discussion notes that a short trading test can reveal problems without establishing significance. Set an observation purpose and review date before outcomes arrive, and retain losing periods and inactive recommendations.

Reject blended charts that erase the boundary between backtest and forward history. Investigate a provider who changes inception dates, drops unavailable publications, or calls simulated results actual trades. A legitimate gap can have an ordinary explanation, including cash or timing differences, so document the bridge before making accusations. The final committee record should state exactly which history supports the proposed use and which claims remain unproven.

Sources and further reading

1. SEC Investor.gov: Investor Bulletin, Performance Claims
2. AQR: About the AQR Data Library
3. Man Group: Overfitting and Its Impact on the Investor

The linked sources provide conceptual background. The worked examples and planning assumptions on this page are this publication's own illustrations.

Continue the research