The decision in brief
Define the question, evaluation period, admissible evidence and allowed use before receiving new results. A research trial can assess documentation, delivery and relevance without granting a provider discretion or treating published model performance as the family's executable return.
| Decision | Illustrative rule | Evidence to retain |
|---|---|---|
| Research question | Does the output add usable context to an existing review? | A written use case |
| Access | Research only; no execution authority | Current terms and permission record |
| Evidence | Time-stamped outputs and documented methodology | Unedited sample and version dates |
| Review | Predeclared committee review | Decision memo and open questions |
| Portfolio action | Separate approval if later proposed | Mandate and implementation review |
Start with a bounded question
Assume an office wants an external research input for an existing portfolio review. It is not looking to outsource custody, discretionary management or execution. The committee writes the question before choosing a provider: what additional observation would make its existing process better, and how would it know the observation was available at the time?
The case is illustrative and names no real client or paid trial. It avoids an invented performance target. A short evaluation cannot establish the reliability of a strategy across every market condition, but it can reveal whether the research is documented, delivered as described and relevant to the stated question.
Request a useful evidence pack
Ask for the output's definition, data timing, revision policy, model version, intended horizon and a sample in the promised format. A result chart should state whether it represents a backtest, a model record, an account or an aggregate. Ask what costs, constraints and missing observations are included.
Retain the sample as received. A changing webpage is not a substitute for a dated evidence record. Material differences between a marketing summary and the underlying method belong in the questions log, with a specific request for clarification rather than an unsupported conclusion about quality.
Separate evaluation from authority
Buying access does not by itself authorise a provider to trade, change allocation or contact a custodian. The office should record who can read the output, who can interpret it for the committee and who can approve any action. The actual licence and mandate determine the permissions; this example does not describe a legal arrangement.
If integration is proposed, test delivery and interpretation before relying on an automated process. Check timestamps, unavailable values and the response to an interrupted feed. A useful research observation can still be unsuitable for a particular operational workflow.
Review the process before the recent result
A favourable observation during the trial can be interesting without proving the model has an edge. Equally, one disappointing outcome does not explain the model's long-run behaviour. Keep the predeclared review criteria visible: documentation quality, correspondence between described and delivered outputs, relevance to the decision and evidence of timely publication.
Where investment performance is part of the eventual assessment, design a separate comparison with appropriate benchmarks, costs and a sufficiently relevant sample. Do not choose a flattering start date after seeing outcomes or rank competing research methods solely on a short convenient period.
Write the next decision
The committee can approve continued research use, request missing evidence, change the use case or stop the evaluation. Record the reasoning and its limits. An approval to read a research product is not approval to allocate capital to every output.
A later implementation proposal should address sizing, overlap with existing exposures, liquidity, costs, authority and monitoring. The strongest outcome of the initial trial may be a clearer understanding of what the model can and cannot contribute, rather than a new performance claim.
Decision points
- Specify the research use case before reviewing fresh results.
- Keep research access, implementation decisions and execution authority distinct.
- Retain dated evidence and a decision record for the next review.
Test a different assumption
The tools use explicit hypothetical inputs. Change the spending, liquidity or portfolio assumptions to examine the arithmetic behind your own illustration.
Sources and further reading
The linked sources provide conceptual background. The worked examples and planning assumptions on this page are this publication's own illustrations.