The decision in brief
Specify the model's intended role, screen for hard constraints, compare evidence on a common basis, and run a bounded observation exercise. Approve research use separately from capital deployment, with an accountable decision owner and explicit reasons to stop.
Define the decision before inviting proposals
Write a short mandate stating the decision frequency, assets covered, decision maker, and acceptable output. An allocation model might inform quarterly policy discussions; a security model might supply candidates for further analysis. Neither automatically determines position sizes, custody arrangements, or execution authority. Name those responsibilities separately so that a useful analytical tool is not assigned an untested portfolio-management job.
CFA Institute's manager-selection framework starts after objectives and allocation have been established. UBS likewise places the investment policy statement at the center of family investment governance. Apply that sequencing here: record cash commitments, spending needs, restricted holdings, and the role of the operating business before comparing models. The model should serve these constraints, not redefine them to improve its presentation.
Turn household constraints into rejection gates
Separate requirements that cannot be traded away from preferences that can. A model needing short positions is unsuitable for a long-only mandate unless a different implementation is independently evaluated. A strategy dependent on frequent turnover may conflict with concentrated legacy positions or account-specific tax considerations. Ask the implementation team to identify these mismatches before anyone calculates a weighted score.
In a hypothetical family office, quarterly distributions and possible business capital calls require a protected liquidity reserve. A model recommending fully invested equities cannot govern that reserve merely because its study shows an attractive average return. The committee could restrict it to an eligible sleeve, evaluate it as commentary only, or reject it. Each choice changes the decision being assessed.
Compare tools by the job they can actually perform
The following comparison is an original committee worksheet, not a rating of any provider. Request evidence for the proposed use rather than assuming that one type of model is more sophisticated. A more detailed output can create more operational obligations without making the underlying forecast more reliable.
| Proposed role | Evidence to request | Common mismatch |
|---|---|---|
| Strategic allocation input | Assumptions, horizon, liabilities, sensitivity ranges | Short-term signals presented as long-term policy |
| Security research screen | Universe history, exclusions, dated outputs | Rankings mistaken for executable orders |
| Risk overlay | Exposure mapping, stress losses, intervention rules | A hedge that introduces an unbudgeted cash demand |
| Managed implementation | Account records, authority, costs, reconciliation | Research references substituted for investment executions |
Build a common evidence sheet
Request the same fields from each candidate: model identity, first publication date, study periods, data availability assumptions, benchmark, return convention, costs, and revision history. The SEC's performance-claims bulletin explains why calculation choices and selective periods matter. Keep historical simulations, timestamped live research publications, and actual funded-account results in separate columns; never splice them into an unlabeled track record.
Ask what the model knows at its decision time and what remains an estimate. Document whether reported returns include cash, income, currency conversion, financing, and fees. An unavailable field is a research gap, not necessarily evidence of misconduct, but the gap limits the decision the committee can defend. Record the provider's explanation and the remaining uncertainty rather than filling blanks with favorable assumptions.
Observe a frozen version against a simple alternative
Set an observation budget in advance: staff hours, subscription spending, eligible outputs, and a review date. Preserve every scheduled publication, including neutral recommendations and missing releases. Compare the model with a suitable simple policy using the same available information. This exercise can reveal delivery problems and decision friction; a short observation period rarely establishes durable investment skill.
In a hypothetical quarterly allocation trial, analysts record whether a frozen model would have changed an existing committee recommendation and why. They separately estimate implementation costs without relabeling those estimates as fills. If analysts repeatedly override the model, investigate whether its mandate is wrong, its outputs are unusable, or the committee lacks a consistent decision protocol. Do not simply discard inconvenient observations.
Approve a bounded role and specify failure conditions
The final decision should name the permitted use, version, portfolio sleeve, accountable owner, and next review. Include the counterfactual: what happens without the model? Declining an expensive tool can be rational when a simpler process addresses the same need. Conditional approval should list unresolved items and prevent expansion until those conditions are satisfied.
Useful failure cases include an unexplained methodology change, inability to reproduce a key output, costs exceeding the approved use case, or exposure that duplicates the family's business risk. Distinguish these from an expected losing period. The committee is evaluating a decision process under uncertainty, not purchasing a guaranteed outcome. Reassessment should preserve the original rationale so later results do not rewrite the selection history.
Decision points
- Choose the portfolio role and hard constraints before comparing historical performance.
- Keep study, live publication, and actual execution evidence separate.
- Approve a defined use with an owner, review date, and explicit failure conditions.
Sources and further reading
- CFA Institute: Investment Manager Selection
- UBS: The role of investment policy statements in family wealth
- SEC Investor.gov: Investor Bulletin, Performance Claims
The linked sources provide conceptual background. The worked examples and planning assumptions on this page are this publication's own illustrations.