Evaluating an investment research provider

Buy an answerable research function

Specify the research gap: security coverage, macro interpretation, risk monitoring, or model-generated candidate selection. Identify who will read the output and what committee decision it could change. A broad stream of interesting commentary may not solve a narrow analytical need. Conversely, a precise signal can be inappropriate when the office needs an explanation of assumptions rather than another suggested trade.

CFA Institute's diligence guidance asks members and candidates to consider the soundness of third-party research, including assumptions, rigor, timeliness, and objectivity. Convert that into an office-specific request list. Ask for the methodology, sample outputs, historical archive, correction process, and intended audience. The purpose is to establish a reasonable basis for reliance, not to obtain a promise of superior future returns.

Request a complete sample rather than selected successes

Choose a continuous historical period and request all scheduled publications, including unchanged views, withdrawn recommendations, and acknowledged errors. Identify whether timestamps show first release or later reconstruction. Ask how the provider treats missed issues and revisions. A curated set of winning calls cannot establish the consistency of a service or the experience a subscriber would have had.

In a hypothetical provider review, the office selects a quarter before examining its recommendations. The provider supplies the full archive except for several missing releases. The analyst records the gap, seeks original delivery evidence, and limits any conclusions drawn from the sample. Replacing missing items with newly written explanations would not recreate the information available to a subscriber during that quarter.

Compare claims with the artifacts that support them

The following original table is a questioning framework, not a provider ranking. Ask for the evidence underlying each claim and record whether it was received, independently checked, or remains unavailable. Avoid combining these different judgments into a precise-looking numerical score whose weighting cannot be defended.

Challenge the research culture and incentives

Ask who can reject a finding, how negative studies are retained, and how the provider controls repeated testing. Man Group's overfitting discussion makes research incentives and the treatment of unsuccessful work relevant questions. A provider need not disclose proprietary code to explain its basic validation architecture, candidate-search discipline, and handling of evidence that contradicts a preferred conclusion.

Map commercial incentives without assuming that every conflict invalidates the research. Ask about sponsored coverage, affiliate relationships, model-linked products, personal dealing policies, and how disclosures reach subscribers. Distinguish a documented mitigation from a general assertion of independence. If the provider's research changes after an investment product launch, ask for the analytical reason and preserve the timeline rather than inferring either innocence or wrongdoing from timing alone.

Run a trial that measures usefulness and friction

Predeclare a trial budget, review date, intended readers, and the questions to evaluate. Track delivery timeliness, clarity, unanswered inquiries, corrections, and analyst time spent integrating outputs. A hypothetical trial might ask two reviewers to independently explain the same recommendation's assumptions and limitations. If they reach incompatible interpretations, investigate whether the report is ambiguous or the office lacks the necessary expertise.

Do not treat a few profitable hypothetical recommendations as proof of skill. Record the full decision chain: study result, live publication, internal interpretation, and any separately authorized execution. Compare with the office's existing research process using the same information dates. The relevant benefit may be better challenge, broader coverage, or faster analysis rather than a measurable return difference during a short trial.

Decide on reliance, not just subscription renewal

Approve the permitted reliance level: background reading, a documented research input, or a model requiring a separate validation process. Confirm licensing for committee members and external advisers, cancellation mechanics, and post-termination access to decision evidence. Subscription fees are only part of the cost; include the office's review and integration effort. SEC fee education is a useful reminder to inspect costs that are absent from a headline price.

Failure cases include unexplained archive gaps, refusal to distinguish hypothetical from actual results, and recurring corrections without a durable history. Not every weakness requires rejection: limited coverage can be acceptable for a limited role. The committee should state what it is buying, what it is not relying on, and what evidence would justify expansion, restriction, or termination. Keep that decision separate from any manager or investment-product approval.

Sources and further reading

1. CFA Institute: Standard V(A), Diligence and Reasonable Basis
2. Man Group: Overfitting and Its Impact on the Investor
3. SEC Investor.gov: How Fees and Expenses Affect Your Investment Portfolio

The linked sources provide conceptual background. The worked examples and planning assumptions on this page are this publication's own illustrations.

Continue the research