What to Measure Instead of Number of Studies Completed

Study count is the default metric for research programmes because it is easy to track. It is also useless for evaluating whether the programme is worth the investment. Here are the metrics that actually matter.

Decision Influence Rate

Of the studies completed in a given period, what percentage directly informed a documented decision? This is the single most important metric.

Ask your team to produce this number. If they cannot, decisions are not being tracked.

Finding Reuse Rate

Of the studies completed, how many had their findings referenced in a subsequent study, experiment brief, product decision, or strategy document?

Ask your team: "How many times have findings from a past study been cited in new work this quarter?"

Knowledge Retrieval Time

How long does it take someone outside the research team to find everything the programme knows about a specific topic?

Ask someone outside the research team to find all research related to [specific product area] from the last two years. Time it.

Stakeholder Satisfaction with Actionability

After each study, ask the commissioning stakeholder: "Did this research give you what you needed to make a decision?"

Ask your team: "Do we collect any feedback from stakeholders on whether our research was actionable?"

Research-to-Decision Latency

How long between a study concluding and a related decision being made?

Ask your team: "For the last five studies that had a relevant product decision in the pipeline, did the findings arrive before or after the decision was made?"

Duplication Rate

What percentage of studies conducted substantially overlap with previous studies?

Ask your team: "How confident are you that no study we are currently running has been done before in some form?"

How to Use These Metrics

Start with decision influence rate and knowledge retrieval time. If your team struggles to produce any of these numbers, that is the finding. It is an infrastructure problem.