Measurement plan
A measurement plan defines the business result, the eligible population, the comparison, the observations, and the interpretation limits. It asks whether the proposed solution improves the business process. A model evaluation answers a narrower question about application behaviour and does not replace this.
- Enterprise
- Prepared by
- Date
- Version
Source: article three, "Define how the business result will become visible", "Explain value for the complete screening process", and "Match costs and benefits to the same decision".
Who fills this in, and what it decides
The FDE lead writes it with the business workflow owner and the finance partner, alongside the delivery plan and before behaviour changes. Late agreement on definitions creates avoidable disputes about the result, so section 9 records who agreed and when. The plan feeds the second commitment (does the limited workflow help against the agreed comparison), the release decision, and the value account the practice lead brings to the portfolio review.
Roles named in this document:
- FDE lead
- owns the plan and the evidence it produces.
- Business workflow owner
- owns the definition of the result and confirms the eligible population.
- Finance partner
- agrees the comparison, the value claim, and the cost allocation.
- Service owner
- collects the operating observations after release.
- Practice lead
- uses the result in the portfolio and practice reviews.
The proposed solution is the change being tested. The workflow is the business process around it, and the plan measures the workflow, not the solution's output.
1. The result, and whose decision it supports
Leaders may need evidence of lower effort, faster decisions, more complete source coverage, or increased capacity. These support different decisions, so name the one that supports the next funding or release decision. Source coverage means whether the output includes the required authoritative sources for an eligible case.
- Intended result
- Who needs it, and for which decision
- If the purpose is an urgent control improvement rather than productivity, say so here
- Date agreed with the finance partner
If the solution addresses an observed exposure, state that purpose directly. Do not invent a savings claim to justify a different benefit.
2. The eligible population
The business workflow owner defines eligibility; the FDE lead records it. In the screening specimen, the characteristics that decide eligibility are sector, transaction type, source availability, target complexity, and reviewer seniority. The specimen is hypothetical.
- Characteristics that define eligibility
- Cases excluded, and why
- Observation period
The boundary must cover the complete workflow: collection, preparation, correction, review, clarification, and manual fallback.
3. The comparison
Attribution means deciding how much of an observed change is associated with the proposed solution rather than with other changes. It requires a comparison with the process that would otherwise operate: one group of eligible cases completed with the solution, and a comparable group completed through the existing process. Sector, complexity, source availability, workload, and reviewer experience can all affect the result.
Choose the strongest method that is practical. The FDE lead proposes; the finance partner agrees.
| Method | What it does | When it is available | Chosen |
|---|---|---|---|
| Random allocation | Assigns eligible cases to the solution or the existing process by chance | Both routes are acceptable, assignment does not breach controls, and the business can tolerate the scheduling constraint | |
| Phased introduction | Starts with one defined group before another | The groups are comparable and the sequence is not chosen by difficulty | |
| Matched concurrent cases | Compares cases with similar characteristics | The matching variables are recorded and available | |
| Prospective baseline | Records the existing process before the solution starts | There is time to measure before release |
- Method chosen
- Remaining uncertainty it leaves
- Whether historical records are sufficient, and what they omit
Historical records often omit analyst effort, failed attempts, manual fallback, and case complexity. If they do, prospective measurement before release is needed, because the missing variables weaken attribution.
4. What gets recorded
Every observation answers a management question. Measuring only successful outputs answers a narrower question about the solution's best cases; failed, abandoned, and unused attempts belong in the result when the claim concerns the complete process. The service owner collects these after release; the FDE lead collects them during the limited workflow.
| Observation | Question it answers | Definition | Collected how | Owner |
|---|---|---|---|---|
| Preparation effort per case | Did the solution reduce the work before review? | |||
| Review effort per case | Did it move work to the reviewer instead? | |||
| Clarification requests per case | Does the output leave the reviewer with questions? | |||
| Correction effort per case | How much of the output must be fixed? | |||
| Completion time | Did the decision arrive sooner, including waiting? | |||
| Unresolved exception rate | Does the workflow still route the hard cases? | |||
| Cases that did not use the solution | What share of eligible work did it not touch? | |||
| Cases that returned to the manual process | Where does the process reject the output? |
5. Behaviour changes to watch for
These can make the comparison groups represent different work. Record what changed at the same time as the solution, so the comparison can be read honestly.
- Difficult cases reserved for manual work, evidence
- A new template or method spread outside the eligible group, evidence
- Staffing or workload changed during the observation period, evidence
- Anything else that changed at the same time
6. The value claim
Each claim needs different evidence. Record which one is being made, and do not convert between them. The finance partner confirms the claim type.
| Claim | Evidence it requires | Made here |
|---|---|---|
| Released capacity | A measured reduction in effort across the complete workflow | yesno |
| Cash saving | Evidence of reduced expenditure: a cancelled service or a lower commitment, subject to timing and offsetting costs | yesno |
| Avoided cost | A credible plan or demand forecast for the expense. An expense cannot become a saving when nobody intended to incur it | yesno |
| Revenue or quality | Its own causal explanation, and defined observed errors | yesno |
Pricing released time at a salary rate describes an economic estimate, not an observed cash saving. If the comparison already covers all eligible cases, it already reflects non-use and manual fallback, so do not apply a second adoption discount. If the effort measure already contains review and correction, do not subtract them again.
For a comparable case category, the relationship is: net human effort change per case equals comparison effort per case minus observed effort per case. Category results combine only through an explicit case mix; otherwise a shift towards simpler cases looks like an improvement.
- How effort is recorded
- Case categories used, and the case mix applied
7. Costs matched to the same decision
Costs and benefits must cover the same period, population, and responsibility. Comparing a mature service's recurring benefit with a short pilot's partial cost overstates the case; assigning all platform costs to one engagement distorts it the other way. The finance partner agrees the allocation method and it stays consistent across comparisons.
| Cost | This period | Allocation method |
|---|---|---|
| Discovery and engineering | ||
| Domain review | ||
| Integration | ||
| Evaluation | ||
| Infrastructure and model usage | ||
| Change management and support | ||
| Shared services used |
- Period, population, and responsibility covered by both the costs and the benefits
- Historical expenditure (what was spent to reach the current result)
- Future commitment (what continuing, expanding, or retiring would cost)
Historical cost does not justify further spending. The next decision depends on what additional work can achieve.
8. Interpretation limits
- What this design cannot establish
- The assumptions a forecast depends on, and which one would change the decision
- The FDE practice's contribution, separated from model, platform, process, and vendor
The full outcome cannot automatically belong to the FDE practice. Assessing the practice needs a credible account of how its delivery arrangement compares with alternatives for similar work.
9. Agreement
| Role | Name | Agrees the definitions above | Date |
|---|---|---|---|
| Business workflow owner | |||
| Finance partner | |||
| FDE lead | |||
| Practice lead |