Measure validated value, not generated output

Code, commits, completed tasks and agent runtime are measures of activity. They are not measures of product progress.

The problem

When implementation can be automated, volume becomes particularly misleading. An agent can generate large amounts of working code while making a product more complicated, increasing operating costs or implementing something customers do not want.

Even a correct implementation can be the wrong investment. Counting output rewards the wrong behaviour.

What to measure instead

Start with the outcome the product needs to achieve: a user can complete an important task, a failure rate is reduced, a workflow becomes faster, or a business assumption is either confirmed or rejected. Technical outcomes should support these goals: correct behaviour, compatibility, recoverability, performance and maintainability.

Consider the whole cost of a decision: implementation, review, tools, compute or token use, integration, rework, operations and the cost of carrying unnecessary complexity. Small token costs do not imply small delivery costs; nor do high token costs necessarily indicate useful progress.

Some progress is learning that removes scope. A feature not built can be a good outcome if the decision is supported by evidence and avoids waste.

Illustrative example

A team delivers a polished automated classification feature, but product trials show users still do not know who owns their requests. More code and higher test counts would make a poor success story. A better outcome is to cut the classification feature, make ownership explicit and retest the workflow.

Practical questions

• What evidence would show that this change matters to its users?
• What does “done” mean beyond a merged branch?
• Can we tell whether the architecture became harder to change?
• Which costs are hidden in repeated agent retries, review or future maintenance?
• What did we decide not to build, and why?

Limits

Not all value is immediately measurable. Reliability work, security improvements and foundational architecture may deliver benefits over time. Use leading indicators and explicit risks rather than pretending every decision has a precise short-term ROI. Avoid metrics that incentivise agents or teams to produce activity for its own sake.

See the worked example: learning and feature removal, value scorecard, release checks.