Turn behaviour into metrics

Metric quality and confidence

A metric is useful only if the team can understand it, trust it, and use it.

Metric quality is not just about whether the calculation is technically correct. A metric also needs a clear purpose, a clear definition, reliable event data, and enough context to support a decision.

Confidence does not mean certainty. It means the team has enough trust in the metric to use it responsibly.

Confidence checklist

A team can have more confidence in a metric when:

the product question is clear
the metric definition is documented
source events are reliable
the formula is visible
population, filters, and exclusions are clear
useful dimensions are available
known limitations are documented
the metric is used for an appropriate decision

Confidence is not certainty

A metric can be useful without being perfect. The problem is false precision: when a number looks exact but hides weak definitions, missing events, unreliable properties, or important caveats.

For example, reporting “registration completion is 64.2%” may look precise. But if completion could mean form submission, email verification, or first product access, the precision is misleading.

A good metric has a clear purpose

A metric should answer a product question.

For example, “registration completion rate” is only useful if the team knows what it is trying to understand:

are users able to create an account?
are users reaching the product after registration?
did a change improve the registration workflow?
are some entry points performing worse than others?

The question gives the metric a job. Without it, the metric may be reported often but used rarely.

A good metric has a clear definition

A metric name is not a definition.

For example, “registration completion rate” could mean:

users who submitted the form divided by users who viewed the form
users who verified their email divided by users who submitted the form
users who accessed the product divided by users who started registration

Each version answers a different question.

A useful metric definition should explain:

what the metric measures
which events are used
how it is calculated
which users or sessions are included
which filters or exclusions apply
what decision it supports

Without this context, teams may use the same metric name while meaning different things.

A good metric is built on reliable evidence

Metrics inherit the quality of the events beneath them.

A metric becomes harder to trust when the underlying events are:

missing
duplicated
inconsistently named
captured at the wrong moment
missing useful properties
interpreted differently by different teams

For example, if registration.completed fires when the form is submitted, the metric will say something different from one based on email verification or first product access.

The event logic shapes the metric.

A good metric is interpretable

A metric should help the team understand behaviour, not hide it.

A completion rate may tell the team that fewer users are finishing registration. It does not explain whether the problem is form abandonment, validation errors, email verification, first product access, or a change in the type of users entering the workflow.

A useful metric can be traced back to the workflow:

Workflow:
Register for an account

Events:
registration.form_viewed
registration.form_submitted
registration.email_verified
registration.completed

Metric:
Registration completion rate

Decision:
Where should we improve the registration experience?

If the team cannot trace a metric back to behaviour, the metric is harder to interpret.

Confidence is more useful than false precision

Product measurement rarely removes uncertainty.

Data can be incomplete. Events can be imperfect. User behaviour can change. Context may be missing. A precise number can still be misleading if the team does not understand how it was produced.

Confidence comes from knowing enough about the metric to use it carefully:

what it measures
what it does not measure
how reliable the underlying events are
which assumptions sit behind the calculation
where the metric may be incomplete or misleading

A metric does not need to be perfect to be useful. It needs to be good enough for the decision it supports.

Common signs of low-quality metrics

A metric may need review if:

nobody knows what question it answers
different teams define it differently
the source events are unclear
the calculation logic is undocumented
the metric changes but nobody knows why
the metric cannot be traced back to user behaviour
the metric appears on a dashboard but does not support a decision

These are signs of weak measurement design, not just reporting issues.

How to apply this

Before using a metric to support a decision, ask:

what question does this metric answer?
how is it calculated?
which events produce it?
what behaviour does it describe?
what context is missing?
what could make it misleading?
are we confident enough to act on it?

These questions help teams improve metric quality before the metric becomes part of reporting, prioritisation, or performance review.

Metric catalogue template
Measurement review checklist
Assessing measurement maturity

Key takeaway

Metric quality is about usefulness, not just correctness.

A good metric has a clear purpose, a clear definition, reliable evidence, and enough context to support a decision. Confidence comes from knowing what the metric can and cannot tell you.