Effort: Low
What to notice
Let users inspect completion and non-completion for the same population and period.
Where it fits
- Fits
- Services communicating outcome rates
- Use it when
- Both outcomes share the same population.
- Avoid it when
- There is no comparable underlying dataset.
How to build it
- Fix the population, period, denominator and calculation.
- Derive completion and its complement from one dataset.
- Check comprehension and satisfaction separately from clicks.
Measure
- Main metric
- Correct comprehension rate
- Guardrail
- Cancellations and questions caused by misunderstanding
- How to measure
- Assign displays within comparable traffic and compare comprehension and later misunderstandings.
When it backfires
Do not call values from different periods or denominators equivalent frames.
Alternatives
Theory behind it
Framing effect
Equivalent facts or options can produce different judgments when presented in different frames.
Limited evidence
Confidence calibration
Treat reported confidence and actual correctness as different information.
Limited evidence
Evidence limits: An application of framing research; conversion effects of this two-sided display are unvalidated.