Week 37 Day 4: Why Vanity Metrics Destroy Focus

Page views are a classic vanity metric. 'We had 10 million page views this month' sounds impressive. But if those 10 million views did not lead to sign-ups, purchases, or engagement, the number is meaningless. The team that chases page views optimizes for traffic. The team that chases conversions optimizes for value. Both teams work equally hard. One team creates business impact.

Here is how vanity metrics destroy focus. The mechanism is indirect but powerful. When a vanity metric is the team's primary measure of success, the team unconsciously optimizes for it. Every prioritization decision tilts toward 'what will move the number' -- the vanity number. Over weeks and months, the team's work drifts away from actual business value and toward the metric. The drift is invisible because the metric keeps improving. Everyone feels productive because the number goes up. But the business outcomes stagnate or decline because the team's effort is misallocated. Here are five common vanity metrics and their value-oriented alternatives. Vanity: lines of code written. Value alternative: features shipped to production that customers used. Vanity: tickets closed. Value alternative: customer issues resolved without recurrence (tickets that close once and stay closed). Vanity: uptime percentage without context. Value alternative: uptime during peak revenue hours weighted by revenue impact. Vanity: number of experiments run. Value alternative: number of experiments that produced a statistically significant positive result that was implemented. Vanity: team velocity (story points per sprint). Value alternative: lead time (time from idea to customer delivery). The practical fix is not to eliminate vanity metrics -- some of them are useful operational indicators. The fix is to never use a vanity metric as the team's primary success measure. The primary measure should always be an outcome metric connected to business value. I made this mistake for two quarters. My team's primary metric was deployment frequency. We optimized aggressively -- deploying 15 times per week, up from 4. The metric looked fantastic in our quarterly review. But our customer satisfaction scores dropped 8% during the same period because we were deploying faster without improving quality. We had optimized for the easy metric (deployments) at the expense of the important metric (customer experience). When I switched the primary metric to customer-reported defects, the team's behavior changed within two sprints. Deployment frequency naturally settled at 8 per week -- slower than our peak but with dramatically better quality. The outcome metric corrected what the vanity metric had distorted.

Vanity metrics are a specific instance of what Goodhart (1975) formalized as 'Goodhart's Law': when a measure becomes a target, it ceases to be a good measure. The law predicts that any metric used as a performance target will be optimized by the people being measured, and the optimization will exploit whatever gap exists between the metric and the actual outcome the metric is meant to represent. Research by Muller (2018) on 'the tyranny of metrics' documents extensive cases where metric-driven optimization produced perverse outcomes: hospitals measured on wait times reclassified waiting areas to exclude patients from the metric, schools measured on test scores narrowed their curriculum to test preparation at the expense of education, and police departments measured on crime rates reclassified crimes to reduce the reported numbers. The deployment frequency example is consistent with what Forsgren, Humble, and Kim (2018) found in their DORA (DevOps Research and Assessment) research: deployment frequency alone was not correlated with organizational performance. Only when deployment frequency was combined with quality metrics (change failure rate, time to restore service) did the combined measure predict performance. Organizations that optimized for frequency alone showed higher failure rates and longer recovery times -- a net decrease in delivery performance despite the increase in the individual metric.