Welcome to the 8th edition of This is Not Advice—a not-advice column for premium subscribers of What Works. If you’re not already a premium subscriber, enjoy today’s excerpt or upgrade your subscription to read or listen to the full piece!
In my work as a podcast producer, I get a lot of questions about metrics.
It just so happened that a couple of weeks ago, a fairly green podcaster brought me a pile of beautiful numbers arranged carefully in rows and columns. Their question was essentially, “What does this mean?”
The answer to this particular podcaster's question was "Not much." The data they had didn’t translate into much useful information for a variety of reasons: insufficient time, lack of scale, and very little to compare it to. They were very gracious about accepting my complete non-answer. But I know it's frustrating to have this data seemingly at your fingertips without having any story to go with it.
Podcasters want to know how they should feel about the number of downloads they're receiving or why their monthly download number was lower one month than another. And even our most analytical podcast hosts can have a hard time separating desired results (like new clients) from inconsequential metrics (like downloads).
Of course, this isn't unique to podcasters.
What job or activity isn't dominated by a dashboard of numbers that go up or down?
It's the first thing I see when I go to compose a new post on Substack. LinkedIn sends me a notification to let me know how many people saw my latest posts. Before I ghosted Instagram, I used to obsessively check the number of shares a post received in the first few days it was live.
My watch keeps track of the number of miles I've walked or run, the speed at which I've walked and run them, and the estimated VO2 max for each activity. The New York Times tells me how fast I did today's crossword and whether or not I'm a "genius" depending on how many Spelling Bee words I've found.
I won't waste (much) more of your time describing the ubiquity of metrics in our daily lives. I also don't want to spend (much) time talking about the consequences of obsessing over metrics. I think those subjects have been covered quite a bit. But coverage of the potentially harmful effects of the data-fication of our lives often misses a critical piece of the puzzle:
Metrics become incentives.
And those incentives warp our choices and behavior.
The reason "what gets measured gets managed" is that moving the number in the desired direction becomes a reward in itself. We get that little dopamine rush that comes from seeing the number tick up or down. And in the process, we tacitly endorse unhelpful and even harmful action for the sole purpose of seeing the number move.
Here's an example.
Very early in my tenure as a podcaster, I learned that you could "double" the downloads your show received simply by releasing twice as many episodes. And you didn't even need to make more episodes! You could just take your back catalog and release one old episode along with your new one each week!
The reason this works (or was at least plausible) is that the bulk of a podcast's downloads come from subscribers. And typically, subscribers have episodes downloaded automatically to their devices. So whether they wanted to listen to the rerun episode or not, subscribers would find it on their devices.
A podcast might receive double the downloads—but it didn't actually mean that more people were listening to the show. It just meant that subscribers were leveraged to change a number.
I'm sorry to say that I was taken by this ridiculous scheme. I released a new episode every Tuesday and a rerun episode every Thursday. Initially, yes, my downloads went up—although they never doubled. But it didn't take long before this release schedule was pissing people off.
Now, there's nothing wrong with releasing reruns periodically. I've done it several times this year. But releasing them for the purpose of juicing your download numbers? That's ridiculous. I really wish I could have seen that at the time, but I really wanted to increase my downloads even though I wasn't seeking outside advertising (the main reason you'd consider downloads as a metric in the first place). But that was just a downstream effect of how the scheme was invented in the first place.
Someone—I don't remember who—was so interested in boosting their downloads that they realized they could exploit a kind of loophole. They eked out a small advantage by leveraging that automatic download loophole and doubled their downloads. Further, they exploited this idea to garner attention from people who, like me, also wanted to influence that magical number.
I'm sure the initial advantage didn't last long, though. And they were probably off to exploit another loophole for the purpose of juicing their numbers in short order.
This, by the way, is how much of late capitalist business growth happens.
A company identifies a small advantage—often by playing fast and loose with metrics (e.g., page views, labor costs, cost of goods, etc). Then it jumps on that advantage and rides it for as long as it can—often just until the next earnings call. Then it finds another small advantage, and the process repeats.
This allows executives and shareholders to accumulate wealth while everyone else gets squeezed. The company doesn't work any better. It doesn't create more value outside of financial markets. It's just more successful on paper. The metrics that matter to investors become incentives to executives.
This isn't to say that making choices based on metrics always results in short-sighted behavior. But picking the wrong metric—the one that's not your real goal or one that you misunderstand the meaning of—will result in short-sighted behavior. The problem here is that most metrics, especially the highly visible ones, are the wrong metrics.
