The bullet-hole plane and why success stories can mislead you

Somewhere in your feed this morning, a founder explained the routine behind their success. Up before dawn. Cold shower. No phone until lunch. Thousands of people liked it, and the replies are full of promises to start tomorrow.

It feels like evidence. Someone who made it is telling you what they did. But notice who gets to post. The people who rose early, took the cold shower, and still failed don’t write threads about their mornings.

That gap has a name, survivorship bias, and its best-known illustration is a bomber covered in red dots. The real story behind that picture is smaller and stranger than the legend. By the end, you’ll be able to tell when a success story is evidence and when it’s just a survivor talking. You’ll also know which question to ask before copying anyone’s playbook.

The planes you never see

Picture the old washing machine in a grandparent’s basement, still running after decades. People say they don’t make them like they used to. But the machines from that era that broke down were hauled away long ago. What’s left in basements is a filtered sample. The ones you can still see are, by definition, the ones that lasted.

That’s the whole principle. When an outcome decides who stays visible, the visible group can’t tell you much about what caused the outcome. Sounds too obvious to matter? It is obvious, right up until a story makes you want to change your life.

Wartime bombers had the same problem. Planes that came back could be inspected hole by hole. Planes that were shot down could not. Any damage pattern drawn from returning aircraft came only from aircraft that had survived their damage.

What Wald actually wrote

The version you’ve probably heard goes like this. Officers study returning bombers and want to add armor where the bullet holes cluster. A lone mathematician, Abraham Wald, stops them. Armor the places without holes, he says, because planes hit there never made it home.

The core logic is real. The scene is not documented.

Here is what the record shows. Wald worked inside the Statistical Research Group, or SRG, at Columbia University. In 1943 his work on the problem came out as a series of eight memoranda for the National Defense Research Committee, titled “A Method of Estimating Plane Vulnerability Based on Damage of Survivors.” The Center for Naval Analyses reprinted them on 1 July 1980.

Wald wasn’t a lone outsider. The SRG also included the economists Milton Friedman and George Stigler, among others. The memos showed how to estimate which parts of a plane were most vulnerable from the damage on planes that returned. By implication, they also reasoned about the damage nobody saw, because those planes never came back.

What the memos didn’t do was tell anyone where to add armor. According to a 2016 review in the American Mathematical Society’s Feature Column, they say nothing about what the military should do. The SRG’s policy was to answer the specific statistical question it was asked, not to issue its own recommendations.

The same review reports that the only reliable primary sources for the famous story are Wald’s memoranda and two vague mentions in the memoir of SRG director W. Allen Wallis. Those mentions don’t even name Wald. Bill Casselman, who wrote the review, draws the line plainly:

Everything not in one of these places must be considered as fiction, not fact.

Statisticians got a formal account decades later. In 1984, Marc Mangel and Francisco Samaniego reconstructed and condensed Wald’s method in the Journal of the American Statistical Association.

And the red-dot bomber? It isn’t from Wald’s papers. Designer Cameron Moll made an earlier red-dot diagram around 2005 for Moll’s own slide decks and blog posts. The now-familiar image was added to the English Wikipedia article on survivorship bias on 12 November 2016 by an editor called McGeddon, using the outline of a Lockheed PV-1 Ventura. Martin Grandjean made a cleaner vector version in 2021. From the 1943 memos to Moll’s diagram around 2005 is roughly six decades.

One way to read this: the story about survivorship bias went through its own filter. Dry technical memos are hard to retell. A hero, a doubtful room, and a clean picture are easy to retell, so that’s the version that spread.

Three questions before you copy a playbook

Wald’s memos point to a simple habit: before trusting a pattern, ask what the sample left out. You can carry that habit into ordinary choices.

First: who did this and didn’t make it?

A habit counts as evidence when you can compare the people who had it with the people who didn’t, including the ones who failed. A podcast guest says dropping out of college was the best decision they ever made. Maybe it was. But dropouts who struggled rarely get booked on podcasts. Before you weigh the advice, picture the people who made the same move and are now quietly rebuilding a résumé. The story stops sounding like a recipe and starts sounding like one data point.

Second: where are the missing holes?

Silence in your data can be information, not a blank. Think of a gym’s wall of before-and-after photos. Every photo shows someone who stuck with it. Nobody pins up the member who quit after a knee injury. A company’s careers page works the same way, featuring people who stayed and thrived rather than the ones who left. Neither one is lying. They just can’t show you who the place didn’t fit. Once you see that, you start asking what kind of person a program loses, and whether that might be you.

Third: has the story been polished?

Stories survive by being retold, and the retold versions tend to be the most dramatic. A coworker explains over drinks how the boss got promoted: one bold email, one big yes. The months of ordinary work don’t fit in the anecdote. When a story has a perfect turning point, hold that detail loosely. Read it the way the red dots should be read, as a teaching sketch rather than a record.

The other side

None of this makes success stories useless. Survivor data is exactly what Wald worked with. The returning planes were the evidence on hand, and they still said something real about the planes that didn’t return. The mistake isn’t studying survivors. It’s assuming survivors stand in for everyone who tried. The reasoning gets sturdier when you compare survivors with those who didn’t make it, or with how often the outcome happens overall.

A success story can also show that something is possible at all. If one person built a living from an unusual craft, that path exists. What the story can’t tell you is how many people tried it and quietly stopped.

Some habits do help. When they do, the case rests on other evidence you can check, not on who happens to be telling the story. So does the founder’s morning routine work? Their post alone can’t say.

Honesty also cuts the other way on Wald. The dramatic scene could have happened. Casselman’s own verdict was careful:

The story might well be true … [but] there is very little evidence for the best bits.

A missing record doesn’t prove nothing happened. It means the showdown is reconstructed color, and the logic stands without it.

Try this today

Pick one success story you run into today. It could be a post, a podcast clip, or a colleague’s advice at lunch. Before you decide whether to copy it, type one line into your notes app: Who did this and didn’t make it?

Then take one of three small steps.

1. If you can name those people, or find out how common failure is, you have something closer to evidence. Weigh the story against it.
2. If you can’t, label the story honestly: a survivor talking. Interesting, maybe useful, not proof.
3. If the story has a perfect hero moment, ask where you first heard it and who was actually there.

Try it for one day. Not forever. One day. You may notice how few stories arrive with their missing planes attached.

That’s the line between evidence and a survivor talking. A success story starts to count as evidence when you can see who else tried and what happened to them. Without that, it’s one plane that came home: sincere, sometimes wise, but only part of the sample. Ask who didn’t make it before you copy anyone’s playbook. Then you’re reading the story the way Wald’s memos read the bombers, counting what came back and reasoning about what didn’t.