Survivorship Bias: Avoiding the Trap: Understanding Survivorship Bias in Research

1. Introduction to Survivorship Bias

Survivorship bias is a logical error that occurs when a person concentrates on the people or things that "survived" some process and inadvertently overlooks those that did not because of their lack of visibility. This often leads to false conclusions in different ways. For example, when companies try to analyze the success stories of profitable startups without considering the vast majority that failed, they may falsely assume that the success strategies of the survivors are significantly more effective than they actually are.

From an investor's perspective, survivorship bias can lead to overly optimistic beliefs because failures are ignored. An investor might look at the performance of existing funds in a market but fail to account for those that have closed down. This can result in an overestimation of fund managers' skill and fund performance.

In military strategy, during World War II, the examination of returning aircraft and the subsequent reinforcement of areas that showed the most damage led to the realization that these were actually the areas least likely to be hit, as the planes had survived. The areas without damage on the returning planes were the most vulnerable, as planes hit there did not return.

Here are some in-depth points about survivorship bias:

1. Definition and Misconceptions: Survivorship bias often causes people to misjudge the success rate or effectiveness of a group or strategy by focusing only on the surviving members or outcomes.

2. Historical Examples: During World War II, the analysis of bomber planes that returned from missions led to the reinforcement of the least damaged areas, which was a classic case of survivorship bias.

3. Financial Markets: In finance, survivorship bias can cause investors to underestimate risks and overestimate returns by only considering currently successful companies or funds.

4. Research and Studies: In research, survivorship bias can lead to skewed results if the study only considers subjects that "survived" a certain process, like a clinical trial.

5. Business and Entrepreneurship: Entrepreneurs might fall into the trap of survivorship bias by only learning from successful businesses, ignoring the lessons from the vast majority that fail.

6. Policy Making: Policymakers might enact laws based on the outcomes of the "survivors," potentially overlooking the needs and circumstances of those who did not "survive."

7. Personal Development: Individuals might adopt habits or strategies from successful people without considering the full context of their success, including luck or unique circumstances.

To illustrate with an example, consider the publishing industry. Many aspiring authors look at best-selling books and try to mimic their strategies or themes, thinking that this will increase their chances of success. However, this ignores the countless manuscripts that never get published or the published works that never become best-sellers, often due to factors beyond the quality of writing, such as marketing and timing.

Understanding survivorship bias is crucial for making more informed decisions, whether in business, investing, policy-making, or personal life. By recognizing this bias, we can seek a more comprehensive view of reality, considering both the successes and the failures, which can lead to more balanced and effective strategies.

2. The Misleading Victories

In the realm of research and historical analysis, the concept of survivorship bias can significantly distort our understanding of events, particularly when considering the outcomes of conflicts or competitions. This cognitive bias leads us to focus on the winners or survivors, often overlooking the lessons that could be gleaned from those who did not fare as well. The misleading victories, therefore, are those that are remembered and studied, while the failures that could provide equally valuable insights are forgotten.

One classic example of survivorship bias in historical context is the analysis of military campaigns. Consider the following points:

1. Selective Memory: Historians and military strategists may study victorious battles extensively, but the defeats, which could offer crucial lessons in strategy and tactics, are often relegated to footnotes. For instance, the Battle of Cannae in 216 BC, where Hannibal's smaller Carthaginian army encircled and decimated a much larger Roman force, is less frequently discussed than his ultimate defeat in the Second Punic War.

2. Technological Advances: In the development of technology, we often celebrate the successful inventions while ignoring the multitude of failed prototypes that paved the way. The Wright brothers are celebrated for their first successful powered flight in 1903, but the countless attempts and failures that led to this achievement are seldom highlighted.

3. Economic Successes: In the business world, companies like Apple and Amazon are lauded for their success, but this overshadows the many startups that fail, which could teach valuable lessons about the market and consumer behavior.

4. Medical Research: In medicine, drugs that make it to market are the focus of attention, but the 'failed' drugs can sometimes lead to important discoveries. For example, the development of the erectile dysfunction drug Viagra was initially a failed attempt to treat angina.

5. Artistic Recognition: In the arts, certain works become iconic and define an era or genre, while other pieces, which may have been innovative or ahead of their time, fade into obscurity. Vincent van Gogh, now considered a master, sold only a few paintings in his lifetime and was not widely recognized until after his death.

By acknowledging the inherent bias in focusing solely on the successes, we can strive for a more balanced and comprehensive understanding of history and progress. It is through examining both the victories and the defeats that we can gain a fuller picture and potentially unlock insights that would otherwise remain hidden by the shadow of the winners. This approach not only enriches our knowledge but also fosters a culture of learning from all outcomes, not just the favorable ones.

3. Defining Survivorship Bias in Research

Survivorship bias is a logical error that occurs when a person focuses on the surviving subjects of a process and overlooks those that did not survive due to their lack of visibility. This bias can lead to overly optimistic beliefs because failures are ignored, and can be particularly problematic in research where it can skew results and conclusions. It's a subtle but pervasive error that can distort our understanding of the world, from the success of companies to the effectiveness of medical treatments.

For instance, consider a study on the effectiveness of a new drug. If only the patients who completed the treatment are included in the results, and those who dropped out due to side effects or other reasons are ignored, the study may indicate that the drug is more effective than it actually is. This is because the 'survivors' are not representative of the entire original group.

Insights from Different Perspectives:

1. Historical Perspective:

- Example: During World War II, planes returning from missions were studied for bullet holes to determine which areas needed reinforced armor. The mistake was to only consider the planes that returned and not those that were lost, leading to the incorrect assumption that the most damaged areas of the returning planes were the most critical to reinforce.

2. Business Perspective:

- Example: When analyzing successful companies, there's a tendency to ignore the vast number of startups that failed. This can lead to the false belief that certain strategies or characteristics guarantee success, when in fact, they may be common to both successful and unsuccessful companies.

3. Medical Perspective:

- Example: Clinical trials for a new therapy might only report on the patients who complete the trial, ignoring those who drop out. This can give a skewed impression of the therapy's effectiveness and its side effects profile.

4. Investment Perspective:

- Example: Financial funds often advertise their performance by highlighting successful funds while ignoring those that performed poorly or were liquidated. This can mislead investors about the average performance of investments in that category.

5. Educational Perspective:

- Example: alumni success stories are frequently used by educational institutions as a marketing tool, potentially giving a distorted view of the average outcomes for all students, including those who may not have achieved similar success.

avoiding Survivorship bias:

To avoid survivorship bias, it's crucial to consider all available data, including those that did not 'survive' the process being studied. This means:

- Including dropouts and failures in research data.

- Considering the performance of all funds, not just the successful ones, when evaluating investment options.

- Looking at outcomes for all students, not just the most successful alumni, when assessing educational programs.

By acknowledging and adjusting for survivorship bias, researchers and decision-makers can arrive at more accurate and representative conclusions. This helps in making better-informed decisions, whether in policy-making, business strategy, or personal investments. Remember, what we see often represents only a fraction of the whole picture, and it's the unseen and unaccounted-for that can provide the most valuable insights.

4. When Success Stories Skew Perception?

In the realm of research and decision-making, success stories often take center stage, casting a spotlight on the winners while inadvertently casting a shadow over the less fortunate or less successful. This selective focus can lead to a skewed perception of reality, where the strategies and characteristics of the successful are overemphasized, potentially at the expense of valuable insights that could be gleaned from those who did not succeed. This phenomenon is a classic example of survivorship bias, a cognitive bias that occurs when we concentrate on people or things that have 'survived' some process and inadvertently overlook those that did not due to their lack of visibility.

1. Overlooking the Full Picture: Often, success stories become the blueprint for aspiring achievers. However, this ignores the many equally or more talented individuals who, due to various factors such as timing, luck, or unseen obstacles, did not come out on top. For instance, for every successful startup, there are numerous others that did not survive, despite having similar or even superior business models.

2. Misattribution of Causes: It's easy to attribute the success of individuals or businesses to their apparent qualities or strategies. Yet, this can be misleading. For example, a company might thrive not because of its unique business strategy but because it entered the market at an opportune time when competition was low.

3. Underestimating Risk: Survivorship bias can lead to an underestimation of risk. If we only look at successful stock traders, we might conclude that stock trading is less risky than it actually is. This can lead to overconfidence and potentially reckless behavior in markets.

4. Simplification of Complex Systems: Success is often the result of a complex interplay of factors. By focusing only on those who succeed, we simplify these systems and ignore the complexity. For example, attributing a sports team's victory solely to a star player overlooks the contributions of the entire team, coaching staff, and other contextual factors.

5. Historical Distortion: In history, the stories of victors are predominantly told, which can distort our understanding of events. The perspectives of the defeated can provide critical insights into the full scope of historical events but are often neglected.

6. Policy and Planning Flaws: When policymakers base decisions on success stories, they may create plans that are not robust to the varied realities of different populations. For example, an educational reform based on a model school's success might not account for the challenges faced by schools in less affluent areas.

7. Innovation Stagnation: If success is only measured by the current standards, then unconventional ideas that could lead to breakthroughs might be discarded prematurely. The Wright brothers, for instance, succeeded in flight not by following the footsteps of successful glider pilots but by innovating through their understanding of aerodynamics.

By recognizing and accounting for survivorship bias, we can strive for a more balanced view that values the lessons from both success and failure. This approach can lead to more informed decisions, richer understandings, and ultimately, more sustainable success that is not just a matter of chance but a product of comprehensive analysis and learning from the entire spectrum of experiences.

5. The Impact of Survivorship Bias in Financial Markets

Survivorship bias in financial markets is a phenomenon that can lead to significant distortions in understanding risk and return. It occurs when analyses only consider existing or "surviving" entities, such as funds or companies, while ignoring those that have failed or disappeared. This bias can create an overly optimistic view of investment opportunities and performance because the data set is cleansed of the worst performers, which have been eliminated from the pool of observable outcomes. For instance, when evaluating the average returns of mutual funds over a period, if the analysis only includes funds that are still active and disregards those that have closed down, the results may suggest a higher average return than what is truly representative of the market's overall performance.

1. Performance Evaluation: Survivorship bias can lead to inaccurate assessments of fund managers' performance. A study might show that the majority of active funds beat the market, but this could be because only the successful ones remain, while those that underperformed have been liquidated.

2. Historical Backtesting: In backtesting investment strategies, survivorship bias can result in overly optimistic outcomes. If a strategy is tested using a current list of stocks, it ignores those that have gone bankrupt or been delisted, thus inflating the strategy's apparent success.

3. Market Indexes: Indexes that track market performance can also be affected by survivorship bias. Companies that fail are removed from indexes, and their negative performance is not reflected in the index's historical data, potentially leading to an overestimation of past returns.

4. Academic Research: Survivorship bias can skew academic research on financial markets. Studies that do not account for failed companies may draw conclusions about market behaviors or corporate characteristics that are not accurate for the entire universe of firms.

5. Investment Decisions: Investors may make decisions based on flawed data, believing that certain sectors or strategies are more successful than they actually are. For example, the tech sector might appear particularly robust if only the current, successful tech companies are considered, ignoring the many that failed during the dot-com bubble.

6. Risk Perception: The bias affects investors' perception of risk. By only observing the winners, investors might underestimate the risks involved in the market and allocate their resources under false pretenses of safety and high returns.

7. Policy Making: Policymakers might draw incorrect conclusions about the health of financial markets or the effectiveness of regulations if their analysis is based on biased data sets that do not include failed institutions.

To illustrate, consider the case of hedge funds. The reported average returns of hedge funds may be misleading if they do not include those that have closed. For instance, if only 25% of hedge funds survive over a 20-year period, and these survivors report an average annual return of 10%, this figure does not account for the potentially negative returns of the 75% that did not survive, which could significantly lower the true average return of all hedge funds started 20 years ago.

Survivorship bias is a subtle yet powerful force that can warp our understanding of financial markets. It's crucial for analysts, investors, and policymakers to be aware of this bias and to seek out data and methodologies that provide a more complete and accurate picture of market dynamics and investment performance.

6. Survivorship Bias in Healthcare Studies

Survivorship bias is a particularly insidious issue in healthcare studies, where the outcomes can significantly influence clinical decisions and policy-making. This form of bias occurs when studies focus on individuals or groups that have 'survived' a particular process, overlooking those who did not make it through. In healthcare, this often translates to an emphasis on patients who respond well to a treatment or those who continue to seek care, inadvertently ignoring the experiences of those who may have dropped out due to adverse effects, financial constraints, or even mortality. The danger here is the potential to draw conclusions that are overly optimistic or not fully representative of the reality, leading to treatments that may not be as effective as they seem or policies that do not address the needs of the entire patient population.

From the perspective of clinicians, survivorship bias can lead to an overestimation of a treatment's effectiveness. For example, if a study only reports on patients who completed a full course of chemotherapy and survived, it may not account for those who had to stop treatment early due to severe side effects. This could result in a skewed perception that chemotherapy is more tolerable than it actually is.

Researchers must also navigate this bias carefully. When designing studies, they need to ensure that dropouts and non-responders are accounted for. Otherwise, the results may not accurately reflect the treatment's efficacy or safety profile.

Patients themselves can be influenced by survivorship bias when they hear success stories from other patients or read about positive outcomes online. This might lead them to have unrealistic expectations about their own treatment journey.

To delve deeper into the implications of survivorship bias in healthcare studies, consider the following points:

1. Selection of Participants: Studies that only include participants who have already survived a certain stage of a disease or treatment can skew results. For instance, a study on long-term cancer survival rates that only includes patients who have survived the first five years post-diagnosis will naturally show a higher survival rate than if all diagnosed patients were considered.

2. Publication Bias: Research that yields positive results is more likely to be published. This means that unsuccessful treatments, which could provide valuable data, are often underrepresented in the literature, potentially leading to an overemphasis on successful interventions.

3. Longitudinal Studies: These studies follow patients over time and can provide a more comprehensive view of treatment outcomes. However, if they do not account for those who drop out or die during the study, the results can reflect survivorship bias.

4. real-world examples: The case of breast cancer screening illustrates this bias well. Mammography tends to detect slower-growing tumors with better prognoses, while aggressive cancers may go undetected until they are advanced. This can lead to the mistaken belief that screening dramatically improves survival rates.

5. Policy Implications: Healthcare policies based on biased data can lead to misallocation of resources. For example, if survivorship bias leads to an overestimation of a treatment's success, resources may be diverted away from other, potentially more effective treatments.

6. Ethical Considerations: Survivorship bias raises ethical questions about informed consent and patient expectations. Patients must be made aware of all potential outcomes of a treatment, not just the most positive ones.

By recognizing and addressing survivorship bias, healthcare professionals and researchers can work towards more accurate studies and better-informed clinical practices. It's crucial for the integrity of healthcare research and the well-being of patients that all voices and outcomes are considered in the pursuit of medical knowledge and treatment efficacy.

7. Strategies to Identify and Overcome Survivorship Bias

Survivorship bias is a form of selection bias that occurs when a study or analysis only considers "survivors" or existing members of a group, overlooking those that have "fallen out" or ceased to exist. This can lead to overly optimistic beliefs because failures are ignored. For instance, when analyzing the success rates of companies, focusing only on those that are currently thriving ignores the lessons that could be learned from those that did not survive. To avoid the trap of survivorship bias, it is crucial to recognize its presence and implement strategies that account for the whole spectrum of experiences, including failures.

Strategies to Identify Survivorship Bias:

1. Historical Analysis: Look back at the entire history of a group, not just the current members. For example, when studying successful businesses, include those that have failed in the past.

2. Control Groups: Use control groups that represent what the population looked like before the selection process that led to the survivors.

3. Multiple Data Sources: Gather data from various sources to get a fuller picture of the population in question.

4. Acknowledging Missing Data: Be aware of and account for data that is missing due to the non-survival of some subjects.

Strategies to Overcome Survivorship Bias:

1. Comprehensive Data Collection: Ensure that data collection processes capture information from all relevant parties, not just the successful ones.

2. Statistical Adjustments: apply statistical techniques to adjust for the effects of survivorship bias.

3. Critical Analysis: Question assumptions and critically analyze results to ensure they are not skewed by only considering survivors.

4. External Validity: Check that findings are applicable to the broader population and not just the surviving sample.

Examples Highlighting Strategies:

- In finance, an investor might look at mutual funds that have performed well over the past decade. However, this view might be skewed if funds that performed poorly and were subsequently closed are not considered. To overcome this, the investor should look at the performance of all funds that started at the beginning of the period, not just those that have survived.

- In clinical trials, researchers might only publish studies where the treatment was effective, ignoring trials where the treatment failed. This can be addressed by registering all trials in advance and committing to publish all results, successful or unsuccessful.

By employing these strategies, researchers and analysts can mitigate the effects of survivorship bias, leading to more accurate and reliable conclusions.

8. Lessons Learned from Survivorship Bias

Survivorship bias is a form of selection bias that occurs when a study or analysis only considers "survivors" or existing subjects, overlooking those that have "fallen away" or ceased to exist due to failure, death, or other reasons. This bias can lead to overly optimistic beliefs because failures are ignored, and can be particularly misleading in fields ranging from finance to healthcare. By examining case studies across various domains, we can uncover the nuanced lessons that survivorship bias teaches us, and learn to apply corrective measures in our research and decision-making processes.

1. Finance and Investment: In the realm of finance, survivorship bias often skews perceptions of fund performance. For instance, when evaluating mutual funds, only those that have survived over time are considered, ignoring those that have dissolved due to poor performance. This can lead to an inflated average return for the sector. A notable example is the analysis of stock performance during the tech bubble of the late 1990s. Many investors believed that tech stocks were a surefire path to wealth, but this belief was largely based on the performance of surviving companies, not taking into account numerous startups that failed and were no longer present in the market data.

2. Military Strategy: A classic example of survivorship bias comes from World War II when the statistician Abraham Wald was asked to help the British Royal Air Force determine how to better protect their aircraft from enemy fire. The initial analysis focused on reinforcing areas that showed the most damage on returning planes. However, Wald pointed out that the planes that did not return were the ones to study, as they likely represented areas critical to a plane's survival. This insight shifted the focus to reinforcing areas that appeared undamaged on returning aircraft, which were likely the areas hit on planes that were lost.

3. Healthcare Research: In medical trials, survivorship bias can occur when patients who drop out of a study are not included in the final results, potentially leading to an overestimation of treatment effectiveness. An example is seen in studies of a particular cancer treatment where the reported survival rates may only include patients who completed the full course of treatment, disregarding those who may have died or stopped treatment early due to adverse effects.

4. Entrepreneurship and Business: The media often highlights successful entrepreneurs, creating a narrative that suggests a high likelihood of success in startup ventures. However, this overlooks the many businesses that fail within their first few years. For example, the oft-cited statistic that "90% of startups fail" is a reminder of the importance of considering the full spectrum of outcomes when evaluating the potential success of a new business venture.

5. Technological Innovation: In technology, products that become industry standards can overshadow failed attempts, even if those failures contributed valuable lessons to the development of successful products. The story of the personal computer is one such case, where the success of brands like Apple and IBM overshadows the many companies that did not survive the competitive market.

By recognizing and adjusting for survivorship bias, researchers and practitioners can make more informed decisions and produce more accurate analyses. It is crucial to consider not just the survivors but also those that did not make it, as they offer critical insights into the risks and realities of any given field.

Get matched with over 155K angels worldwide!

FasterCapital uses warm introductions and an AI system to approach investors effectively with a 40% response rate!

Join us!

9. Building a More Accurate Picture of Reality

In our quest to understand the world, we often look to those who have succeeded, drawing conclusions from their experiences. However, this approach can lead to a skewed perception of reality, as it overlooks the many more who may have attempted and failed. This phenomenon, known as survivorship bias, can cause significant distortions in how we interpret success and failure. To build a more accurate picture of reality, it is crucial to consider both the winners and the losers in any given situation.

1. comprehensive Data analysis: To avoid survivorship bias, it's essential to analyze data from a complete spectrum of experiences. For instance, when studying successful companies, also examine those that failed. This broader view can reveal common pitfalls and the true factors behind success.

2. Historical Context: Understanding the context in which decisions were made is vital. During World War II, planes returning from missions were studied for bullet holes to improve armor placement. However, planes that didn't return, which likely had critical damage, were not considered initially. This oversight could have led to misguided conclusions.

3. Diverse Perspectives: Engage with a variety of viewpoints. In medicine, clinical trials that only publish positive results may give an inflated sense of a treatment's effectiveness. Including trials with negative or neutral outcomes provides a more balanced understanding.

4. Statistical Significance: Ensure that the data used to draw conclusions is statistically significant. For example, if a basketball player has a streak of successful games, it's important to analyze their overall performance to determine if the streak is an outlier or indicative of true skill.

5. Control Groups: Use control groups to compare outcomes. In business, comparing startups that received venture capital with those that didn't can highlight the impact of funding on success.

By incorporating these approaches, we can mitigate the effects of survivorship bias and arrive at conclusions that more accurately reflect the complexities of reality. For example, when examining the success of tech startups, it's not enough to only consider the unicorns like Facebook or Google. We must also study the vast number of startups that didn't make it, to understand the full range of factors that contribute to success in the industry. Only then can we claim to have a more accurate picture of reality, one that takes into account the full journey, not just the destination.