Risk of astronomical suffering

Scope–severity grid from Bostrom's paper "Existential Risk Prevention as Global Priority"

Risks of astronomical suffering, also called suffering risks or s-risks, are risks involving much more suffering than all that has occurred on Earth so far. They are sometimes categorized as a subclass of existential risks.

According to some scholars, s-risks warrant serious consideration as they are not extremely unlikely and can arise from unforeseen scenarios. Although they may appear speculative, factors such as technological advancement, power dynamics, and historical precedents indicate that advanced technology could inadvertently result in substantial suffering. Thus, s-risks are considered to be a morally urgent matter, despite the possibility of technological benefits.

Sources of possible s-risks include embodied artificial intelligence and superintelligence, as well as space colonization, which could potentially lead to "constant and catastrophic wars" and an immense increase in wild animal suffering by introducing wild animals, who "generally lead short, miserable lives full of sometimes the most brutal suffering", to other planets, either intentionally or inadvertently.

Types of S-risk

Artificial intelligence

Artificial intelligence is central to s-risk discussions because it may eventually enable powerful actors to control vast technological systems. In a worst-case scenario, AI could be used to create systems of perpetual suffering, such as a totalitarian regime expanding across space. Additionally, s-risks might arise incidentally, such as through AI-driven simulations of conscious beings experiencing suffering, or from economic activities that disregard the well-being of nonhuman or digital minds. Steven Umbrello, an AI ethics researcher, has warned that biological computing may make system design more prone to s-risks. Brian Tomasik has argued that astronomical suffering could emerge from solving the AI alignment problem incompletely. He argues for the possibility of a "near miss" scenario, where a superintelligent AI that is slightly misaligned has the maximum likelihood of causing astronomical suffering, compared to a completely unaligned AI.

Space colonization

Space colonization could increase suffering by introducing wild animals to new environments, leading to ecological imbalances. In unfamiliar habitats, animals may struggle to survive, facing hunger, disease, and predation. These challenges, combined with unstable ecosystems, could cause population crashes or explosions, resulting in widespread suffering. Additionally, the lack of natural predators or proper biodiversity on colonized planets could worsen the situation, mirroring Earth’s ecological problems on a larger scale. This raises ethical concerns about the unintended consequences of space colonization, as it could propagate immense animal suffering in new, unstable ecosystems. Phil Torres argues that space colonization poses significant "suffering risks", where expansion into space will lead to the creation of diverse species and civilizations with conflicting interests. These differences, combined with advanced weaponry and the vast distances between civilizations, would result in catastrophic and unresolvable conflicts. Strategies like a "cosmic Leviathan" to impose order or deterrence policies are unlikely to succeed due to physical limitations in space and the destructive power of future technologies. Thus, Torres concludes that space colonization could create immense suffering and should be delayed or avoided altogether.

Genetic engineering

David Pearce has argued that genetic engineering is a potential s-risk. Pearce argues that while technological mastery over the pleasure-pain axis and solving the hard problem of consciousness could lead to the potential eradication of suffering, it could also potentially increase the level of contrast in the hedonic range that sentient beings could experience. He argues that these technologies might make it feasible to create "hyperpain" or "dolorium" that experience levels of suffering beyond the human range.

Excessive criminal punishment

S-risk scenarios may arise from excessive criminal punishment, with precedents in both historical and in modern penal systems. These risks escalate in situations such as warfare or terrorism, especially when advanced technology is involved, as conflicts can amplify destructive tendencies like sadism, tribalism, and retributivism. War often intensifies these dynamics, with the possibility of catastrophic threats being used to force concessions. Agential s-risks are further aggravated by malevolent traits in powerful individuals, such as narcissism or psychopathy. This is exemplified by totalitarian dictators like Hitler and Stalin, whose actions in the 20th century inflicted widespread suffering.

Exotic risks

According to David Pearce, there are other potential s-risks that are more exotic, such as those posed by the many-worlds interpretation of quantum mechanics.

Other classifications

According to Tobias Baumann, s-risks can be grouped into three main categories:

• Incidental s-risks: These arise when suffering is an unintended consequence of pursuing efficient solutions, rather than being the intended goal. Examples include factory farming, where economic efficiency leads to animal suffering, or advanced AI simulations that could unintentionally cause widespread suffering as a byproduct of optimizing tasks like learning or problem-solving.

• Agential s-risks: These occur when suffering is deliberately inflicted by agents with harmful intent, driven by motivations such as sadism, hatred, or retribution. Historical examples include totalitarian regimes like those of Hitler or Stalin. In the future, these risks could be exacerbated by powerful technologies enabling individuals or groups to inflict even greater harm.

• Natural s-risks: These involve suffering that occurs naturally, without any human or artificial agents being responsible. A prime example is the suffering of wild animals in nature due to hunger, disease, or predation. On a larger scale, if suffering were common on other planets or spread through human terraforming efforts, it could become a natural s-risk on an astronomical level.

Baumann emphasizes that these examples are speculative and acknowledges the uncertainty of future developments. He also warns of availability bias, which can lead to overestimating the likelihood of certain scenarios, stressing the importance of considering a broad spectrum of potential s-risks.

Mitigation strategies

To mitigate s-risks, efforts focus on researching and understanding the factors that exacerbate them, particularly in emerging technologies and social structures. Targeted strategies include promoting safe AI design, ensuring cooperation among AI developers, and modeling future civilizations to anticipate risks. Broad strategies may advocate for moral norms against large-scale suffering and stable political institutions. According to Anthony DiGiovanni, prioritizing s-risk reduction is essential, as it may be more manageable than other long-term challenges, while avoiding catastrophic outcomes could be easier than achieving an entirely utopian future.

Induced amnesia

Induced amnesia has been proposed as a way to mitigate s-risks in locked-in conscious AI and certain AI-adjacent biological systems like brain organoids.

Cosmic rescue missions

David Pearce's concept of "cosmic rescue missions" proposes the idea of sending probes to alleviate potential suffering in extraterrestrial environments. These missions aim to identify and mitigate suffering among hypothetical extraterrestrial life forms, ensuring that if life exists elsewhere, it is treated ethically. However, challenges include the lack of confirmed extraterrestrial life, uncertainty about their consciousness, and public support concerns, with environmentalists advocating for non-interference and others focusing on resource extraction.

See also

• AI control problem
• Ethics of artificial intelligence
• Ethics of terraforming
• Existential risk from artificial general intelligence
• Experience machine
• Global catastrophic risk
• Suffering-focused ethics
• Waluigi effect
• Wild animal suffering
• Wirehead (science fiction)

Further reading

• Baumann, Tobias (2022). Avoiding the Worst: How to Prevent a Moral Catastrophe. Independently published. ISBN 979-8359800037.
• Metzinger, Thomas (2021-02-19). "Artificial Suffering: An Argument for a Global Moratorium on Synthetic Phenomenology". Journal of Artificial Intelligence and Consciousness. 08: 43–66. doi:10.1142/S270507852150003X. ISSN 2705-0785.
• Minardi, Di (2020-10-15). "The grim fate that could be 'worse than extinction'". BBC Future. Retrieved 2021-02-11.
• Baumann, Tobias (2017). "S-risks: An introduction". Center for Reducing Suffering. Retrieved 2021-02-10.
• Althaus, David; Gloor, Lukas (2016-09-14). "Reducing Risks of Astronomical Suffering: A Neglected Priority". Center on Long-Term Risk. Retrieved 2021-02-10.