Superintelligence

Abstract: In this paper we focus on the most efficacious AI applications for life extension and anti-aging at three expected stages of AI development: narrow AI, AGI and superintelligence. First, we overview the existing research and commercial work performed by a select number of startups and academic projects. We find that at the current stage of “narrow” AI, the most promising areas for life extension are geroprotector-combination discovery, detection of aging biomarkers, and personalized anti-aging therapy. These advances could help currently living people reach longevity escape velocity and survive until more advanced AI appears. When AI comes close to human level, the main contribution to life extension will come from AI integration with humans through brain-computer interfaces, integrated AI assistants capable of autonomously diagnosing and treating health issues, and cyber systems embedded into human bodies. Lastly, we speculate about the more remote future, when AI reaches ...

In 1964, Nikolai Kardashev proposed the Kardashev scale, a system for measuring the extent of technological advancement of a civilization based on the magnitude of energy consumption. According to Carl Sagan, the American astrophysicist, we are approaching an inevitable type-1 Kardashev civilized existence. An artificial superintelligence superior to that of humans can concur with a higher-hierarchy Kardashev civilization. Futuristic technological innovations mandate an acceleration in artificial intelligence and superintelligence. This scenario can be pessimistic, as digital superintelligence can render human-based activities obsolete. However, integrating artificial intelligence with humans, via brain-computer interface technologies, can be protective. Nonetheless, legislation in connection with information technologies is essential to regulate upcoming digital knowledge and superintelligence. Keywords: Artificial general intelligence, consciousness, human-level machine intelligen...

An artificial superintelligence (ASI) is artificial intelligence that is significantly more intelligent than humans in all respects. While ASI does not currently exist, some scholars propose that it could be created sometime in the future, and furthermore that its creation could cause a severe global catastrophe, possibly even resulting in human extinction. Given the high stakes, it is important to analyze ASI risk and factor the risk into decisions related to ASI research and development. This paper presents a graphical model of major pathways to ASI catastrophe, focusing on ASI created via recursive self-improvement. The model uses the established risk and decision analysis modeling paradigms of fault trees and influence diagrams in order to depict combinations of events and conditions that could lead to AI catastrophe, as well as intervention options that could decrease risks. The events and conditions include select aspects of the ASI itself as well as the human process of ASI research, development, and management. Model structure is derived from published literature on ASI risk. The model offers a foundation for rigorous quantitative evaluation and decision making on the long-term risk of ASI catastrophe.

Discussions about the possible consequences of creating superintelligence have included the possibility of existential risk , often understood mainly as the risk of human extinction. We argue that suffering risks (s-risks) , where an adverse outcome would bring about severe suffering on an astronomical scale, are risks of a comparable severity and probability as risks of extinction. Preventing them is the common interest of many different value systems. Furthermore, we argue that in the same way as superintelligent AI both contributes to existential risk but can also help prevent it, superintelligent AI can both be a suffering risk or help avoid it. Some types of work aimed at making superintelligent AI safe will also help prevent suffering risks, and there may also be a class of safeguards for AI that helps specifically against s-risks.

While Nick Bostrom offers some issues of concern on the topics of Transhumanism, Transcendence, Superintelligence, and the Singularity, Ray Kurzweil sees only potential.  These two authors offer a wide range of ideas on the potential of Transhumanism, not all of which agree, except for the fact that the way humanity currently lives may change greatly.  The question to ask is this: Will the age of Transhumanism bring humanity closer together or push people apart?

The long unbearable sufferings in the past and agonies experienced in some future timelines in which a malevolent AI could torture people for some idiosyncratic reasons (s-risks) is a significant moral problem. Such events either already happened or will happen in causally disconnected regions of the multiverse and thus it seems unlikely that we can do anything about it. However, at least one pure theoretic way to cure past sufferings exists. If we assume that there is no stable substrate of personal identity and thus a copy equals original, then by creating many copies of the next observer-moment of a person in pain in which he stops suffer, we could create indexical uncertainty in her future location and thus effectively steal her consciousness from her initial location and immediately relieve her sufferings. However, to accomplish this for people who have already died, we need to perform this operation for all possible people thus requiring enormous amounts of computations. Such computation could be performed by the future benevolent AI of Galactic scale. Many such AIs could cooperate acausally by distributing parts of the work between them via quantum randomness. To ensure their success, they need to outnumber all possible evil AIs by orders of magnitude, and thus they need to convert most of the available matter into computronium in all universes where they exist and cooperate acausally across the whole multiverse. Another option for curing past suffering is the use of wormhole time-travel to send a nanobot in the past which will, after a period of secret replication, collect the data about people and secretly upload them when their suffering becomes unbearable. • The existence of intense past sufferings a significant moral problem. • If we create many copies of the next observer-moment of a person in which sufferings end, it creates indexical uncertainty about her future location and effectively saves her from hell. • To save any past person from sufferings, a future benevolent AI needs to create many possible extensions of each of her observer-moments. • This requires multiverse-wide acausal cooperation between superintelligences to perform an enormous amount of computations. • Curing past suffering and resurrection of all the dead is basically the same procedure. • An alternative way of curing past sufferings is to send a nanorobot into the past via a wormhole-or travelling between everettian branches.

Trust is believed to be a foundational cornerstone for artificial intelligence (AI). In April 2019 the European Commission High Level Expert Group on AI adopted the Ethics Guidelines for Trustworthy AI, stressing that human beings will only be able to confidently and fully reap the benefits of AI if they can trust the technology. Trustworthy AI is defined as ethical, lawful and robust AI. Three things strike me about the EC Guidelines. Firstly, though building trust in AI seems to be a shared aim, it is not explicated what trust is, and how it can be built and maintained. Secondly, the Guidelines ignore the widespread distinction made in philosophical literature between trust and reliance. Thirdly, it is not clear how the values have been selected with which AI has to align and what would happen if they came into conflict. In this paper, I shall provide a conceptual analysis of trust in contrast to reliance and ask when it is warranted to talk about trust in AI and trustworthy AI. I shall show how trust and risk are related and what benefits and risks are associated with narrow and general AI. Also, I shall point out that metaphorical talk about ethically aligned AI ignores the real disagreements we have about ethical values.

The aim of this course is to engage students in contemporary debates in the philosophy of mind. The course is divided into four modules.  In the first segment, which lays the foundation, we survey a variety of attempts to clarify the relationship between mind and body.  In the second segment, we consider the ‘self-model’ theory of subjectivity as a framework for studying conscious experience, the phenomenal self, and the emergence of the first-person perspective.  In the third segment, we challenge the assumption that mind and consciousness are confined inside the boundaries of the brain, and learn about alternative approaches that highlight the embodied, embedded, extended, and enactive (‘4e’) character of cognition. In the last segment, we deal with the question of whether machines can think, and ponder how the creation of different forms of ‘superintellligence’ might affect the future of the human condition.

The combination of superintelligence and massive computational power will make it possible for computers in the future to create and sustain virtual environments of a size and complexity that is way beyond anything we are currently capable of devising. Will it be possible — or desirable — to 'upload' ourselves into these virtual worlds? David Chalmers has interesting things to say on this issue; I will be suggesting a slightly different take on it. The existence of these virtual worlds leads to another question: might it not be possible—or even probable—that we ourselves are among their (virtual) inhabitants? This issue has been discussed fairly widely in recent years. I will be proposing some relevant (and hopefully, useful) distinctions and offering some reflections on it.

As artificial intelligence (AI) systems are becoming increasingly autonomous and will soon be able to make decisions on their own about what to do, AI researchers have started to talk about the need to align AI with human values. The AI 'value alignment problem' faces two kinds of challenges-a technical and a normative one-which are interrelated. The technical challenge deals with the question of how to encode human values in artificial intelligence. The normative challenge is associated with two questions: "Which values or whose values should artificial intelligence align with?" My concern is that AI developers underestimate the difficulty of answering the normative question. They hope that we can easily identify the purposes we really desire and that they can focus on the design of those objectives. But how are we to decide which objectives or values to induce in AI, given that there is a plurality of values and moral principles and that our everyday life is full of moral disagreements? In my paper I will show that although it is not realistic to reach an agreement on what we, humans, really want as people value different things and seek different ends, it may be possible to agree on what we do not want to happen, considering the possibility that intelligence, equal to our own, or even exceeding it, can be created. I will argue for pluralism (and not for relativism!) which is compatible with objectivism. In spite of the fact that there is no uniquely best solution to every moral problem, it is still possible to identify which answers are wrong. And this is where we should begin the value alignment of AI.

There has been a dramatic increase in media interest in Artificial Intelligence (AI), in particular with regards to the promises and potential pitfalls of ongoing research, development and deployments. Recent news of success and failures are discussed. The existential opportunities and threats of extreme goals of AI (expressed in terms of Superintelligence/AGI and SocioEconomic impacts) are examined with regards to this media " frenzy " , and some comment and analysis provided. The application of the paper is in two parts, namely to first provide a review of this media coverage, and secondly to recommend project naming in AI with precise and realistic short term goals of achieving really useful machines, with specific smart components. An example of this is provided, namely the RUMLSM project, a novel AI/Machine Learning system proposed to resolve some of the known issues in bottom-up Deep Learning by Neural Networks, recognised by DARPA as the " Third Wave of AI. " An extensive, and up to date at the time of writing, Internet accessible reference set of supporting media articles is provided.

This article seeks to illuminate ways in which interality studies provides unique philosophical ways to engage Artificial Intelligence research and its implications. Rather than rejecting AI methodologies wholesale as replications of classic Cartesian dualism or embracing them via rationalist philosophical models, the article forwards an interological illumination of " machines " , " learning " , and human being-in-the-world in order to make sense of the " human-like " in AI. Although the algorithmic models of machine learning are computational, the intelligibility of their outcomes—like human knowing-how—cannot always be articulated by AI researchers. Rather than a problem to be overcome, the article suggests that herein lies the trace to be followed in future interalogical work done in forging a third, synergistic, way forward in human machine relations. [Reno Lauro. AI and the Posthuman (Mental) Ecology: Interological Illuminations. China Media Research 2017; 13(4): 70-76]. 8

Some people think that the natural evolution of intelligence is that we will all end up uploading our brains to computers, and then building out our hardware base so that we can become ultra intelligent.

In this paper I present two reasons for thinking that making your brain physically bigger, and achieving ultra intelligence that way, would be a better strategy.

Faut-il ou ne faut-il pas permettre l’avancement des nouvelles technologies? Où pouvons-nous tracer les limites du développement technologique?

Pour résoudre ces questions, nous prendrons comme point de départ les deux propositions les plus célèbres sur ce sujet: le transhumanisme et les bioconservateurs. En examinant les généalogies, les concepts et les valeurs de chaque idéologie, nous parviendrons à comprendre les principes qui constituent ses positions face à la technologie. D’une part, nous avons le transhumanisme de la main d’auteurs comme Hottois, Bostrom, Roux, cette idéologie est basée sur la théorie de la nature technologique évolutive de l’homme, à partir de laquelle il forge une philosophie optimiste de la technologie. D’un autre côté, les bioconservateurs comme Fukuyama et Sandel se méfient des bienfaits de la technologie en l’accusant d’être un mythe du progrès produit par les illusions de grandeur de l’homme.

  Ces deux positions seront ensuite soumises au cas de l’avenir dystopique de la Super-intelligence afin d’évaluer leur pertinence et leur efficacité face à une situation limite, autrement dit, on évaluera sa capacité à réguler la technologie face à son pouvoir d'éviter la destruction de l’humanité. Ce cas lance un défi à ces idéologies en intégrant des éléments comme l’incertitude sur la date exacte de l’apparition de la Superintelligence et l'existence des multiples voies qui peuvent nous conduire à sa création. Le résultat de cette évaluation nous permettra d’entrevoir les point fortes et les faiblesses de chaque idéologie dans la prise de décisions technologiques, ainsi que sa capacité à nous guider vers un avenir non dystopique.

Im Zusammenhang mit der Weiterentwicklung der KI und damit einhergehender technischer Konstruke stellt sich die Frage, wie sich Künstliche Intelligenz von menschlicher Intelligenz unterscheidet und was genau unter Intelligenz zu verstehen ist. Wird hier irgendwann eine eigenständige Art von bewusstem Leben entstehen, das nicht leibgebunden und körperbezogen ist? Ob und wie werden Autonomie und Intentionalität in künstlichen Intelligenzsystemen ausgeübt und Verantwortung adressiert? Welche theologischen Impulse gibt es zur Einordnung und kritischen Begleitung der KI? Die Beiträge dieses Buches geben Antworten in interdisziplinärer Perspektive, werfen aber auch neue Fragen auf.

Discussions about the possible consequences of creating superintelligence have included the possibility of existential risk , often understood mainly as the risk of human extinction. We argue that suffering risks (s-risks) , where an adverse outcome would bring about severe suffering on an astronomical scale, are risks of a comparable severity and probability as risks of extinction. Preventing them is the common interest of many different value systems. Furthermore, we argue that in the same way as superintelligent AI both contributes to existential risk but can also help prevent it, superintelligent AI can both be a suffering risk or help avoid it. Some types of work aimed at making superintelligent AI safe will also help prevent suffering risks, and there may also be a class of safeguards for AI that helps specifically against s-risks.

Abstract: In this paper we focus on the most efficacious AI applications for life extension and anti-aging at three expected stages of AI development: narrow AI, AGI and superintelligence. First, we overview the existing research and commercial work performed by a select number of startups and academic projects. We find that at the current stage of “narrow” AI, the most promising areas for life extension are geroprotector-combination discovery, detection of aging biomarkers, and personalized anti-aging therapy. These advances could help currently living people reach longevity escape velocity and survive until more advanced AI appears. When AI comes close to human level, the main contribution to life extension will come from AI integration with humans through brain-computer interfaces, integrated AI assistants capable of autonomously diagnosing and treating health issues, and cyber systems embedded into human bodies. Lastly, we speculate about the more remote future, when AI reaches ...

This paper assesses the ethical implications of existing Deep Learning technologies as well as potentially emerging general AI which matches or surpasses human intelligence. I dwell on the concept of responsibility and the conditions required for its application. Subsequently, I identify the potential risks of developing a superintelligent AI and show that conventional concepts of responsibility cannot adequately address these challenges. In the final segment, I contemplate how these problems may be accounted for.

A major approach to the ethics of artificial intelligence (AI) is to use social choice, in which the AI is designed to act according to the aggregate views of society. This is found in the AI ethics of " coherent extrapolated volition " and " bottom-up ethics ". This paper shows that the normative basis of AI social choice ethics is weak due to the fact that there is no one single aggregate ethical view of society. Instead, the design of social choice AI faces three sets of decisions: standing, concerning whose ethics views are included; measurement, concerning how their views are identified; and aggregation, concerning how individual views are combined to a single view that will guide AI behavior. These decisions must be made up front in the initial AI design— designers cannot " let the AI figure it out ". Each set of decisions poses difficult ethical dilemmas with major consequences for AI behavior, with some decision options yielding pathological or even catastrophic results. Furthermore, non-social choice ethics face similar issues, such as whether to count future generations or the AI itself. These issues can be more important than the question of whether or not to use social choice ethics. Attention should focus on these issues, not on social choice.

The Meca Sapiens Blueprint is a System Architecture to build conscious machines. A system based on the Meca Sapiens Blueprint will be a unique and autonomous entity that is self-aware and capable of intentional mutations. It will interact with humans as an independent and self-directed being. The Blueprint is complete (at the system architecture level) and ready for design and implementation. Its content is unique and entirely original. Using the Blueprint, a development team that is familiar with conventional software techniques can transform an autonomous agent into a conscious synthetic being. The Meca Sapiens Blueprint is primarily intended for software developers. However, its content is readily accessible to the informed layman. It will be of interest to anyone interested in Artificial Intelligence or concerned about the impacts of this emerging technology on society. By discovering the mechanisms that generate synthetic consciousness, all readers will also gain a renewed unde...

Człowiek czasem pyta: dlaczego ciągle tak mało wiem, dlaczego moje zdolności fizyczne są ograniczone, dlaczego popadam w choroby, dlaczego nie mogę być szczęśliwy? I ostatecznie – dlaczego moje życie nie może być doskonałe? Transhumaniści nie udzielą jednoznacznej odpowiedzi na zadane powyżej pytania, żywią jednak przekonanie, że dzięki racjonalnym metodom, nauce i zaawansowanym technologiom możliwe będzie udoskonalenie ludzkiego bytu.

Artificial intelligence (AI) experts are currently divided into " presentist " and " futurist " factions that call for attention to near-term and long-term AI, respectively. This paper argues that the presentist-futurist dispute is not the best focus of attention. Instead, the paper proposes a reconciliation between the two factions based on a mutual interest in AI. The paper further proposes a realignment to two new factions: an " intellectualist " faction that seeks to develop AI for intellectual reasons (as found in the traditional norms of computer science) and a " societalist faction " that seeks to develop AI for the benefit of society. The paper argues in favor of societalism and offers three means of concurrently addressing societal impacts from near-term and long-term AI: (1) advancing societalist social norms, thereby increasing the portion of AI researchers who seek to benefit society; (2) technical research on how to make any AI more beneficial to society; and (3) policy to improve the societal benefits of all AI. In practice, it will often be advantageous to emphasize near-term AI due to the greater interest in near-term AI among AI and policy communities alike. However, presentist and futurist societalists alike can benefit from each others' advocacy for attention to the societal impacts of AI. A reconciliation between the presentist and futurist factions can improve both near-term and long-term societal impacts of AI. 1. Introduction Artificial intelligence (AI) experts are—to generalize—of two minds about long-term AI, especially regarding the potential for radically transformative types of future AI such as artificial superintelligence (ASI) and brain emulation. One faction views long-term AI as a profound issue, warranting extensive attention and immediate action. The other faction views long-term AI as an unhelpful distraction from the more pressing near-term AI issues. The purpose of this paper is to explore the potential for reconciliation between these two seemingly divergent factions. A reconciliation would enable them to expend less energy debating each other and more energy on activities that both agree are positive. In particular, if there are certain activities that simultaneously address issues associated with both near-term and long-term AI, then there is no need to engage in a near-term vs. long-term AI debate, because either position yields the same prescriptions. I will refer to the two factions as the futurists and the presentists. Each can be said to have a core claim:

This paper discusses means for promoting artificial intelligence (AI) that is designed to be safe and beneficial for society (or simply " beneficial AI "). The promotion of beneficial AI is a social challenge because it seeks to motivate AI developers to choose beneficial AI designs. Currently, the AI field is focused mainly on building AIs that are more capable, with little regard to social impacts. Two types of measures are available for encouraging the AI field to shift more towards building beneficial AI. Extrinsic measures impose constraints or incentives on AI researchers to induce them to pursue beneficial AI even if they do not want to. Intrinsic measures encourage AI researchers to want to pursue beneficial AI. Prior research focuses on extrinsic measures, but intrinsic measures are at least as important. Indeed, intrinsic factors can determine the success of extrinsic measures. Efforts to promote beneficial AI must consider intrinsic factors by studying the social psychology of AI research communities. 1. Introduction The challenge of building technologies that are safe and beneficial for society is really two challenges in one. There is the technical challenge of developing safe and beneficial technology designs, and there is the social challenge of ensuring that such designs are used. The two challenges are interrelated. Motivating technologists to pursue safe and beneficial designs is itself a social challenge. Furthermore, motivating people to use safe and beneficial designs is made easier when the designs also have other attractive features such as low cost and ease of use; creating these features is a technical challenge. This paper is concerned with the social challenge. Specifically, the paper examines a range of approaches to motivating technologists to pursue safe and beneficial technology designs. The paper focuses on artificial intelligence (AI) technologies, including both near-term AI and the proposed future " strong " or " superintelligent " AI that some posit could bring extreme social benefits or harms depending on its design. Much of the paper's discussion also applies to other technologies. That AI has significant social impacts is now beyond question. AI is now being used in finance, medicine, military, transportation, and a range of other critical sectors. The impact is likely to grow over time as new technologies are adopted, such as autonomous vehicles and lethal autonomous weapons (unless the latter are banned or heavily restricted). The prospects for strong AI are controversial; this paper takes the position that the stakes are sufficiently high that it warrants careful attention even if the probability of achieving it appears to be low. Regardless, while the paper is motivated in part by the risk of strong AI, the insights are more general. 1 For brevity, the paper uses the term " beneficial AI " to refer to AI that is safe and beneficial for society. It also uses the term " promoting beneficial AI " to refer to efforts to encourage technologists to design and build beneficial AI, or to have them avoid designing and building AI that is not beneficial. The technologists include AI researchers/designers/developers (the paper uses these terms more or less interchangeably) as well as adjacent personnel in management, business development, etc.

Some emerging technologies promise to significantly improve the human condition, but come with a risk of failure so catastrophic that human civilization may not survive. This article discusses the great downside dilemma posed by the decision of whether or not to use these technologies. The dilemma is: use the technology, and risk the downside of catastrophic failure, or do not use the technology, and suffer through life without it. Historical precedents include the first nuclear weapon test and messaging to extraterrestrial intelligence. Contemporary examples include stratospheric geoengineering, a technology under development in response to global warming, and artificial general intelligence, a technology that could even take over the world. How the dilemma should be resolved depends on the details of each technology's downside risk and on what the human condition would otherwise be. Meanwhile, other technologies do not pose this dilemma, including sustainable design technologies, nuclear fusion power, and space colonization. Decisions on all of these technologies should be made with the long-term interests of human civilization in mind. This paper is part of a series of papers based on presentations at the event Emerging Technologies and the Future of Humanity held at the Royal Swedish Academy of Sciences, 17 March 2014.

Artificial superintelligence (ASI) is increasingly recognized as a significant future risk. In the absence of adequate safety mechanisms, an ASI may even be likely to cause human extinction. Thus ASI risk scenarios merit attention even if their probabilities are low. ASI risk can be addressed in at least two ways: by building safety mechanisms into the ASI itself, as in ASI safety research, and by managing the human process of developing ASI, in order to promote safety practices in ASI research and development (R&D). While ASI researchers and developers typically do not intend to cause harm through their work, harm may nonetheless occur due to accidents and unintended consequences. Thus opportunities may exist to reduce ASI risk through engagement with the R&D process. This paper surveys established methodologies for risk analysis and risk management, emphasizing fault trees and event trees, and describes how these techniques can be applied to risk from ASI R&D. A variety of risk methodologies have been developed for other risks, including other emerging technology risks, but their application to ASI has thus far been limited. Insights from risk literatures could improve on what existing analyses of ASI risk have yet been conducted. Likewise, a more thorough and rigorous analysis of ASI R&D processes can inform decision making to reduce the risk. The decision makers include governments and non-governmental organizations active in ASI oversight, as well anyone conducting ASI R&D. All of these individuals and groups have roles to play in addressing ASI risk.

Résumé :
  "D'après la thèse dite de la "clôture cognitive" (défendue notamment par Colin McGinn et Steven Pinker), il existe des limites à ce que l'humanité peut potentiellement connaître ou comprendre, en raison de contraintes naturelles inhérentes à la cognition humaine. La prise en compte de la naturalité de l'esprit humain devrait ainsi conduire à une forme de modestie épistémologique : nous ne pouvons pas tout connaître, ni tout comprendre de la nature, tout simplement parce que nous sommes des êtres vivants soumis à des contraintes évolutionnaires.
    Plutôt que de discuter les fondements (controversés) de cette thèse de la "clôture cognitive", je m'intéresserai ici à un retournement optimiste de ce courant de pensée. Chez certains auteurs, en effet, cette thèse a pour conséquence paradoxale une forme d'optimisme quant à ce que pourrait fournir les progrès technologiques futurs à l'égard de l'entreprise de la connaissance. Si certains problèmes nous paraissent aujourd'hui insolubles, à nous humains, peut-être seront-ils aisément solubles demain, lorsque la technologie permettra d'améliorer artificiellement le cerveau humain, ou encore de créer des intelligences artificielles supérieures à l'intelligence humaine (hypothèse de la "superintelligence") ? Certains auteurs vont ainsi jusqu'à suggérer que la philosophie, en tant que discipline se caractérisant par le traitement de problèmes "insolubles" par la science, pourrait disparaître en même temps que sa "matière première".
    Mon propos consistera en trois points principaux :
1) synthétiser rapidement les arguments habituellement formulés en faveur de la thèse de la "clôture cognitive"
2) analyser les arguments en faveur d'un optimisme épistémologique, si l'on admet la thèse de la "clôture cognitive"
3) interroger les tentatives d'application de cet optimisme à la philosophie elle-même, ainsi que sa caractérisation comme discipline traitant de problèmes "insolubles" par l'esprit humain."

Bibliographie indicative :
- Bostrom, Nick, Superintelligence (Oxford University Press, 2014)
- McGinn, Colin, The Mysterious Flame (Basic Books, 1999)
- Pinker, Steven, How the Mind Works (Penguin Books, 1997)
- Walker, Mark Alan, "Prolegomena to Any Future Philosophy", Journal of Evolution and Technology (Vol. 10, mars 2002)

"The Superintelligent Leviathan: An Outline of the Problem with Human Autonomy and the Autonomy of Devices"

The aim of this essay is to apply the concept of “Leviathan” by
Thomas Hobbes to the concept of superintelligence or superhuman
intelligence, which is discussed among transhumanists and touched
upon by such philosophers and futurologists as Nick Bostrom,
Stanisław Lem, or Ray Kurzweil, among others. The inspiration for
this work came from considering questions of the kind: "when does
a human being cease to be an autonomous subject?" or "can a human
being be independent at all?" That said, it seems to me that such questions may arise when we recognize a human being as a political
creature (a political zoon in the sense of Aristotle). For it is difficult
to imagine how to separate the notion of politics from a lonely person,
that is to say, in isolation from other people and without making
reference to or addressing them. I will briefly argue from the
perspective of an evolutionary historian. Karol Darwin, Samuel
Butler, George Dyson, and specifically Yuval Noah Harari are
researchers who have contributed to this understanding. I focus on
the work of the last two.

Celem niniejszej pracy jest zastosowanie wizji „Lewiatana” Thomasa
Hobbesa do koncepcji superinteligencji lub nadludzkiej inteligencji,
które dyskutowane jest wśród transhumanistów i poruszone jest
przez takich filozofów i futurologów jak między innymi Nick Bostrom,
Stanisław Lem, albo Ray Kurzweil. Inspiracją mojej pracy były
pytania w rodzaju: „kiedy człowiek przestaje być autonomicznym
podmiotem?” albo „czy człowiek w ogóle może być samodzielny?”.
Niemniej jednak wydaje mi się, że takie pytania mogą się pojawić
wtedy, kiedy człowieka rozpoznamy jako zwierzę polityczne (politikon
zoon w sensie Arystotelesowskim). Albowiem trudno wyobrażać sobie
wyłonienie pojęcia polityki wobec człowieka samotnego, to znaczy
w izolacji od innych ludzi i bez odniesienia się lub ustosunkowania się
do nich. Będę argumentował z perspektywy historyka ewolucyjnego.
Za badaczy posiadających wkład do takiego stanowiska można uznać
Karola Darwina, Samuela Butlera, George’a Dysona i szczególnie
Yuvala Noaha Harariego. Opieram się głównie na pracach dwóch
ostatnich.

Contrary to the prevailing pessimistic AI takeover scenarios, the theory of the Global Brain (GB) argues that this foreseen collective, distributed superintelligence is bound to include humans as its key beneficiaries. This prediction follows from the contingency of evolution: we, as already present intelligent forms of life, are in a position to exert selective pressures onto the emerging new ones. As a result, it is foreseen that the cognitive architecture of the GB will include human beings and such technologies, which will best prove to advance our collective wellbeing. This paper aims to nuance and problematize this forecast by offering a novel combination of several existing theories: Kauffmann's theory of adjacent possible, Lotman's concept of the semiosphere, Luhmann's theory of social systems, and Heylighen's theory of intelligence. The resulting framework allows for a reinterpretation of the history of the human species in a way which suggests that it may not be individual humans, but our social systems, who are the most advanced intelligence currently operating on Earth. Our unique social systems, emerging from as early as the Neolithic out of mutual interrelations of the occurrences of symbolic communication of humans, are argued to be capable of individuating into autonomous, intelligent agents. The resulting distributedness of the currently dominating form of intelligence might challenge the predicted cognitive architecture of the Global Brain, as it is likely to introduce additional powerful sources of selective pressures. Since the rapid evolution of interconnecting technologies appears to open up immense emancipatory possibilities not only for humans, but also for the intelligently evolving ‘creatures of the semiosphere’, it is concluded that in the context of the rapidly self-organizing Global Brain, a close watch needs to be kept over the dynamics of the latter.

Artificial intelligence (AI) is a growing social phenomenon, both as a technical infrastructure and as a sociotechnical imaginary. While there is ample sociological research on AI's implications for different societal sectors, there has been limited sociological discussion on AI itself. This article engages with the current debate on superintelligence-a future AI that transcends human intelligence-to show that the development of AI concerns central sociological issues, and that sociological thinking contributes to and challenges prevailing concepts of AI. ARtificiell intelligens (Ai) är ett område som fått allt större uppmärksamhet. Det finns stora förhoppningar knutna till dess utveckling och stat och näringsliv gör mycket stora satsningar. 2 Samtidigt finns en medvetenhet om att AI kan användas för destruktiva ändamål och därför behöver forskning och utveckling på detta område regleras. Dessa farhågor utgör även argument för att Europeiska unionen och Sverige bör satsa kraftfullt så att demokratiska värderingar styr denna teknikutveckling. 3 1 Jag vill tacka Monika Berg och Sverre Wide (Örebro universitet) samt tre granskare för kon-struktiva kommentarer på tidigare versioner av denna artikel. 2 Ett aktuellt svenskt exempel är Wallenbergstiftelsernas pågående satsning på grundforskning om AI och kvantteknologi (totalt drygt 4 miljarder kronor). I satsningen ingår även 660 miljoner för samhällsvetenskaplig och humanistisk forskning om utmaningar och effekter av AI och autonoma system. 3 I TV-programmet "De smarta maskinernas tid" (sänt i Vetandets värld på SVT den 27 augusti 2018), säger Sveriges näringsminister Mikael Damberg att "Ny teknik innebär inte med automatik att det är gott eller illa, det måste fyllas med värderingar och moral. Det kan vi i Europa och Sverige bidra med". Han oroas också över AI-forskningen i Kina och säger att "Kina har ett helt annat samhällssystem och syn på mänskliga rättigheter än Sverige. Vi måste därför fylla AI med moraliska värderingar, och sätta gränser". I samma program intervjuas AI-forskaren Max Tegmark som betonar att "det krävs krafttag av våra politiker, inte bara för att vi [inte] ska halka efter ekonomiskt utan också för att våra europeiska värderingar ska vara med att styra den här tekniken". https://doi.org/10.37062/sf.57.19591 Sociologisk Forskning, årgång 57, nr 2, sid 99-120. © Författaren och Sveriges Sociologförbund, ISSN 0038-0342, 2002-066X (elektronisk).

The following notes are speculative & philosophical, not futurological, in intent. The background question is how 'the human' might be thought given the ongoing development of artificial intelligence/life, synthetic biology and human enmeshment therewith.

The development of human-level AI has been a core goal of the AI field since its inception, though at present it occupies only a fraction of the field's efforts. To help understand the viability of this goal, this article presents an assessment of expert opinions regarding human-level AI research conducted at AGI-09, a conference for this AI specialty. We found that various experts strongly disagree with each other on certain matters, such as timing and ordering of key milestones. However, we did find that most experts expect human-level AI to be reached within upcoming decades, and all experts give at least some chance that some milestones will be reached within this time. Furthermore, a majority of experts surveyed favor an integrative approach to human-level AI rather than an approach centered on a particular technique. Finally, experts are skeptical about the impact of massive research funding, especially if it is concentrated in relatively few approaches. These results suggest that the possibility of achieving human-level AI in the near term should be given serious consideration.