Turing’s Test, a Beautiful Thought Experiment

Abstract.

In the wake of large language models, there has been a resurgence of claims and questions about the Turing test and its value for AI, which are reminiscent of decades of practical “Turing” tests. If AI were quantum physics, by now several “Schrödinger’s” cats could have been killed. Better late than never, it is time for a historical reconstruction of Turing’s beautiful thought experiment. In this paper I present a wealth of evidence, including new archival sources, give original answers to several open questions about Turing’s 1950 paper, and address the core question of the value of Turing’s test.

Figure 1. Alan Turing (1912-1954). Photographs of Alan Turing, copyright The Provost and Scholars of King’s College Cambridge 2023. Archives Centre, King’s College, Cambridge, AMT/K/7/12. Reproduced with permission.

1. Introduction

In recent decades, the Turing test has been used as a practical experiment for publicity purposes and aptly criticized (Shieber, 1994a, b; Vardi, 2014), and has been the whipping boy of AI (Hayes and Ford, 1995), cognitive sciences and analytic philosophy (Shieber, 2004), and increasingly, with the rise of AI, the humanities and social sciences (Brynjolfsson, 2022). It is not uncommon for criticism coming from all these areas to take Turing’s test literally, assuming that he encouraged deception as a criterion and/or proposed a crucial experiment to establish the existence of machine intelligence.

Now, in the wake of large language models, science and technology outlets ask whether Turing’s test can be a ‘benchmark’ for AI (Biever, 2023), and whether it is ‘dead’ (Wells, 2023). Based on recent primary research (Gonçalves, 2023e, a, b, c, d), in this paper I present a mass of evidence, including newly discovered archival sources, and a new perspective on Turing’s test.

I address a few problems that we will keep track as we go along: ($P_{1}$) Why would Turing design a test based on imitation, which can be seen as encouraging deception? ($P_{2}$) Why to present multiple versions of a test, as opposed to a well-defined, controlled experiment? ($P_{3}$) Why gender imitation in a test of machine intelligence, and why conversation for an intelligence task? ($P_{4}$) Why are practical “Turing” tests circular, as Hayes and Ford (Hayes and Ford, 1995) claimed? Finally, the core problem, ($P_{5}$) What is the value of Turing’s test for AI?

Section 2 presents a reading of Turing’s 1950 paper. Section 3 introduces newly discovered archival sources and examines Turing’s concept of imitation, addressing $P_{1}$. Section 4 shows that his presentation of the imitation game fits “the basic method of thought experiments” (Mach, 1897), addressing $P_{2}$. Section 5 reconstructs the historical conditions of Turing’s proposal, addressing $P_{3}$. Section 6 draws a parallel with the history of “the most beautiful experiment in the history of science” (Palmieri, 2005), addressing $P_{4}$. Finally, Section 7 revisits the history of AI, addressing $P_{5}$, and Section 8 concludes the paper.

2. What is the Turing Test?

In 1950, Alan Turing (Fig. 1) published the second of his three seminal papers, ‘Computing machinery and intelligence’ (Turing, 1950). The text has 28 pages, divided in seven sections, §1-§7. Three main logical steps can be identified in his argument: the proposal (§1-§3, 3+ pp.), the science (§4-§5, 6 pp.), and the discussion (§6-§7, 18+ pp.).

The proposal sought to replace the question “Can machines think?,” which he considered “too meaningless to deserve discussion” (p. 442), with the imitation game. The purpose was to change the common meaning of the word ‘machine’ (e.g., a steam engine, a bulldozer) in light of the new mathematical science of ‘universal’ digital computing. The imitation game would allow for a grounded discussion of ‘machine’ and ‘thinking,’ seeking to expand the meaning of ‘thinking’ and detach it from the human species, much as the meaning of ‘universe’ was once detached from the Earth.

In 1950, one of the OED definitions of ‘machine’ was: “a combination of parts moving mechanically as contrasted with a being having life, consciousness and will $\ldots$ Hence applied to a person who acts merely from habit or obedience to a rule, without intelligence, or to one whose actions have the undeviating precision and uniformity of a machine.” Thus, by definition, common sense did not allow the meanings of ‘machine’ and ‘thinking’ to overlap. Despite Turing’s emphasis in his opening paragraph that he did not intend to discuss how these words were “commonly used” (p. 433), the hostility to his proposal can be seen from one of the first reactions, from a participant in the 1949 Manchester seminars, who quoted the above OED definition to appeal to common sense (Mays, 1952).

The new question, which Turing considered to have a “more accurate form” (Turing, 1950, p. 442), would be based on a vivid image, his “criterion for ‘thinking”’ (p. 436), which he called interchangeably the ‘imitation game’ and his ‘test.’ The new question is whether a machine, playing A, the deceiver, can imitate a woman, a man, a human being, or another machine, playing B, the assistant, in a remotely played conversation game, and deceive an average interrogator, playing C, the judge, about its machine condition.

However, the details and exact conditions of the imitation game as an experiment slipped through Turing’s text in a series of variations that defies interpretation. A structural reading of the text identifies four different conditions of the game with respect to players A-B, namely, man-woman (p. 433), machine-woman (p. 434), machine-machine (p. 441), and machine-man (p. 442). These different conditions relate to four variants of the “new” question that Turing posed to replace his “original” question (see Box 1). In addition to varying the genus/species (types) of the players, he also increased the storage and speed of the machine and provided it with a hypothetically appropriate program ($Q^{\prime\prime\prime}$), and suggested a base time for the interrogation session ($Q^{\prime\prime\prime\prime}$). Other seemingly relevant parameters were not mentioned, such as the number of interrogators used to arrive at a statistically sound conclusion, although their profile is mentioned — they should be “average” —, and later reiterated — they “should not be expert about machines.”

BOX 1
 
Turing’s various questions and conditions
 
$Q$: “I propose to consider the question, ‘Can machines think?’ ” (p. 433) $Q^{\prime}$: “What will happen when a machine takes the part of A in this [man-woman] game?’ Will the interrogator decide wrongly as often when the game is played like this [machine-woman] as he does when the game is played between a man and a woman? These questions replace our original, ‘Can machines think?’ ” (pp. 433-434) $Q^{\prime\prime}$: “There are already a number of digital computers in working order, and it may be asked, ‘Why not try the experiment straight away? It would be easy to satisfy the conditions of the game. A number of interrogators could be used, and statistics compiled to show how often the right identification was given.’ The short answer is that we are not asking whether all digital computers would do well in the game nor whether the computers at present available would do well, but whether there are imaginable computers which would do well.” (p. 436) $Q^{\prime\prime\prime}$: “It was suggested tentatively that the question [$Q$] should be replaced by [$Q^{\prime\prime}$] $\ldots$ But in view of the universality property we see that either of these questions is equivalent to this, ‘Let us fix our attention on one particular digital computer C. Is it true that by modifying this computer to have an adequate storage, suitably increasing its speed of action, and providing it with an appropriate programme, C can be made to play satisfactorily the part of A in the imitation game, the part of B being taken by a man?’ ” (p. 442) $Q^{\prime\prime\prime\prime}$: “I believe that in about fifty years’ time it will be possible to programme computers, with a storage capacity of about $10^{9}$, to make them play the imitation game so well that an average interrogator will not have more than 70 per cent. chance of making the right identification after five minutes of questioning.” (p. 442)

3. Imitation: from 1936 to 1950

Because the machine must imitate stereotypes of what it is not, Turing’s proposal has often been criticized for encouraging fakes and tricks. But this is part of the literal reading that Turing would have meant his test as a practical experiment about short conversations, gendered machines, and how to fine-tune them to fool average human interrogators. It misses the point of Turing’s irony (Gonçalves, 2023b) and that his notion of imitation in 1950 was largely in continuity with his 1936 paper (Turing, 1936), as hinted at in his 1947 lecture, and as newly discovered correspondence with the Mexican-Canadian computer pioneer Beatrice Worsley (1921-1972) helps to clarify (see Box 2).

In his letter to Worsley, Turing seems to be more interested in the relations between “the motions” of Turing machines and infinite computers, whose behavior can be non-periodic. Perhaps he thought of the living human brain as an infinite computer, in the sense that it has a continuous interface with its environment, which constantly intervenes and changes its logical structure. Now, the imitation game puts into empirical form the relation between digital computers, whose behavior is ultimately periodic, and the behavior of the human players. Can the behavior of their brains be approximated by a digital computer? Turing pursued this question. For his May 1951 broadcast, he wrote: “the view which I hold myself, that it is not altogether unreasonable to describe digital computers as brains $\ldots$ If it is accepted that real brains, as found in animals, and in particular in men, are a sort of machine it will follow that our digital computer, suitably programmed, will behave like a brain.”

Even if the human brain can only be compared to an infinite computer, could it not be simulated by a digital computer equipped with a sufficiently large memory? An excerpt of another newly discovered Turing letter to Worsley from mid-1951 can give more contour and provide further insight into Turing’s views (see Box 3).

BOX 2
 
Turing’s mathematical concept of imitation
 
Dear Miss Worsley,
I was interested in your work on the relation between computers and Turing machines. I think it would be better though if you could try and find a realtion [sic] between T machines and infinite computers, rahter [sic] than between finite T machines and computers. The relation that you suggest is rather too trivial. The fact is that the motions of either a finite T machine or a finite computer are ultimately periodic, and therefore any sequence computed by them is ultimately periodic. It is easy therefore in theory to make one imitate the other, though the size of the imitating machine will (if this technique is adopted) have to be of the order of the exponential of the size of the imitated machine. Probably your methods could prove that this exponential relation could be reduced to a multiplicative factor.
Yours sincerely, A. M. Turing

A highlight in this excerpt is Turing’s view that to the extent that the behavior of a neuron can be described as a stochastic process, it would be possible to “calculate random samples” of the mechanism that embodies the brain and then imitate it. An effective imitation of the brain by a machine would require knowledge of the anatomy and physiology of the brain to inspire an appropriate program, as well as much more storage and speed than was available to the Ferranti Mark I at the time (see Fig. 2). Another important element in the excerpt is Turing’s point that, even if a thinking machine is possible, the relation he has in mind is not one of identity but one of analogy: “It’ll just be another species of the thinking genus.”

An original answer to problem $P_{1}$, why design a test based on imitation, which can be seen as encouraging deception, is as follows. Actually, imitation was Turing’s fundamental principle of the new science of universal digital computing. He conceived his 1950 paper largely in continuity with his 1936 paper. Both were based on his core concepts of machine and imitation, i.e., what it takes for a machine to imitate another machine. A point of difference is that by 1950 he had generalized the machine architecture and how ‘universal’ imitation can be achieved. Using Turing’s 1948 language, universality could be achieved with an ‘organized’ machine (1936), or with an ‘unorganized’/‘learning’ machine (1948/1950). Whereas in 1936 the machine would be given an a priori, specified, and fixed table of instructions for each task, in 1950 it would also be able to perform a new task by changing its logical structure as a result of learning from experience, much as the brain does, “by changing its neuron circuits by the growth of axons and dendrites.”

The kind of forgery and trickery that occurs in publicity-oriented practical “Turing” tests has nothing to do with Turing’s 1950 proposal. In 1951, he warned: “It would be quite easy to arrange the [machine’s] experiences in such a way that they automatically caused the structure of the machine to build up into a previously intended form, and this would obviously be a gross form of cheating, almost on a par with having a man inside the machine.” The “human fallibility” that Turing encouraged the machine to show was not meant to be artificially introduced, but rather a by-product of learning from experience. This 1950 passage clarifies: “Another important result of preparing our machine for its part in the imitation game by a process of teaching and learning is that ‘human fallibility’ is likely to be omitted in a rather natural way, i.e., without special ‘coaching’ ” (Turing, 1950, p. 459). That is, for the machine to be a valid player, it must not be specially prepared for the test. This is one of the reasons why we have never seen a practical Turing test.

BOX 3
 
Simulation of neuron processes
 
Dear Miss Worsley, $\,\ldots$
I do not think you will be able to find any clue to essential differences between brains and computing machines (if there are any), in neuron behaviour. So long as what we know about a neuron can be embodied in the description of stochastic processes, the behaviour of any mechanism embodying such neurons can, in principle, be calculated by a suitable enlarged and speeded up Ferranti [Mark II] machine. More accurately I should say that one can calculate random samples of its behaviour. I think any attempt to draw any sharp line between what machine and brain can do will fail. I think it is largely a quantitative matter. Probably one needs immensely more storage capacity then [sic] we have got, and possibly more than we shall ever have. Perhaps we may have enough capacity, but just won’t find an appropriate programme. Naturally one won’t make a man that way ever. It’ll just be another species of the thinking genus.
Yours sincerely, A. M. Turing

Figure 2. Control console of Ferranti Mark I and a group with Turing’s secretary at the Computing Machine Laboratory, Sylvia Robinson, pretending to play chess with the machine, c. 1955. Courtesy of The University of Manchester.

4. The method of thought experiments

The various rhetorical questions Turing posed, $Q^{\prime}\ldots Q^{\prime\prime\prime\prime}$, to replace the original question, $Q$, can be generalized as follows (Gonçalves, 2023e):

Question $Q^{\star}$: could player A imitate intellectual stereotypes associated with player B’s type successfully (well enough to deceive player C), despite A and B’s physical differences?

It has been largely unnoticed that the various questions instantiating $Q^{\star}$ follow a case-control methodology, applied in two stages. At the more obvious intra-game level, A plays the case, and B plays the control. However, at the inter-game level, two variants set the case (machine-woman and the machine-man) and the other two set the control (man-woman and the machine-machine). While the first two are open, creating suspense around the test, the latter two are resolved as follows. It is known that a man (A) can possibly imitate gender stereotypes associated with a woman (B) to the point of deceiving an interrogator (C) despite their physical differences. This is the very premise of the parlor games that existed at the time. Further, regarding the machine-machine variant, it is also known that a digital computer (A), because of its universality property, as Turing explained in the paper (Turing, 1950, §§4, 5), can possibly imitate any discrete-state machine (B), despite their architectural differences.

We can now explore how Turing’s presentation of his test conforms to Ernst Mach’s reconstruction of “the basic method of thought experiments,” which is variation, continuously if possible. Mach is the author of perhaps the most classic text on thought experiments in the modern scientific tradition (Mach, 1897), in which he developed observations and insights based on a wealth of examples from the history of modern physics, mathematics, and common sense experience. He wrote: “By varying the conditions (continuously if possible), the scope of ideas (expectations) tied to them is extended: by modifying and specializing the conditions we modify and specialize the ideas, making them more determinate, and the two processes alternate” (p. 139). Mach illustrated his point with the process of discovery of universal gravitation (Mach, 1897, pp. 138-139):

A stone falls to the ground. Increase the stone’s distance from the earth, and it would go against the grain to expect that this continuous increase would lead to some discontinuity. Even at lunar distance the stone will not suddenly lose its tendency to fall. Moreover, big stones fall like small ones: the moon tends to fall to the earth. Our ideas would lose the requisite determination if one body were attracted to the other but not the reverse, thus the attraction is mutual and remains so with unequal bodies, for the cases merge into one another continuously $\ldots$ discontinuities are quite conceivable, but it is highly improbable that their existence would not have betrayed itself by some experience. Besides, we prefer the point of view that causes less mental exertion, so long as it is compatible with experience.

The conditions, i.e., the distance of the fall and the size of the stones, are continuously varied in the physicist’s mind and eventually stretched to the celestial scale. Reciprocally, the concept of a celestial body, such as the Earth or the Moon, becomes interchangeable with the concept of a stone, and quite unequal stones can then become mutually attracted. The cases continuously merge into one another, and a conceptual integration is established that connects near-earth bodies to celestial bodies under a unified concept.

Turing’s imitation game extended the scope of ideas and expectations established earlier in his 1936 paper, moving from machine-machine and restricted human-machine imitation in 1936 to more general human-machine imitation in 1950.

To understand this better, let us take a brief look at Turing’s 1948 report “Intelligent Machinery” (op. cit.). In section (§3) ‘Varieties of machinery,’ he noted: “All machinery can be regarded as continuous, but when it is possible to regard it as discrete it is usually best to do so.” A brain, he wrote, “is probably” a ‘continuous controlling’ machine, but in light of the digital nature of neural impulses, it “is very similar to much discrete machinery.” In section (§6) ‘Man as Machine,’ he referred to the imitation of “any small part of a man” by machines: “A great positive reason for believing in the possibility of making thinking machinery is the fact that it is possible to make machinery to imitate any small part of a man” (p. 420). In light of this, he argued: “One way of setting about our task of building a ‘thinking machine’ would be to take a man as a whole and to try to replace all the parts of him by machinery.” But Turing dismissed such a method as “altogether too slow and impracticable,” and later alluded to moral and aesthetic reasons as well.

We can now follow Turing’s use of the method of continuous variation in the design of his imitation tests. The essential question ($Q^{\star}$) Turing asks is whether the intellectual and cultural performances (the stereotypes) associated with woman, man, machine (the types) could be imitated, and thus softly transposed. Note that for any arbitrarily chosen type, say, a woman, further specific subtypes can be continuously conceived and considered as varied conditions of the imitation game: women having property $p$, women having subproperty $p^{\prime}\subset p$, and so on. For any two arbitrarily chosen types, a new type can be conceived, whether as a specialization or a modification (“any small part of a man”). Because concepts are fluid entities, there is an evolving continuum of levels and types.

The question across the various versions of the game can be posed this way: how does C’s perception of A’s performance against B’s performance change as the game’s conditions are (continuously) varied? Will it change if gendered verbal behavior is required as a subtype of human verbal behavior? Will it change if the machine’s hardware is increased and/or its learning program is modified? For Turing, there is no conceptual discontinuity among the various conditions that instantiate his thought experiment.

From 1948 to 1952, Turing presented various imitation tests based on both the game of chess and conversation. A historically sound problem, because it does not struggle with the materiality of Turing’s texts and their chronological coherence, does not erase some of his tests in favor of others, does not ignore the historical conditions of his proposal, is ($P_{2}$) Why would Turing present multiple versions of his test, as opposed to a well-defined, controlled experiment? I have presented an answer by reconstructing Turing’s use of “the method of thought experiments,” and this will be reinforced in the next section. Turing’s purpose in his 1950 paper was not to propose a ‘benchmark’ for the non-existent field of AI, but to respond to critics. Especially in 1949, he felt compelled to point out that the new science of universal digital computing would eventually have an impact and expand our view of ‘thinking.’

5. 1949, the crucial year

As is often the case with thought experiments, Turing proposed his test out of a controversy (Gonçalves, 2023d). He was coming from his continuing disputes with the physicist and computer pioneer, Fellow of the Royal Society (FRS), Douglas Hartree (1897-1958), over the meaning of the newly existing digital computers, which had started in 1946 (Gonçalves, 2023c). Now, in mid-1949, new opponents had arrived, notably the neurosurgeon Geoffrey Jefferson (1886-1961), and the chemist and philosopher Michael Polanyi (1891-1976), both also FRS and based at the same institution as Turing, the University of Manchester, where Turing had spent a year as a Reader in the Department of Mathematics (Hodges, 1983). These three thinkers challenged Turing’s claims about the future possibilities and limitations of digital computers.

In June 1949, Hartree published his Calculating Instruments and Machines (Hartree, 1949), in which Ada Lovelace’s work was acknowledged seemingly for the first time by a twentieth-century computer pioneer (Gonçalves, 2023c). Since November 1946, Hartree had been opposing the use of the term ‘electronic brain.’ He wrote in a letter to the Times: “These machines can only do precisely what they are instructed to do by the operators who set them up.” Now in 1949, Hartree added strength to his argument by quoting the words of Ada Lovelace from the 1840s about Charles Babbage’s machine: “The Analytical Engine has no pretensions to originate anything $\ldots$ It can do whatever we know how to order it to perform” (her emphasis) (Hartree, 1949, p. 70). Noting Hartree’s anachronism in taking Lovelace’s words out of their time and place, Turing further developed his earlier, 1947 response to Hartree’s challenge, now calling it ‘(6) Lady Lovelace’s objection’ (Turing, 1950, p. 450). Turing argued that intelligent behavior is the result of learning, a capability he had no problem attributing to future digital computers. He also questioned the implicit assumption of Hartree’s challenge: “Who can be certain that ‘original work’ that he has done was not simply the growth of the seed planted in him by teaching, or the effect of following well-known general principles” (p. 450). In the imitation game, Turing suggested, the interrogator would be able to evaluate the machine’s ability to learn: “The game (with the player B omitted) is frequently used in practice under the name of viva voce to discover whether some one really understands something or has ‘learnt it parrot fashion’ ” (p. 446). But then we might ask, what is player B doing in the imitation game? Following the 1949 events will suggest an answer.

On June 9, in London, Jefferson delivered his prestigious Lister Oration on ‘The Mind of Mechanical Man,’ which was published in the debuting British Medical Journal on June 25 (Jefferson, 1949). His lecture was headlined in the Times on June 10, emphasizing his claim that “Not until a machine can write a sonnet or compose a concerto because of thoughts and emotions felt, and not by the chance fall of symbols, could we agree that machine equals brain” (p. 1110). This rendered Turing’s ironic response: “I do not think you can even draw the line about sonnets, though the comparison is perhaps a little bit unfair because a sonnet written by a machine will be better appreciated by another machine.” In October and December 1949, two seminars on ‘Mind and Machine’ were organized by Polanyi et al., and attended by Jefferson, Turing et al., at the Philosophy Department in Manchester (Polanyi, 1958, p. 275; cf. also note 1 above). These seminar discussions, followed by Jefferson giving Turing an offprint of his Lister Oration, which Turing read and marked with a pencil, led him to write his 1950 paper and propose his test (Gonçalves, 2023d).

In his Lister Oration (Jefferson, 1949), Jefferson had characterized intelligence as an emergent property of the animal nervous system. He emphasized that “sex hormones introduce peculiarities of behaviour often as inexplicable as they are impressive” (p. 1107). Because “modern automata” are not moved by male and female sex hormones, they could not exhibit such peculiarities to imitate the actions of animals or ‘men.’ Specifically, he used a thought experiment to criticize Grey Walter’s mechanical turtles by suggesting that gendered behavior is causally related to the physiology of sex hormones (ibid.):

[…It] should be possible to construct a simple animal such as a tortoise (as Grey Walter ingeniously proposed) that would show by its movements that it disliked bright lights, cold, and damp, and be apparently frightened by loud noises, moving towards or away from such stimuli as its receptors were capable of responding to. In a favourable situation the behaviour of such a toy could appear to be very lifelike — so much so that a good demonstrator might cause the credulous to exclaim ‘This is indeed a tortoise.’ I imagine, however, that another tortoise would quickly find it a puzzling companion and a disappointing mate.

Jefferson thus brought forward the image of a genuine individual of a kind, which is placed side by side with the artificial one so that the latter’s artificiality is emphasized. The function of the genuine individual is to expose the artificiality of the impostor. The means of exposure is to fail at demonstrating interesting (sexual) behavior. This can explain Turing’s introduction of a (gendered) control player B, who appears in Turing’s 1950 test, whose design was prompted by his reading of Jefferson, but not in Turing’s 1948, 1951, and 1952 tests. In discussing “(4) The Argument from Consciousness,” Turing addressed Jefferson directly and quoted in full his conditions for agreeing “that machine equals brain,” including “be warmed by flattery” and “be charmed by sex” (Turing, 1950, pp. 445-446). In discussing the “(5) Argument from Various Disabilities,” Turing again mentioned Jefferson and argued that to say that a machine could never “fall in love” or “make someone fall in love with it” was a flawed scientific induction from the capabilities of present machines.

Thus, an answer to the first part of problem $P_{3}$, why gender imitation in a test of machine intelligence, is that Turing’s design of his test was an ironic response to Jefferson’s association of sex and gender with intelligence, particularly his suggestion that gendered behavior is causally related to the physiology of male and female sex hormones. It remains to address the second part of problem $P_{3}$, why Turing chose conversation as the intelligence task of his test.

Surviving minutes of the ‘Mind and Machine’ seminar held on October 27, 1949, were published in 2000 by a participant, Wolfe Mays. In the first session, Polanyi presented a statement, ‘Can the mind be represented by a machine?,’ which was a Gödelian argument that humans can do things that machines cannot. Although Turing had already addressed this argument in his 1947 lecture (op. cit.), Polanyi’s insistence may help explain Turing’s inclusion of “(3) The Mathematical Objection” (Turing, 1950). Further, the minutes (op. cit.) show that Polanyi tried to distinguish the formal “rules of the logical system” from the informal “rules which determine our own behaviour,” and this can explain Turing’s inclusion of “(8) The Argument from Informality of Behaviour” (Turing, 1950).

Years later (Polanyi, 1958, p. 275), Polanyi remembered “a communication to a Symposium held on ‘Mind and Machine’ at Manchester University in October, 1949,” in which “A.M. Turing has shown that it is possible to devise a machine which will both construct and assert as new axioms an indefinite sequence of Gödelian sentences.” Polanyi resumed, showing that he assimilated the punch: “Any heuristic process of a routine character—for which in the deductive sciences the Gödelian process is an example—could likewise be carried out automatically.” However, Polanyi used the same argument to dismiss the game of chess as a testbed for machine intelligence, noting: “A routine game of chess can be played automatically by a machine, and indeed, all arts can be performed automatically to the extent to which the rules of the art can be specified.”

Chess, not conversation, had been Turing’s chosen field to illustrate, develop, and test machine intelligence since at least February 1946. In his 1948 ‘Intelligent Machinery’ report (op. cit.), Turing had discussed a tradeoff between convenient and impressive intellectual fields for exploring machine intelligence. After discussing “various games e.g. chess,” Turing wrote: “Of the above possible fields the learning of languages would be the most impressive, since it is the most human of these activities.” However, he avoided language learning because it seemed “to depend rather too much on sense organs and locomotion to be feasible,” stuck with chess, and ended up describing a chess-based imitation game. Now in October 1949, he saw chess being dismissed as an unimpressive to make the case for machine intelligence because its rules could be specified.

Some time later, probably around Christmas 1949, Turing will read Jefferson’s Lister Oration (Jefferson, 1949) and mark the passage quoting René Descartes (p. 1106), which starts: “Descartes made the point, and a basic one it is, that a parrot repeated only what it had been taught and only a fragment of that; it never used words to express its own thoughts.” Overall, Jefferson suggested ‘speech’ to be the distinguishing feature of human intelligence compared to other kinds of animal intelligence: “Granted that much that goes on in our heads is wordless $\ldots$ we certainly require words for conceptual thinking as well as for expression $\ldots$ It is here that there is the sudden and mysterious leap from the highest animal to man, and it is in the speech areas of the dominant hemisphere $\ldots$ that Descartes should have put the soul, the highest intellectual faculties” (p. 1109).

Unlike chess, which is governed by definite rules, good performance in conversation was thought to be reserved for humans. So Turing’s 1950 change to “the learning of languages” as the intellectual field addressed by his test can best be understood as yet another concession to Jefferson, and in this case to Polanyi as well.

In summary, Turing varied the design of his imitation tests to respond to the challenges posed by Hartree, Polanyi, and Jefferson. This fits neatly into Popper’s methodological rule for “the use of imaginary experiments in critical argumentation” (Popper, 1959): “the idealizations made must be concessions to the opponent, or at least acceptable to the opponent” (p. 466, Popper’s emphasis).

6. A hammer and a feather on the moon

On August 2, 1971, more than three centuries after Galileo’s death, a live anecdotal demonstration of Galileo’s legendary tower experiment was performed for the television cameras by astronaut David Scott during the final Apollo 15 moonwalk (Fig. 3). Far from the Earth’s atmosphere, essentially in a vacuum, the astronaut simultaneously released a heavy object (an aluminum geological hammer) and a light object (a falcon feather) from approximately the same height, which fell to the ground at the same rate to the naked eye. The performer, who attributed their successful mission in part to “a rather significant discovery about falling objects in gravity fields” made long ago by “a gentleman named Galileo,” celebrated: “How about that! Mr Galileo was correct in his findings.”

Variants of Galileo’s falling-bodies experiment first appeared in his De motu drafts written in the 1590s. Decades later came the punchy presentation of his 1638 Two New Sciences (Galilei, 1638, pp. 66-67):

SALVIATI: But without experiences, by a short and conclusive demonstration, we can prove clearly that it is not true that a heavier moveable is moved more swiftly than another, less heavy, these being of the same material, and in a word, those of which Aristotle speaks. Tell me, Simplicio, whether you assume that for every heavy falling body there is a speed determined by nature such that this cannot be increased or diminished except by using force or opposing some impediment to it $\ldots$ [SIMPLICIO agrees]
Then if we had two moveables whose natural speeds were unequal, it is evident that were we to connect the slower to the faster, the latter would be partly retarded by the slower, and this would be partly speeded up by the faster $\ldots$

Figure 3. Alan Bean’s painting “The Hammer and the Feather,” 1986. Reproduced with permission. Source: http://alanbeangallery.com/hammerfeather-story.html.

From the outset, Galileo makes Simplicio accept Salviati’s carefully formulated assumption that for every heavy falling body there is a natural speed that cannot be altered except by external intervention. However, the Aristotelian could have found a way out by noting the imprecision and denying that weight and natural speed are physically determinate for connected but not unified bodies (Gendler, 1998).

Could Galileo have run a variant of this experiment from the Leaning Tower of Pisa and obtained the results claimed in the story? Physicists went to the laboratory and concluded that it is doubtful (Adler and Coulter, 1978). Decades earlier, it was found (Cooper, 1935) that the only source for the legendary story was actually an apologetic biography written by Galileo’s disciple Vincenzo Viviani 12 years after Galileo’s death and first published in 1717. For centuries, Galileo was largely considered “the first true empiricist,” but the role of experiment in Galilean science was more complex than previously thought (Segre, 1989; Palmieri, 2005).

Galileo’s falling-bodies experiment suggests an anomaly in Aristotle’s theory of motion under certain idealized conditions, the existence of motion in a void, which was unacceptable at the time. Testing such an existential hypothesis was in infinite regress with the conditions it required, and creating those conditions would require long-term scientific and technological progress.

The Galilean impasse could only be broken by the power of his thought experiments as propaganda for a next generation of scientists (Gonçalves, 2023a). Robert Boyle (1627-1692) was one of them. As a teenager, he visited Florence shortly before Galileo’s death and was impressed by “the new paradoxes of the great star-gazer Galileo” (Fulton, 1960, p. 119). In his career, Boyle built air pumps and special chambers to study vertical fall in small evacuated environments, and became an exponent of experimental philosophy in the Royal Society. There is no record of Boyle interpreting Galilean science literally and performing any tower experiments. The value of Galileo’s thought experiments was to lay conceptual foundations and to conjecture a class of idealized phenomena to be pursued by progressive science and technology. There is a path that connects Galileo’s thought experiments and Boyle’s vacuum chambers to the space programs of the 1950s and finally to the anecdotal confirmation of Galileo’s hypothesis by the crew of the Apollo 15 mission to the moon.

Now note the analogy with the Turing test (Gonçalves, 2023a). As Hayes and Ford claimed: “The tests are circular: they define the qualities they are claiming to be evidence for” (p. 974). Turing’s existential hypothesis of a Turing-test-passing machine is in infinite regress with the conditions assumed by the test: an idealized computer equipped with a hypothetically appropriate program (Box 1). But if these conditions exist, why do we need a Turing test at all? This shows that practical Turing tests can serve at best as anecdotal confirmation of Turing’s hypothesis, and at worst, as we have seen for decades, as publicity stunts. The value of Turing’s test must lie elsewhere.

7. The value of Turing’s test for AI

By May 1953, John McCarthy and Claude Shannon were working on their collection Automata Studies (McCarthy and Shannon, 1956), which revolved around “the theory of Turing machines” (p. vii), and to which they invited Turing to contribute. Turing declined the invitation, saying that he had been working for the last two years on “the mathematics of morphogenesis,” although he expected “to get back to cybernetics very shortly.” One year and four days later, Turing was dead, and early AI would not note his biological turn. Commenting on “the Turing definition of thinking” (p. vi), McCarthy and Shannon found it “interesting” because it “has the advantages of being operational or, in the psychologists’ term, behavioristic $\ldots$ No metaphysical notions of consciousness, ego and the like are involved.” They also thought that this very strength could be a weakness, because it has “the disadvantage” of being susceptible to a memorizing machine playing the imitation game by looking up “a suitable dictionary.”

McCarthy and Shannon referred interchangeably to ‘definition’ and to a word that Turing actually used, ‘criterion:’ “While certainly no machines at the present time can even make a start at satisfying this rather strong criterion, Turing has speculated that within a few decades it will be possible to program general purpose computers in such a way as to satisfy this test” (McCarthy and Shannon, 1956, p. v, emphasis added).

In 1955, before the publication of Automata Studies, McCarthy and Shannon, together with Marvin Minsky and Nathaniel Rochester, co-authored their well-known ‘Proposal’ for AI research (McCarthy et al., 1955). Unlike Turing himself, they seem to have thought of machine intelligence in terms of Turing machines, as their opening paragraph suggests: “The study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.” Whether or not they followed Turing on the ‘how,’ they certainly followed him on the ‘what,’ in writing: “For the present purpose the artificial intelligence problem is taken to be that of making a machine behave in ways that would be called intelligent if a human were so behaving” (p. 7). This definition — compare it with “the Turing definition of thinking” — would stay.

In the early 1960s, Edward Feigenbaum and Julian Feldman noted in Computers and Thought (Feigenbaum and Feldman, 1963) that Turing’s 1950 paper “appeared five years before concrete developments in intelligent behavior by machine began to occur;” and “yet,” they continued, “it remains today one of the most cogent and thorough discussions in the literature on the general question “Can a machine think?” (pp. 9-10). They observed Turing’s “behavioristic posture relative to the question,” which “is to be decided by an unprejudiced comparison of the alleged ‘thinking behavior’ of the machine with normal ‘thinking behavior’ in human beings” (emphasis added). They concluded: “He proposes an experiment — commonly called ‘Turing’s test’ — in which the unprejudiced comparison could be made $\ldots$ Though the test has flaws, it is the best that has been proposed to date.”

Minsky, in the preface to his 1967 collection (Minsky, 1968), reiterates the definition of AI as “the science of making machines do things that would require intelligence if done by men” (p. v). Around the same time, Minsky collaborated with Stanley Kubrick and Arthur Clarke on their 1968 screenplay, also written as a novel, 2001: A Space Odyssey (Clarke, 1968), which featured a futuristic computer named HAL:

Whether HAL could actually think was a question which had been settled by the British mathematician Alan Turing back in the 1940s. Turing had pointed out that, if one could carry out a prolonged conversation with a machine — whether by typewriter or microphone was immaterial — without being able to distinguish between its replies and those that a man might give, then the machine was thinking, by any sensible definition of the word. HAL could pass the Turing test with ease.

The “Turing definition of thinking” was to become legendary.

Stuart Shieber studied McCarthy and Shannon’s memorizing machine objection in depth, elaborated on its assumptions, and concluded that it is invalid (Shieber, 2014). But McCarthy’s concept of ‘memorizing’ may have been more elastic, as his later comment on Deep Blue’s defeat of Gary Kasparov seems to suggest (McCarthy, 1997). He expressed disappointment that it was mostly an achievement of computational power rather than thinking, and gave a clear argument why he thought so. Essentially, McCarthy pointed out, computer chess advanced by replacing heuristic techniques, which relied on the expertise of human players to prune the search space of possible moves, with brute force computing. “[I]t is a measure of our limited understanding of the principles of artificial intelligence,” McCarthy wrote, “that this level of play requires many millions of times as much computing as a human chess player does.” It may be, but that the problem was “largely a quantitative matter” was hinted at by Turing in his letter to Worsley of c. June 1951 (Box 3).

Ten years after Deep Blue vs. Kasparov, McCarthy referred to Turing’s 1947 lecture (op. cit.) as “the first scientific discussion of human level machine intelligence,” and to Turing’s 1950 paper as “amplifying” that discussion into a “goal” (McCarthy, 2007, p. 1174).

In 1992, Minsky co-authored a work of fiction, The Turing Option (Warner, New York), in which Turing’s test is featured in the preface. In 1995, Minsky took a stand against Loebner’s Weinzenbaum experiments, pleading to “revoke his stupid prize, save himself some money, and spare us the horror of this obnoxious and unproductive annual publicity campaign.” In 2013, when asked about the Turing test in a taped interview, Minsky said: “The Turing test is a joke, sort of, about saying ‘A machine would be intelligent if it does things that an observer would say must be being done by a human’ $\ldots$ it was suggested by Alan Turing as one way to evaluate a machine but he had never intended it as being the way to decide whether a machine was really intelligent.” This materially connects McCarthy et al.’s definition of “the AI problem” with Turing’s test, if material evidence were still needed.

Overall, it seems that all of these AI pioneers understood and were inspired by Turing’s test at the level of conceptual foundations. Even if some of them also used the term ‘experiment,’ none of them took it literally as a practical experiment, which would indeed imply an astonishing lack of imagination on their part. Turing’s test moved the burgeoning field of AI away from unproductive debates about the meaning of words, for example, allowing Minsky to write in 1967 (Minsky, 1967): “Turing discusses some of these issues in his brilliant article, ‘Comput­ing Machines and Intelligence” [sic], and I will not recapitulate his arguments $\ldots$ They amount, in my view, to a satisfactory refutation of many such objections” (p. 107).

The value of Turing’s test ($P_{5}$) is that it has long been and still is a unifying ‘definition,’ a ‘criterion,’ a ‘goal’ for, in the words of McCarthy et al., the science and engineering of “making a machine behave in ways that would be called intelligent if a human were so behaving.” Every time AI succeeds in automating a new task that was once reserved for humans because it requires intelligence, “the Turing definition” conquers new territory, and the significance of Turing’s early message to his contemporaries becomes clearer.

8. Conclusion

In this paper, I presented a mass of evidence, including newly discovered archival sources, and a new perspective on Turing’s test.

New light was shed on Turing’s concept of imitation, emphasizing that it does not encourage deception in AI. Rather, it is a mathematical concept, largely in continuity with his 1936 paper. I also showed that Turing’s presentation of the various versions of his test fits what Mach characterized as “the basic method of thought experiments” in science. I reconstructed the historical conditions of Turing’s proposal, explaining that gender imitation was his ironic response to Jefferson, and conversation was yet another concession to his opponents. Further, I compared Turing’s test to Galileo’s falling-bodies experiment, showing that the problem of circularity is inherent in existential hypotheses, and that the solution may lie in propaganda and the progressive scientific and technological developments of a next generation of scientists. I then revisited the history of AI, showing that Turing’s test provided McCarthy, Minsky, and others with a definition of the AI problem that, at the level of conceptual foundations, still drives AI research today.

But whatever its utility, Turing’s test has secured its place as one of the most beautiful thought experiments in the history of science.