This paper provides a survey of philosophical discussion of the "the Turing Test". In particular, it provides a very careful and thorough discussion of the famous 1950 paper that was published in Mind.

The Turing Test is claimed by many to be a way to test for the presence, in computers, of such ``deep'' phenomena as thought and consciousness. Unfortunately, attempts to build computational systems able to pass TT have devolved into shallow symbol manipulation designed to, by hook or by crook, trick. The human creators of such systems know all too well that they have merely tried to fool those people who interact with their systems into believing that these (...) systems really have minds. And the problem is fundamental: the structure of the TT is such as to cultivate tricksters. A better test is one that insists on a certain restrictive epistemic relation between an artificial agent A, its output o, and the human architect H of A – a relation which, roughly speaking, obtains when H cannot account for how A produced o. We call this test the ``Lovelace Test'' in honor of Lady Lovelace, who believed that only when computers originate things should they be believed to have minds. (shrink)

I have discussed the frame problem and the Turing test at length, but I have not attempted to spell out what I think the implications of the frame problem ...

Stuart M. Shieber’s name is well known to computational linguists for his research and to computer scientists more generally for his debate on the Loebner Turing Test competition, which appeared a decade earlier in Communications of the ACM. 1 With this collection, I expect it to become equally well known to philosophers.

The Turing test has been studied and run as a controlled experiment and found to be underspecified and poorly designed. On the other hand, it has been defended and still attracts interest as a test for true artificial intelligence (AI). Scientists and philosophers regret the test’s current status, acknowledging that the situation is at odds with the intellectual standards of Turing’s works. This article refers to this as the Turing Test Dilemma, following the (...) observation that the test has been under discussion for over seventy years and still is widely seen as either too bad or too good to be a valuable experiment for AI. An argument that solves the dilemma is presented, which relies on reconstructing the Turing test as a thought experiment in the modern scientific tradition. It is argued that Turing’s exposition of the imitation game satisfies Mach’s characterization of the basic method of thought experiments and that Turing’s uses of his test satisfy Popper’s conception of the critical and heuristic uses of thought experiments and Kuhn’s association of thought experiments to conceptual change. It is emphasized how Turing methodically varied the imitation game design to address specific challenges posed to him by other thinkers and how his test illustrates a property of the phenomenon of intelligence and suggests a hypothesis on machine learning. This reconstruction of the Turing test provides a rapprochement to the conflicting views on its value in the literature. (shrink)

The Turing Test, originally proposed as a simple operational definition of intelligence, has now been with us for exactly half a century. It is safe to say that no other single article in computer science, and few other articles in science in general, have generated so much discussion. The present article chronicles the comments and controversy surrounding Turing's classic article from its publication to the present. The changing perception of the Turing Test over the last (...) fifty years has paralleled the changing attitudes in the scientific community towards artificial intelligence: from the unbridled optimism of 1960's to the current realization of the immense difficulties that still lie ahead. I conclude with the prediction that the Turing Test will remain important, not only as a landmark in the history of the development of intelligent machines, but also with real relevance to future generations of people living in a world in which the cognitive capacities of machines will be vastly greater than they are now. (shrink)

Turing''s test has been much misunderstood. Recently unpublished material by Turing casts fresh light on his thinking and dispels a number of philosophical myths concerning the Turing test. Properly understood, the Turing test withstands objections that are popularly believed to be fatal.

In 1950, Alan Turing proposed his eponymous test based on indistinguishability of verbal behavior as a replacement for the question "Can machines think?" Since then, two mutually contradictory but well-founded attitudes towards the Turing Test have arisen in the philosophical literature. On the one hand is the attitude that has become philosophical conventional wisdom, viz., that the Turing Test is hopelessly flawed as a sufficient condition for intelligence, while on the other hand is the (...) overwhelming sense that were a machine to pass a real live full-fledged Turing Test, it would be a sign of nothing but our orneriness to deny it the attribution of intelligence. The arguments against the sufficiency of the Turing Test for determining intelligence rely on showing that some extra conditions are logically necessary for intelligence beyond the behavioral properties exhibited by an agent under a Turing Test. Therefore, it cannot follow logically from passing a Turing Test that the agent is intelligent. I argue that these extra conditions can be revealed by the Turing Test, so long as we allow a very slight weakening of the criterion from one of logical proof to one of statistical proof under weak realizability assumptions. The argument depends on the notion of interactive proof developed in theoretical computer science, along with some simple physical facts that constrain the information capacity of agents. Crucially, the weakening is so slight as to make no conceivable difference from a practical standpoint. Thus, the Gordian knot between the two opposing views of the sufficiency of the Turing Test can be cut. (shrink)

In this paper we discuss the views on the Turing test of four influential thinkers who belong to the tradition of analytic philosophy: Ludwig Wittgenstein, Noam Chomsky, Hilary Putnam and John Searle. Based on various beliefs about philosophical and/or linguistic matters, they arrive at different assessments of both the significance and suitability of the imitation game for the development of cognitive science and AI models. Nevertheless, they share a rejection of the idea that one can treat Turing (...) test as a test for “machine thinking.” This seems to stem from a concern for the proper use of language —one that is a fundamental methodological commitment of analytic philosophy. (shrink)

The main factor of intelligence is defined as the ability tocomprehend, formalising this ability with the help of new constructsbased on descriptional complexity. The result is a comprehension test,or C- test, which is exclusively defined in computational terms. Due toits absolute and non-anthropomorphic character, it is equally applicableto both humans and non-humans. Moreover, it correlates with classicalpsychometric tests, thus establishing the first firm connection betweeninformation theoretical notions and traditional IQ tests. The TuringTest is compared with the C- (...) class='Hi'>test and the combination of the two isquestioned. In consequence, the idea of using the Turing Test as apractical test of intelligence should be surpassed, and substituted bycomputational and factorial tests of different cognitive abilities, amuch more useful approach for artificial intelligence progress and formany other intriguing questions that present themselves beyond theTuring Test. (shrink)

An important lesson that philosophy can learn from the Turing Test and computer science more generally concerns the careful use of the method of Levels of Abstraction (LoA). In this paper, the method is first briefly summarised. The constituents of the method are “observables”, collected together and moderated by predicates restraining their “behaviour”. The resulting collection of sets of observables is called a “gradient of abstractions” and it formalises the minimum consistency conditions that the chosen abstractions must satisfy. (...) Two useful kinds of gradient of abstraction – disjoint and nested – are identified. It is then argued that in any discrete (as distinct from analogue) domain of discourse, a complex phenomenon may be explicated in terms of simple approximations organised together in a gradient of abstractions. Thus, the method replaces, for discrete disciplines, the differential and integral calculus, which form the basis for understanding the complex analogue phenomena of science and engineering. The result formalises an approach that is rather common in computer science but has hitherto found little application in philosophy. So the philosophical value of the method is demonstrated by showing how making the LoA of discourse explicit can be fruitful for phenomenological and conceptual analysis. To this end, the method is applied to the Turing Test, the concept of agenthood, the definition of emergence, the notion of artificial life, quantum observation and decidable observation. It is hoped that this treatment will promote the use of the method in certain areas of the humanities and especially in philosophy. (shrink)

The Turing Test is routinely understood as a behaviourist test for machine intelligence. Diane Proudfoot has argued for an alternative interpretation. According to Proudfoot, Turing’s claim that intelligence is what he calls ‘an emotional concept’ indicates that he conceived of intelligence in response-dependence terms. As she puts it: ‘Turing’s criterion for “thinking” is…: x is intelligent if in the actual world, in an unrestricted computer-imitates-human game, x appears intelligent to an average interrogator’. The role of (...) the famous test is thus to provide the conditions in which to examine the average interrogator’s responses. I shall argue that Proudfoot’s analysis falls short. The philosophical literature contains two main models of response-dependence, what I shall call the transparency model and the reference-fixing model. Proudfoot resists the thought that Turing might have endorsed one of these models to the exclusion of the other. But the details of her own analysis indicate that she is, in fact, committed to the claim that Turing’s account of intelligence is grounded in a transparency model, rather than a reference-fixing one. By contrast, I shall argue that while Turing did indeed conceive of intelligence in response-dependence terms, his account is grounded in a reference-fixing model, rather than a transparency one. This is fortunate, because, as an account of intelligence, the transparency model is arguably problematic in a way that the reference-fixing model isn’t. (shrink)

The Turing Test (TT), as originally specified, centres on theability to perform a social role. The TT can be seen as a test of anability to enter into normal human social dynamics. In this light itseems unlikely that such an entity can be wholly designed in an off-line mode; rather a considerable period of training insitu would be required. The argument that since we can pass the TT,and our cognitive processes might be implemented as a Turing (...) Machine(TM), that consequently a TM that could pass the TT could be built, isattacked on the grounds that not all TMs are constructible in a plannedway. This observation points towards the importance of developmentalprocesses that use random elements (e.g., evolution), but in these casesit becomes problematic to call the result artificial. This hasimplications for the means by which intelligent agents could bedeveloped. (shrink)

In 2014, widespread reports in the popular media that a chatbot named Eugene Goostman had passed the Turing test became further grist for those who argue that the diversionary tactics of chatbots like Goostman and others, such as those who participate in the Loebner competition, are enabled by the open-ended dialog of the Turing test. Some claim a new kind of test of machine intelligence is needed, and one community has advanced the Winograd schema competition (...) to address this gap. We argue to the contrary that implicit in the Turing test is the cooperative challenge of using language to build a practical working understanding, necessitating a human interrogator to monitor and direct the conversation. We give examples which show that, because ambiguity in language is ubiquitous, open-ended conversation is not a flaw but rather the core challenge of the Turing test. We outline a statistical notion of practical working understanding that permits a reasonable amount of ambiguity, but nevertheless requires that ambiguity be resolved sufficiently for the agents to make progress. (shrink)

The Turing Test is a verbal-behavioral operational criterion of artificial intelligence. If a machine can participate in question–and–answer conversation adequately enough to deceive an intelligent interlocutor, then it has intelligent information processing abilities. Robert M. French has argued that recent discoveries in cognitive science about subcognitive processes involving associational primings prove that the Turing Test cannot provide a satisfactory criterion of machine intelligence, that Turing's prediction concerning the feasibility of building machines to play the imitation (...) game successfully is false, and that the test should be rejected as ethnocentric and incapable of measuring kinds and degrees of nonhuman intelligence. But French's criticism is flawed, because it requires Turing's sufficient conditional criterion of intelligence to serve as a necessary condition. Turing's Test is defended against these objections, and French's claim that the test ought to be rejected because machines cannot pass it is deemed unscientific, resting on the empirically unwarranted assumption that intelligent machines are possible. (shrink)

The Turing Test blurs the distinction between a model and irrelevant) instantiation details. Modeling only functional modules is problematic if these are interconnected and cognitively penetrable.

The so-called Turing test, as it is usually interpreted, sets a benchmark standard for determining when we might call a machine intelligent. We can call a machine intelligent if the following is satisfied: if a group of wise observers were conversing with a machine through an exchange of typed messages, those observers could not tell whether they were talking to a human being or to a machine. To pass the test, the machine has to be intelligent but (...) it also should be responsive in a manner which cannot be distinguished from a human being. This standard interpretation presents the Turing test as a criterion for demarcating intelligent from non-intelligent entities. For a long time proponents of artificial intelligence have taken the Turing test as a goalpost for measuring progress. (shrink)

This paper discusses some difficulties in understanding the Turing test. It emphasizes the importance of distinguishing between conceptual and empirical perspectives and highlights the former as introducing more serious problems for the TT. Some objections against the Turingian framework stemming from the later Wittgenstein’s philosophy are exposed. The following serious problems are examined: 1) It considers a unique and exclusive criterion for thinking which amounts to their identification; 2) it misidentifies the relationship of speaking to thinking as that (...) of a criterion; 3) it neglects the “natural” course of the development in semantics. However, these considerations suggest only that it is problematic to label a successful chatbot as a “thinking entity” without further qualifications, but not necessarily and once and for all incorrect. Philosophy has only little to say about the technical possibility of creating such an effective program. (shrink)

Why did the plan of using zombie manufacture as a means of studying consciousness ever seem plausible? Why does it impress so many people today? The immediate reason surely lies in fascination with the Turing Test -- the suggestion that computer programs would be proved to be conscious if they managed to carry on conversations in a way that made them seem conscious to a naive observer.

In the last 20 years the Turing test has been left further behind by new developments in artificial intelligence. At the same time, however, these developments have revived some key elements of the Turing test: imitation and adversarialness. On the one hand, many generative models, such as generative adversarial networks, build imitators under an adversarial setting that strongly resembles the Turing test. The term “Turing learning” has been used for this kind of setting. (...) On the other hand, AI benchmarks are suffering an adversarial situation too, with a ‘challenge-solve-and-replace’ evaluation dynamics whenever human performance is ‘imitated’. The particular AI community rushes to replace the old benchmark by a more challenging benchmark, one for which human performance would still be beyond AI. These two phenomena related to the Turing test are sufficiently distinctive, important and general for a detailed analysis. This is the main goal of this paper. After recognising the abyss that appears beyond superhuman performance, we build on Turing learning to identify two different evaluation schemas: Turing testing and adversarial testing. We revisit some of the key questions surrounding the Turing test, such as ‘understanding’, commonsense reasoning and extracting meaning from the world, and explore how the new testing paradigms should work to unmask the limitations of current and future AI. Finally, we discuss how behavioural similarity metrics could be used to create taxonomies for artificial and natural intelligence. Both testing schemas should complete a transition in which humans should give way to machines—not only as references to be imitated but also as judges—when pursuing and measuring machine intelligence. (shrink)

The standard interpretation of the imitation game is defended over the rival gender interpretation though it is noted that Turing himself proposed several variations of his imitation game. The Turing test is then justified as an inductive test not as an operational definition as commonly suggested. Turing's famous prediction about his test being passed at the 70% level is disconfirmed by the results of the Loebner 2000 contest and the absence of any serious (...) class='Hi'>Turing test competitors from AI on the horizon. But, reports of the death of the Turing test and AI are premature. AI continues to flourish and the test continues to play an important philosophical role in AI. Intelligence attribution, methodological, and visionary arguments are given in defense of a continuing role for the Turing test. With regard to Turing's predictions one is disconfirmed, one is confirmed, but another is still outstanding. (shrink)