A short history of AI, part 1: from Turing to Deep Blue (1936–1997)

My recent posts have been about how to use today’s AI tools well: what they do to our writing, how to catch their errors, and when vibe coding can be trusted. Several students asked a more basic question: how did we get here? This is the first of three posts on that question. This one covers the sixty years from Alan Turing’s paper machine to the computer that beat the world chess champion (1936–1997). The second will cover the turn to data and deep learning (1997–2017), and the third the transformer and the large language models of today (2017–2026).

History is worth the detour for a practical reason. Many of the debates about AI in 2026 (whether a machine can think, whether rules or learning is the better path, how to tell real progress from hype, and how to measure intelligence at all) were first argued in the period covered here. Knowing how those arguments went the first time makes the current ones easier to judge.

Figure 1. Milestones in AI from 1936 to 1997, discussed in this post. Rows are evenly spaced rather than drawn to scale in time. The two “AI winters,” periods of reduced funding and interest, are shown with approximate dates because historians date them differently. Figure by Daeha Kim, based on the sources cited below.

A machine on paper (1936–1950)

The story usually starts with a mathematician, not a computer. In 1936 Alan Turing, then 24, wanted to answer a question in logic: is there a mechanical procedure that can decide whether any given mathematical statement is provable? To make “mechanical procedure” precise, he imagined a machine that reads and writes symbols on a tape according to a short table of rules. He then showed that a single “universal” machine could imitate any other such machine if given its rule table as input (Turing 1937). That is, in essence, the idea of the stored-program computer, described a decade before one was built.

Seven years later, the neurophysiologist Warren McCulloch and the logician Walter Pitts proposed that a nerve cell could be modeled as a simple logical unit that fires when the sum of its inputs passes a threshold, and that networks of such units could compute logical functions (McCulloch & Pitts 1943). Their paper is the ancestor of every neural network in use today.

In 1950, after his wartime codebreaking work, Turing returned to the question of thinking machines in the journal Mind. He judged the question “Can machines think?” too vague to answer, and replaced it with a game: an interrogator exchanges typed messages with a person and a machine and tries to tell which is which (Turing 1950). Turing predicted that in about fifty years a computer with a memory of about 10⁹ bits would play the game well enough that an average interrogator would have no better than a 70% chance of identifying it correctly after five minutes of questioning. He also suggested that the most practical route to such a machine might be to build a “child machine” and educate it rather than program it directly, an idea that anticipates machine learning by decades. Turing died in 1954 and did not see the field he had imagined take shape.

Naming the field (1955–1969)

The phrase “artificial intelligence” first appeared in a funding proposal dated 31 August 1955. John McCarthy, Marvin Minsky, Nathaniel Rochester and Claude Shannon asked the Rockefeller Foundation to support a two-month summer study at Dartmouth College, based on the conjecture that every aspect of learning and intelligence could in principle be described precisely enough for a machine to simulate it. They expected that a carefully chosen group working together for one summer could make a significant advance (McCarthy et al. 2006). The workshop took place in the summer of 1956. It did not produce the advance, but it gave the field its name and its founding generation.

Progress in the following years was real and quick. Allen Newell and Herbert Simon’s Logic Theorist proved theorems from Whitehead and Russell’s Principia Mathematica by searching through possible chains of reasoning (Newell & Simon 1956). Arthur Samuel’s checkers program improved by playing against itself and remembering which positions led to wins; his 1959 paper helped popularize the term “machine learning” (Samuel 1959). Frank Rosenblatt’s perceptron took McCulloch and Pitts’s artificial neuron one step further: instead of having its connection weights set by hand, it adjusted them automatically from examples until it classified them correctly (Rosenblatt 1958). Rosenblatt built it in hardware, and the press coverage was enthusiastic.

The most famous program of the decade was also the simplest. Joseph Weizenbaum’s ELIZA imitated a psychotherapist by matching keywords in the user’s sentences and turning them into questions (Weizenbaum 1966). It understood nothing, and Weizenbaum said so, yet many users confided in it as if it did. He was disturbed enough by this to write a book arguing that some tasks should never be handed to computers (Weizenbaum 1976). Sixty years later, the tendency he observed, our readiness to attribute understanding to fluent text, is one of the main themes of my previous post on writing with LLMs.

Limits, and the first winter (1969–1980)

In 1969 Minsky and Seymour Papert published Perceptrons, a mathematical analysis of what a single layer of Rosenblatt-style neurons could and could not compute (Minsky & Papert 1969). The best-known limitation is easy to draw. A single perceptron separates its inputs with one straight line (or plane), so it can learn AND and OR but not XOR, which outputs 1 when exactly one of its two inputs is 1. Adding a hidden layer of neurons solves the problem, but in 1969 nobody had a practical way to train the weights of such hidden layers.

Figure 2. AND and OR can be computed by a single perceptron because one straight line separates the inputs that should give 1 from those that should give 0. For XOR no such line exists. Two hidden units, each drawing one line, solve the problem; training such hidden units became practical only in the 1980s. Figure by Daeha Kim; illustrative, no data.

The book is often blamed for a decade of neglect of neural networks. Its authors doubted that multilayer networks would do much better, but what they proved concerned single-layer ones. The wider field was in trouble too. Early programs that worked on small, clean problems failed on realistic ones, because the number of possibilities to search grows explosively with the size of the problem. In 1973 the mathematician James Lighthill, asked by the UK Science Research Council to review the field, concluded that it had failed to deliver on its promises outside a few narrow areas (Lighthill 1973). Funding in the UK was cut sharply, and in the United States military funding moved toward projects with clear applications. The period of reduced support that followed, roughly the second half of the 1970s, came to be called the first “AI winter.”

Expert systems, and the second winter (1980–1993)

AI recovered in the 1980s by narrowing its ambition. Instead of general intelligence, researchers built “expert systems” that encoded the knowledge of specialists as hundreds or thousands of if-then rules. The best-known commercial success was R1 (later called XCON), which configured orders for Digital Equipment Corporation’s VAX computers, a task that had required experienced engineers (McDermott 1982). Companies founded AI groups, specialized “LISP machines” were sold to run AI software, and in 1982 Japan launched its ambitious Fifth Generation Computer Systems project.

The boom did not last. Expert systems were brittle: they failed abruptly on cases their rules did not anticipate, and keeping thousands of rules consistent as the world changed proved expensive. Cheaper general-purpose workstations made the specialized hardware obsolete, and the market for it collapsed around 1987. The second AI winter, from the late 1980s to the early 1990s, was as much a commercial collapse as a scientific one.

Meanwhile, the neural network tradition had quietly solved its 1969 problem. In 1986 David Rumelhart, Geoffrey Hinton and Ronald Williams showed how to train networks with hidden layers by propagating the output error backward through the network and adjusting each weight in proportion to its contribution to the error (Rumelhart et al. 1986). The method, backpropagation, had been described earlier by others, but this paper made it widely known and showed that hidden units learned useful internal representations. Multilayer networks could now learn XOR and much more.

Learning from data, and a chess match (1990s)

In the 1990s, much of AI quietly changed character. Instead of writing rules, researchers increasingly fitted statistical models to data, and judged them by how well they predicted cases they had not seen. Hydrology was an early adopter. Kuolin Hsu, Hoshin Gupta and Soroosh Sorooshian trained a three-layer neural network to simulate daily streamflow in the Leaf River basin in Mississippi and found that it represented the rainfall-runoff relationship better than both a linear time-series model and the conceptual Sacramento soil moisture accounting model. They added that, because its components had no physical meaning, it was no substitute for conceptual watershed modeling (Hsu et al. 1995). By 2000 an ASCE task committee had written a two-part review of neural networks in hydrology (ASCE Task Committee 2000). The tension they named, between models that predict well and models we can interpret, has never left our field.

The decade, and this first part of the story, ended with a symbolic victory for the older tradition. In May 1997 IBM’s Deep Blue beat the world chess champion Garry Kasparov 3.5–2.5 in a six-game match. Deep Blue did not learn chess in the modern sense. Its builders credit a massively parallel computer with hundreds of special-purpose chess chips, an emphasis on deep search, a complex evaluation function and a database of grandmaster games (Campbell et al. 2002). It was a triumph of search and engineering on a problem with clear rules and a clear score, exactly the kind of problem the Dartmouth founders had in mind, forty years later than they had hoped.

What the first sixty years teach

Two traditions, both still alive. From the start there were two ideas of how to build intelligence: write down the rules (logic, search, expert systems) or let the machine learn them from examples (artificial neurons, perceptrons, backpropagation). The first dominated until the 1990s; the second dominates today. But today’s systems still combine them, for example when a language model is asked to call a calculator or run a test suite.

Promises ran ahead of data and computing power. Both winters followed periods in which researchers and funders expected general intelligence within a decade or two. The ideas were often sound; what was missing was enough data and computing power to make them work at scale. That is worth remembering whenever a forecast about AI sounds certain, in either direction.

Turing’s question is still the hard one. Turing replaced “can it think?” with “can we tell the difference?”, a test based on behavior. The benchmark debates in my recent posts are versions of the same move, and they inherit the same weakness: passing a test is not the same as having the ability the test was meant to measure.

Hydrology has been here before. The 1995 neural network that beat a conceptual model, and the authors’ caution about what it could not tell us, anticipate today’s debates about machine learning in hydrology almost exactly.

Bottom line

• Turing defined the computer (1936) and proposed a behavioral test of machine intelligence (1950), including the idea of a machine that learns.
• The field was named in 1955 and grew quickly, but early optimism ran into the explosion of possibilities in real problems and into the limits of single-layer perceptrons.
• Two AI winters followed: in the late 1970s after early promises failed, and around 1987–1993 after the expert-system boom collapsed.
• Backpropagation (1986) made multilayer neural networks trainable, and in the 1990s learning from data, including in hydrology, began to replace writing rules.
• Deep Blue’s 1997 win was a triumph of search and engineering, not of learning, and it closed the first era.

Questions to think about

1. Turing replaced “can machines think?” with a behavioral test. What would be a good behavioral test of whether an AI “understands” a hydrological model, and what could it miss?
2. Why can one perceptron learn AND but not XOR? Try drawing it before looking at Figure 2.
3. Hsu and colleagues found that a neural network predicted streamflow better than a conceptual model but warned that it was no substitute for one. Do you agree, thirty years later?

---

한 줄 요약 (Korean summary). 인공지능의 역사를 세 편으로 정리하는 연재의 첫 편으로, 앨런 튜링부터 딥 블루까지(1936–1997)를 다룬다. 튜링은 1936년 기호를 읽고 쓰는 가상의 기계와 어떤 기계든 흉내 낼 수 있는 “보편 기계”를 제안해 컴퓨터의 개념을 세웠고, 1950년에는 “기계가 생각할 수 있는가”라는 질문을 사람과 기계를 구별할 수 있는지 묻는 행동 시험(이미테이션 게임)으로 바꾸면서 학습하는 기계라는 생각까지 내놓았다. 1955년 다트머스 연구 제안서에서 “인공지능”이라는 말이 처음 쓰였고, 논리 이론가, 스스로 배우는 체커 프로그램, 예시로부터 가중치를 조정하는 퍼셉트론, 대화 흉내를 낸 ELIZA가 잇따라 나왔다. 그러나 실제 문제에서는 탐색해야 할 경우의 수가 폭발적으로 늘었고, 1969년 민스키와 페퍼트는 단층 퍼셉트론이 XOR 같은 문제를 풀 수 없음을 보였다. 1973년 라이트힐 보고서 이후 첫 번째 “AI 겨울”이 왔고, 1980년대 전문가 시스템 붐도 규칙의 경직성과 유지 비용 때문에 무너지며 두 번째 겨울(약 1987–1993)로 이어졌다. 그 사이 1986년 역전파 알고리즘이 은닉층을 가진 신경망을 학습시킬 수 있게 했고, 1990년대에는 규칙을 쓰는 대신 데이터로 학습하는 방식이 퍼졌다. 수문학도 일찍 이를 받아들여 1995년 신경망이 강우-유출 관계를 시계열 모형과 개념적 모형보다 잘 재현했지만, 저자들은 물리적 의미가 없어 개념적 모형을 대신할 수 없다고 경고했다. 1997년 딥 블루가 체스 세계 챔피언 카스파로프를 이긴 것은 학습이 아니라 탐색과 공학의 승리였다. 규칙과 학습이라는 두 전통, 데이터와 계산 능력을 앞질러 간 기대, 그리고 지능을 어떻게 측정할 것인가라는 튜링의 질문은 지금도 그대로 남아 있다.

---

References

ASCE Task Committee on Application of Artificial Neural Networks in Hydrology (2000). Artificial neural networks in hydrology. I: Preliminary concepts. Journal of Hydrologic Engineering, 5(2), 115–123. https://doi.org/10.1061/(ASCE)1084-0699(2000)5:2(115)

Campbell, M., Hoane, A. J., & Hsu, F.-h. (2002). Deep Blue. Artificial Intelligence, 134(1–2), 57–83. https://doi.org/10.1016/S0004-3702(01)00129-1

Hsu, K., Gupta, H. V., & Sorooshian, S. (1995). Artificial neural network modeling of the rainfall-runoff process. Water Resources Research, 31(10), 2517–2530. https://doi.org/10.1029/95WR01955

Lighthill, J. (1973). Artificial intelligence: A general survey. In Artificial Intelligence: A Paper Symposium. Science Research Council, London.

McCarthy, J., Minsky, M. L., Rochester, N., & Shannon, C. E. (2006). A proposal for the Dartmouth summer research project on artificial intelligence, August 31, 1955. AI Magazine, 27(4), 12–14. https://doi.org/10.1609/aimag.v27i4.1904

McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5(4), 115–133. https://doi.org/10.1007/BF02478259

McDermott, J. (1982). R1: A rule-based configurer of computer systems. Artificial Intelligence, 19(1), 39–88. https://doi.org/10.1016/0004-3702(82)90021-2

Minsky, M., & Papert, S. (1969). Perceptrons: An Introduction to Computational Geometry. MIT Press. Reissued 2017. https://doi.org/10.7551/mitpress/11301.001.0001

Newell, A., & Simon, H. A. (1956). The logic theory machine: A complex information processing system. IRE Transactions on Information Theory, 2(3), 61–79. https://doi.org/10.1109/TIT.1956.1056797

Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65(6), 386–408. https://doi.org/10.1037/h0042519

Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323, 533–536. https://doi.org/10.1038/323533a0

Samuel, A. L. (1959). Some studies in machine learning using the game of checkers. IBM Journal of Research and Development, 3(3), 210–229. https://doi.org/10.1147/rd.33.0210

Turing, A. M. (1937). On computable numbers, with an application to the Entscheidungsproblem. Proceedings of the London Mathematical Society, s2-42(1), 230–265 (read in 1936). https://doi.org/10.1112/plms/s2-42.1.230

Turing, A. M. (1950). Computing machinery and intelligence. Mind, 59(236), 433–460. https://doi.org/10.1093/mind/LIX.236.433

Weizenbaum, J. (1966). ELIZA: A computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1), 36–45. https://doi.org/10.1145/365153.365168

Weizenbaum, J. (1976). Computer Power and Human Reason: From Judgment to Calculation. W. H. Freeman.

Both figures are original. The Python scripts that produce them are available on GitHub: Figure 1 and Figure 2.