A short history of Artificial Intelligence

This page is an overview of how artificial intelligence developed, from the first theories of computation to the breakthrough of deep learning and the Transformer. It stops around 2020, just before AI reached the general public. The pattern of the story matters as much as the dates: progress came when algorithms, data, and computing power improved together.

Foundations before the name

The idea that a machine could do more than arithmetic is older than the computer. In 1843, Ada Lovelace wrote notes on Charles Babbage’s Analytical Engine. She saw that it could manipulate symbols, not only numbers. She also argued that it could not originate anything, only do what it was told. That objection still comes up today.

In 1936, Alan Turing described a theoretical machine that could carry out any computation that can be written as a procedure. This is the Turing machine, and it is the basis of every modern computer. In 1943, Warren McCulloch and Walter Pitts published a simplified mathematical model of a neuron. It showed that networks of simple on-off units could compute logical functions. That paper is the starting point of neural networks.

In 1950, Turing asked directly whether machines can think, in the paper “Computing Machinery and Intelligence”. He replaced the question with a test he called the imitation game, now known as the Turing test. A machine passes if a human judge, chatting in text, cannot reliably tell it from a person.

The Dartmouth workshop and early optimism

The term “artificial intelligence” comes from a 1955 funding proposal by John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon. It led to a summer workshop at Dartmouth College in 1956, which is usually named as the founding event of the field. The proposal assumed that every aspect of learning and intelligence could be described precisely enough for a machine to simulate it.

The following years produced real results:

• Logic Theorist (1956) by Allen Newell, Herbert Simon, and Cliff Shaw proved theorems from Principia Mathematica. It is often called the first AI program.
• LISP (1958) by John McCarthy became the main programming language of AI research for decades.
• The Perceptron (1958) by Frank Rosenblatt was a single-layer neural network that learned to classify simple images by adjusting its weights.
• Machine learning as a term was popularized by Arthur Samuel in 1959. His checkers program improved by playing against itself.
• ELIZA (1966) by Joseph Weizenbaum imitated a psychotherapist with simple pattern matching. Many users still believed it understood them.
• Shakey (1966–1972) at the Stanford Research Institute was the first mobile robot that planned its own actions.

Expectations ran ahead of the results. In 1965, Herbert Simon predicted that machines would be able to do any work a person can do within twenty years.

The first AI winter

By the early 1970s, the gap between the promises and the systems was obvious. Programs that worked on small, clean examples broke down on real problems. The number of possibilities they had to search grew too fast for the computers of the time. Language translation, one of the early goals, produced poor results.

In 1969, Marvin Minsky and Seymour Papert published Perceptrons. It proved that a single-layer perceptron cannot learn some simple functions, such as XOR. The book was widely read as a verdict against neural networks in general, and research on them slowed for more than a decade.

In 1973, the Lighthill report in the United Kingdom concluded that AI had not delivered on its goals. British funding was cut sharply, and DARPA in the United States reduced its support around the same time. This period of low funding and low interest is called the first AI winter.

Expert systems and the second winter

AI returned in the 1980s in a practical form: expert systems. An expert system stores the knowledge of human specialists as hundreds or thousands of if-then rules. MYCIN, built at Stanford in the 1970s, recommended antibiotics for blood infections. XCON, used by Digital Equipment Corporation from about 1980, configured computer orders and saved the company millions of dollars per year.

Companies invested heavily, and Japan launched its Fifth Generation Computer project in 1982. Specialized Lisp machines were sold to run AI software.

Neural networks also came back. In 1986, David Rumelhart, Geoffrey Hinton, and Ronald Williams popularized backpropagation, a method for training networks with several layers. Multi-layer networks can learn the functions that a single perceptron cannot.

The boom did not last. Expert systems were expensive to build and hard to maintain, because every rule was written and checked by hand. They failed in situations their rules did not cover. In 1987, the market for Lisp machines collapsed when ordinary workstations became fast enough. Funding dropped again into the early 1990s. This was the second AI winter, and for some years researchers avoided the term “AI” and used names like “machine learning” or “informatics” instead.

Machine learning takes over

From the 1990s, the field shifted from writing rules to learning them from data with statistical methods. Instead of asking experts for rules, researchers collected examples and let algorithms find the patterns.

• In 1989, Yann LeCun trained a convolutional neural network to read handwritten zip codes. Its successor, LeNet-5 (1998), read checks for banks.
• In 1997, Sepp Hochreiter and Jürgen Schmidhuber introduced the LSTM, a network that can remember information over long sequences. It later powered speech recognition and translation.
• In 1997, IBM’s Deep Blue beat world chess champion Garry Kasparov. Deep Blue relied on fast search and hand-tuned evaluation, not on learning, but it changed public perception of what machines could do.
• In the 2000s, machine learning moved quietly into products: spam filters, search ranking, product recommendations, and fraud detection.

Two ingredients that deep learning would later need arrived in this period. The web produced data at a scale nobody had before. In 2009, Fei-Fei Li’s team released ImageNet, a labeled dataset of millions of images. At the same time, graphics cards (GPUs) became programmable for general computation, with NVIDIA’s CUDA released in 2007. In 2006, Geoffrey Hinton and colleagues showed a practical way to train deep networks layer by layer, and the term deep learning spread.

The deep learning breakthrough

In 2012, a neural network called AlexNet, built by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton, won the ImageNet competition. Its top-5 error rate was about 15 percent. The next best entry, using older methods, was about 26 percent. AlexNet was trained on two consumer GPUs. The result convinced most of the field that deep neural networks, trained on large datasets with GPUs, beat hand-designed approaches.

The next years moved quickly:

• Speech recognition at Google, Microsoft, and others switched to deep learning, and error rates fell sharply.
• word2vec (2013) represented words as vectors, so that words with similar meaning sit close together.
• DeepMind’s DQN (2013–2015) learned to play Atari games from the raw pixels, using reinforcement learning.
• Generative adversarial networks (2014) by Ian Goodfellow generated realistic images.
• Attention (2014) let translation models focus on the relevant parts of a sentence.

In March 2016, DeepMind’s AlphaGo beat Lee Sedol, one of the strongest Go players, 4 games to 1. Go has far too many positions for brute-force search, so the result showed that learned intuition, combined with search, could master a problem many experts expected to stay unsolved for another decade.

Transformers and the first large language models

In 2017, researchers at Google published “Attention Is All You Need”. It introduced the Transformer, an architecture built only on attention. Unlike earlier sequence models, a Transformer processes all words of a text in parallel, which makes it far faster to train on GPUs. Nearly every large language model today is a Transformer.

Two lines of work followed in 2018. OpenAI released GPT (Generative Pre-trained Transformer), which first learned from a large amount of unlabeled text and was then adapted to specific tasks. Google released BERT, which improved search and many language benchmarks. Both showed that pre-training on general text and then adapting the model works better than training from scratch for each task.

In 2019, OpenAI released GPT-2 in stages, because it was concerned about misuse of generated text. In January 2020, OpenAI researchers published scaling laws: model quality improved predictably as the model, the data, and the compute grew. In May 2020, GPT-3, with 175 billion parameters, could translate, answer questions, and write code from a few examples in the prompt, without task-specific training. Also in 2020, DeepMind’s AlphaFold 2 predicted protein structures with accuracy close to laboratory methods.

At this point, the technology behind today’s AI assistants existed. The public breakthrough came on 30 November 2022, when OpenAI released ChatGPT. Frontier AI companies and Agentic AI pick up the story from there.

Milestones at a glance

What the history explains

Three lessons from this history help when reading today’s AI news.

Progress needs three ingredients at once. Neural networks were proposed in the 1940s and trained with backpropagation in the 1980s. They only won in 2012, when large datasets and GPUs were also available. Algorithms, data, and compute advanced together.

Two schools took turns. Symbolic AI wrote knowledge down as rules and logic. Connectionist AI let networks learn from examples. Symbolic AI led the field until the 1990s, and learning has led it since. Modern systems combine both again: a language model is learned, and the code and tools around it are written by hand.

Hype comes in cycles. Each wave of progress came with predictions that went far beyond the results, followed by disappointment and cuts in funding. The current wave has produced products that millions of people use every day, which earlier waves never did. The habit of over-predicting is still worth keeping in mind.