What is AI?
The term artificial intelligence means different things to different people. A useful and non-committal definition is that AI denotes any technology used to automate tasks that would otherwise require human intelligence, judgment, or perception.
Computer scientists working in AI build systems designed to learn, adapt, and solve problems. The objective is not only to automate tasks but to produce systems that respond to changing environments with some approximation of intelligence and improve from experience. Some people bristle at attributing reasoning, understanding, and learning to computer systems. Others regard exactly that as the field’s purpose. To have a view worth holding on that debate you need to know a good deal more about how the systems work, which is what the rest of Part I is for.
AI covers a range of technologies whose functions can be loosely analogized to human capacities. Computer vision interprets visual information, allowing systems to recognize objects, navigate environments, and analyze medical images. Speech recognition processes spoken language for transcription and command. Natural language processing handles text in ways that resemble reading and writing. Robotics puts AI into physical systems that manipulate the world.
We already live with AI. It sets credit limits, screens job applications, flags suspected credit card fraud, routes traffic, recommends medical treatments, drives cars in several American cities, and drafts documents. Its reach into ordinary life is substantial and still growing quickly.
Two things commonly discussed alongside AI are not AI. Blockchain is a technology for maintaining distributed records without a trusted intermediary. Cryptocurrencies are speculative assets built on it. Both raise interesting legal and regulatory questions, but neither counts as AI in my view.
How we got here
The history of AI in five eras
The history of AI can be divided, with a little of bit of fudging, into five eras.
In the foundational era of the 1940s and 1950s, the basic intellectual machinery was assembled. Alan Turing and John von Neumann (a Hungarian mathematician like my grandmother) helped establish the theory of computation and the architecture of modern computers. Claude Shannon developed information theory. Warren McCulloch and Walter Pitts showed that simplified networks of artificial neurons could perform logical operations. Norbert Wiener’s cybernetics treated animals and machines alike as systems of information, control, and feedback. There was not yet a single approach called artificial intelligence. There were several different ideas about what intelligence was and how a machine might reproduce it.
The first boom began in 1956. The term “artificial intelligence” was coined for the Dartmouth Workshop by John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon, and the meeting is generally treated as the field’s founding. The dominant approach became symbolic: represent the world using symbols, write rules for manipulating them, and hope that something resembling logical reasoning emerges. Programs learned to prove theorems, solve puzzles, plan sequences of actions, and manipulate simple worlds described in language. The mood of the period was that human-level AI was close.
It wasn’t.
What followed Is sometimes called the AI winter, but looking more closely, what we actually see is a cycle of winters and revivals from the 1970s into the early 1990s. Early systems proved brittle, computational resources were limited, and so Enthusiasm gave way to disappointment and a contraction of funding. Expert systems revived the field by encoding the knowledge of human specialists as large collections of explicit rules. They found real commercial applications and generated another wave of enthusiasm. They were also expensive to build, difficult to maintain, and poor at dealing with situations their designers had not anticipated. By the late 1980s, enthusiasm that expert systems would lead to AI waning again, but expert systems still had their uses and lived on in rule-based systems that formed a key component of businesses such as SAP, Siebel, and Oracle.
Machine-learning was an old idea that took a long time to come into its own. In the 2000s, unsupervised learning methods took off, and this is arguably the point at which machine learning became conceptually the dominant paradigm for thinking about AI. Unsupervised learning means that rather than encoding knowledge as rules, researcher could built systems that inferred patterns from data. Statistics and probability overtook logic and symbols. In the 2010s, deep learning pushed the approach much further: instead of specifying which features mattered, large neural networks learned useful representations for themselves. Large datasets, improved algorithms, and cheap parallel computation produced rapid advances in vision, speech, and language. The success of deep learning after roughly 2012 made neural networks the dominant approach across much of AI. Note that a lot of the literature discussing the social and economic implications of machine learning in this period used the voguish term “Big Data”.
Whether generative AI constitutes a fifth era or merely the latest stage of the fourth is still open. The underlying technologies did not suddenly appear in 2022 with the launch of ChatGPT. Large language models, transformers, and generative systems had been developing for years. What changed around 2022 was their reach. Systems that had largely been the concern of researchers, engineers, and specialist users became general-purpose products that ordinary people could use by typing instructions in natural language. AI ceased to be something most people encountered indirectly, inside search engines, recommendation systems, or software products, and became something they could talk to.
The distinction between the fourth and fifth eras matters less than the fact that something changed, and that the change was large enough for people outside the field to notice.
The 2024 Nobel Prizes
In 2024 artificial intelligence received an unusual kind of recognition. The Nobel Prize in Physics went to John Hopfield and Geoffrey Hinton for work underlying machine learning with artificial neural networks. The Nobel Prize in Chemistry recognized Demis Hassabis and John Jumper for protein structure prediction using AlphaFold, alongside David Baker for computational protein design. Hinton and Hassabis are people worth knowing about if you want to understand the state of modern AI.
Hinton had spent much of his career pursuing an idea that periodically looked unfashionable: intelligence might be produced not by giving a computer enough rules, but by constructing networks that could learn useful representations for themselves. Researchers in the “connectionist” school had long advocated the use of artificial neural networks consisting of layers of simple computational units connected by adjustable weights. But without an effective way of adjusting those weights, this approach seemed unlikely to yield significant advances.
This is where backpropagation enters the story. A neural network takes an input and produces an output. During training, that output is compared with the desired result and an error is calculated. Backpropagation works backwards through the network, calculating how much each connection contributed to the error. The weights are then adjusted slightly in the direction that reduces it. Repeat the process millions or billions of times and the network gradually becomes better at the task.
Hinton did not invent backpropagation, which has a long and multi-authored history. But he was one of the researchers who demonstrated its power for learning representations in multilayer neural networks, and he continued working on neural networks through periods when much of the field had moved elsewhere. His 1986 work with David Rumelhart and Ronald Williams helped establish backpropagation as a practical method for training multilayer networks; two decades later he was still finding ways to make deeper networks train effectively.
The potential of deep learning with back propagation became clear in 2012 when Hinton and his students Alex Krizhevsky and Ilya Sutskever trained a large convolutional neural network on more than a million images and dramatically improved the state of the art in the ImageNet image-recognition competition. The system later became known as AlexNet. Deep learning had long been dismissed by some of the biggest names in the field (e.g. Minsky). After 2012 it was impossible to credibly argue that it didn’t work.
Demis Hassabis enters the story at the other end of this transition. DeepMind, which he co-founded, was built around the idea that the same general learning principles might solve many different kinds of problems. Games were an obvious place to test the proposition because success is easy to measure and enormous amounts of experience can be generated cheaply. In 2016 AlphaGo defeated Lee Sedol, one of the strongest Go players in the world. This was a big deal because Go is so much more computationally difficult than chess. AlphaGo learned to evaluate positions using neural networks, improved through reinforcement learning, and combined what it had learned with search. Its successors went further: AlphaGo Zero learned by playing against itself rather than by first studying human games.
Building on the success of AlphaGo, DeepMind transitioned from games to science and developed AlphaFold. Proteins are chains of amino acids that fold into complex three-dimensional shapes, and their shape largely determines what they can do. Predicting that structure from the amino-acid sequence had been a central problem in biology for roughly half a century. In 2020, the DeepMind team led by Hassabis and John Jumper introduced AlphaFold2, which achieved a dramatic increase in the accuracy of computational protein-structure prediction. DeepMind later released predictions covering roughly 200 million protein sequences, making structural information available at a scale that would have been impossible to obtain experimentally.
Five milestones
Deep Blue (1997). IBM’s chess machine defeated Garry Kasparov under tournament conditions. Deep Blue worked mostly by evaluating enormous numbers of positions very quickly, combined with chess knowledge encoded by grandmasters. Critics said this was calculation rather than intelligence, and they had a point, but the result still showed that machines could beat the best humans at a task long treated as a proxy for thought.
AlexNet (2012). A deep convolutional network built by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton won the ImageNet visual recognition challenge by a wide margin. Its success was partly architectural and partly a matter of having realized that graphics processors, built for video games, were extremely good at the arithmetic neural networks require. AlexNet made deep learning the dominant approach in computer vision almost overnight.
AlphaGo (2016). DeepMind’s system beat Lee Sedol, one of the world’s strongest Go players. Go has vastly more possible positions than chess, too many for brute-force search, and had been expected to resist computers for decades. AlphaGo combined deep networks with reinforcement learning, improving by playing itself millions of times. It also played moves that human experts initially judged to be mistakes and later judged to be insights, which was the first widely noticed instance of an AI system doing something creative rather than merely competent.
The Transformer (2017). Researchers at Google published Attention Is All You Need, introducing an architecture that processes sequences by weighing the relevance of every element to every other element, regardless of distance. Earlier approaches read text word by word and struggled to connect the beginning of a passage to the end. The transformer could take in a whole passage at once. It was also much easier to run in parallel, which sounds like an engineering detail and turned out to be the thing that made scale possible. Nearly every significant language model since is a transformer.
ChatGPT (2022). OpenAI put a conversational interface on a capable language model and released it publicly on 30 November 2022. It reached a million users in five days and a hundred million within two months, making it the fastest-adopted consumer application in history. Nothing about the underlying model surprised specialists. The surprise was how many people wanted to use it, and what they wanted to use it for.
← Primer contents · Contents · Next: How Large Language Models Work →