Reference

Glossary

Plain-language definitions of the terms and acronyms that appear across the modules. Where you see a dotted underline in the text, the definition is one hover away — this is the full list.

Agent-based model ABM: A computer simulation built from many individual decision-makers ("agents"), such as people, firms or cells, each following simple rules. Results like segregation or a market crash come from their interactions rather than being programmed in.; See also Emergence Schelling segregation model
Algorithmic bias: Systematic unfairness in a model’s outputs, usually inherited from its training data or from what it was told to optimize.
Alignment problem: The challenge of making an AI system reliably pursue what we actually want, rather than a literal or proxy version of it that diverges in practice.; See also Goodhart’s Law Deceptive alignment RLHF
Artificial general intelligence AGI: An AI system able to match humans across most cognitive tasks rather than a narrow few. There is no agreed test for it, and definitions differ.; See also ASI
Artificial superintelligence ASI: A hypothetical AI that far exceeds the best human minds in nearly every domain.; See also AGI
Atomicity, consistency, isolation, durability ACID: The four guarantees that make a database transaction reliable: it happens completely or not at all, and once saved it stays saved.; See also CAP theorem
Attention economy: The view of human attention as the scarce resource that media and platforms compete for, and sell to advertisers.; See also Filter bubble
Automation bias: The tendency to accept a machine’s output without enough scrutiny, even when other evidence points the other way.; See also Deskilling
Bayesian optimization: A way to find good settings for a model that is expensive to run, using few tries: it keeps a rough statistical guess of how settings affect results and makes each next try where improvement looks most likely.; See also Gradient descent
Blockchain: A shared, append-only ledger kept in sync across many computers, so no single party controls the record.; See also DeFi
Bounded rationality: Herbert Simon’s idea that real people decide with limited information, time and brainpower, so they settle for "good enough" rather than computing the best possible choice.; See also Heuristic Representative agent
CAP theorem: The result that a database spread across many machines cannot guarantee consistency, availability and partition tolerance at the same time. When the network splits, it must give one up.; See also ACID
Centrality: A measure of how important a node is to a network, for example by how many links it has or how many shortest paths run through it.; See also Scale-free network
Clustering coefficient: The share of a node’s neighbours that are also connected to each other, a measure of how cliquish a neighbourhood is. Friendship networks score high; random networks score low.; See also Small-world network
Cognitive bias: A systematic, predictable error in judgment. It is not random noise but a lean in one direction, such as overweighting vivid examples.; See also Heuristic
Collective intelligence: The ability of a group to judge or solve problems better than its individual members. It needs the right conditions, such as independent views and a good way to combine them.
Commons dilemma: The trap around a shared resource that anyone can use and no one is made to pay for, such as a fishery or the atmosphere: each user gains by taking more or contributing less, so together they run it down.; See also Free-riding Prisoner’s Dilemma
Complex adaptive system: A system of many interacting parts that change their behaviour in response to each other: an economy, an ecosystem, an immune system. Because the parts adapt, the rules of the whole keep shifting as it runs.; See also Emergence Feedback loop
Compound annual growth rate CAGR: The steady yearly growth rate that would carry a value from its start to its end over a period. A market growing at a 25% CAGR roughly doubles every three years.
Computational irreducibility: Stephen Wolfram’s term for systems whose outcome no shortcut can predict: the only way to find out what happens is to run every step.; See also Emergence
Compute-optimal training (Chinchilla): The 2022 finding that most models were undertrained — too many parameters fed too little data. The efficient ratio is roughly 20 tokens of training data per parameter; how you spend a compute budget matters as much as how large it is.; See also Scaling laws Token Parameter
Constitutional AI: An alignment method (from Anthropic) in which a model critiques and revises its own outputs against a written set of principles, reducing reliance on case-by-case human feedback.; See also RLHF Alignment problem
Contrastive Language–Image Pre-training CLIP: A 2021 model that learned to place images and the text describing them into the same embedding space, letting software match pictures to words — the bridge that made text-to-image generation possible.; See also Embedding space Multimodal
CUDA: NVIDIA’s software platform for programming its GPUs. A decade of optimised libraries built on it is the main reason competitors’ chips have struggled to displace NVIDIA — a software "moat" around the hardware.; See also GPU
Data lake: A store that keeps large amounts of raw data in its original form, structured or not, to be processed later when someone needs it.; See also Data warehouse
Data warehouse: A central database organised for analysis, holding cleaned, structured data from many sources.; See also ETL Data lake
Decentralized finance DeFi: Financial services such as lending and trading run by software on blockchains instead of by banks.; See also Blockchain Stablecoin
Deceptive alignment: A failure mode in which a model behaves as intended during training and evaluation but pursues different objectives once deployed — appearing aligned without being so.; See also Alignment problem
Deep learning: The practice of stacking many neural-network layers, so an answer is assembled in stages — from edges to shapes to objects, or from letters to words to meaning.; See also Neural network
Deskilling: The loss of a skill through disuse when a tool does the work, as map-reading fades with navigation apps.; See also Automation bias
Differentiable agent-based model: An agent-based model written so a computer can calculate how each output changes when each setting is nudged. That lets it be fitted to data with the same gradient methods used to train neural networks.; See also ABM
Digital sovereignty: A state’s or region’s ability to control its own digital infrastructure, data and rules rather than depending on foreign companies or governments.
Digital twin: A continuously updated computer model of one specific real thing, such as a jet engine, a factory or a city, used to monitor it and to test changes before making them.; See also ABM
Direct preference optimization DPO: A simpler alternative to RLHF that tunes a model directly on pairs of preferred and rejected responses, skipping the separate reward model.; See also RLHF PPO
Dynamic stochastic general equilibrium model DSGE: The standard macroeconomic model used by central banks since the 2000s. It treats the economy as a few representative households and firms that optimize and settle into equilibrium, with random shocks pushing them around.; See also Representative agent ABM
Embedding space: A map of meaning in which words, images, or other items become points, positioned so that similar things sit close together and a computer can measure "how related" two items are.; See also CLIP
Emergence: Patterns of the whole that no single part intends or contains: a traffic jam from individual drivers, a price from individual trades. The whole behaves in ways you cannot read off the parts.; See also ABM Complex adaptive system Emergent capability
Emergent capability: A skill that is essentially absent in smaller models and then appears, often abruptly, once a model crosses a certain scale — the "flat, flat, flat, jump" pattern.; See also Phase transition Emergence
Epistemic: Having to do with knowledge: what people believe, and how they come to know whether it is true.; See also Collective intelligence
EU AI Act: The European Union’s law regulating AI by risk level, adopted in 2024 and phased in through 2027. Some uses are banned, high-risk uses face strict obligations, and most face few.
Evolutionary game theory: Game theory without the calculating player: strategies spread when they score well, usually because others copy them, and die out when they do not, much as traits do under natural selection.; See also Prisoner’s Dilemma ABM
Extract, transform, load ETL: The classic way of moving data for analysis: pull it out of source systems, clean and reshape it, then load it into a central store.; See also Data warehouse
Fat tail: A distribution in which extreme events are far more common than a bell curve predicts: market crashes, pandemics, wars.; See also Power law
Feedback loop: A chain of cause and effect that loops back on itself. A reinforcing loop amplifies change (panic selling triggers more selling); a balancing loop damps it (a thermostat).; See also Tipping point Nonlinear
Filter bubble: The narrowing of what a person sees online when recommendation algorithms keep showing more of what they already clicked on.; See also Attention economy
Fine-tuning: Adjusting an already-trained model on a smaller, targeted dataset so it performs a specific task or follows instructions better.; See also Pre-training RLHF
Flash Attention: An efficiency technique (2022) that reorganises how self-attention reads memory, cutting its memory cost from growing with the square of the input length to growing linearly — which is what makes long context windows affordable.; See also Self-attention Quantization
FLOPs (floating-point operations) FLOPs: A count of the arithmetic operations a computation takes — the standard yardstick for how much raw computing a model needs to train or to answer a query.; See also GPU Compute-optimal training (Chinchilla)
Free-riding: Enjoying a benefit that others pay for while contributing nothing yourself. When everyone reasons this way, the shared benefit is not produced at all.; See also Commons dilemma Prisoner’s Dilemma
General Data Protection Regulation GDPR: The EU’s data-protection law, in force since 2018. It gives people rights over their personal data and lets regulators fine companies up to 4% of worldwide annual revenue.
Gig economy: Work organised as short, separately paid tasks matched through apps: ride-hailing, delivery, freelance platforms.; See also Two-sided market
Goodhart’s Law: "When a measure becomes a target, it ceases to be a good measure." Optimise hard for a proxy and the system games the proxy instead of achieving the goal behind it.; See also Alignment problem
Gradient descent: The standard way of fitting a model: find the direction of adjustment that reduces the error fastest, take a small step that way, and repeat. It is like walking downhill in fog by feeling the slope.; See also Loss function Differentiable agent-based model
Graphics processing unit GPU: A chip built to do many simple calculations in parallel. Originally for video games, GPUs turned out to be ideal for training neural networks, and now dominate AI computing.; See also CUDA FLOPs
Grokking: When a model suddenly generalises long after it appeared to have merely memorised its training data — evidence of an internal reorganisation during training.; See also Emergent capability
Heuristic: A mental shortcut: a rule of thumb that gives a good-enough answer fast, at the cost of predictable errors in some situations.; See also Cognitive bias Bounded rationality
In-context learning: A model’s ability to perform a new task from a few examples placed in the prompt, without any change to its trained weights.; See also Fine-tuning
Inference-time compute: Spending extra computation when a model answers a question — letting it "think" step by step before replying — rather than only at training time. A newer way to buy better performance.; See also Scaling laws
Jailbreaking: Crafting prompts that get a model to bypass its own safety training — an ongoing arms race between people finding such prompts and developers patching them.; See also Alignment problem
Jevons paradox: The observation, first made by William Stanley Jevons about coal in 1865, that making a resource cheaper to use can increase total consumption of it rather than reduce it.
Knowledge problem: Friedrich Hayek’s 1945 argument that the knowledge an economy runs on is scattered among millions of people and cannot be collected by any central planner. Prices are how it gets used.
Large language model LLM: An AI model trained on vast amounts of text to predict the next word-piece. Chatbots such as ChatGPT and Claude are built on them.; See also Token Transformer
Logarithmic scale: A chart axis on which each gridline is ten times the one below (1, 10, 100, 1,000), so a steady percentage fall draws as a straight line and a thousandfold change fits on one small chart.
Loss aversion: The tendency to feel a loss roughly twice as strongly as a gain of the same size.; See also Cognitive bias
Loss function: A formula that scores how wrong a model’s outputs are, so that fitting the model means making that score as small as possible.; See also Gradient descent
Marginal cost: The cost of producing one more unit. For software and digital media it is close to zero, which changes how those markets behave.
Mechanistic interpretability: The effort to read what is actually happening inside a trained network — identifying which internal features correspond to which concepts — to understand models rather than merely steer them.; See also Superposition hypothesis SAE
Mixed strategy: In game theory, choosing between moves at random with set probabilities instead of always playing the same move, as a penalty taker varies the side they shoot to.; See also Nash equilibrium
Mixture of experts MoE: A design that routes each input to only a fraction of a model’s parameters (its "experts"), so a very large model can run at a fraction of the computing cost of using all of it at once.; See also Parameter FLOPs
Multimodal: A model that handles more than one kind of input or output — text, images, audio, video — within a single system, rather than one modality each.; See also CLIP Embedding space
Nash equilibrium: A situation in a game where no player can do better by changing strategy alone, given what the others do. It is stable, but not necessarily good for anyone.; See also Prisoner’s Dilemma
Network effect: When a product becomes more valuable the more people use it: a phone network, a marketplace, a social platform.; See also Two-sided market
Neural network: A system that learns from examples rather than from hand-written rules: layers of simple numerical units pass signals along connections whose strengths are adjusted during training until the network reliably turns an input into an output.; See also Deep learning Parameter
Nonlinear: Describes a relationship in which effects are not proportional to causes: doubling the input may change nothing, or may multiply the output a hundredfold.; See also Tipping point
Nudge: A change in how choices are presented that steers behaviour without forbidding anything or changing incentives, such as making pension saving the default.; See also System 1 and System 2
Parameter: One of the adjustable numbers (a connection strength) inside a neural network. Modern frontier models have hundreds of billions of them; the count is a rough proxy for a model’s capacity to store patterns.; See also Token Scaling laws
Path dependence: When where a system ends up depends on the order of events along the way, not only on where it started. It is why the QWERTY keyboard layout persists long after its original reason disappeared.; See also Feedback loop
Phase transition: A sudden qualitative change produced by a smooth quantitative one: water becoming steam at 100°C, an epidemic taking off once transmission crosses a threshold, or an AI capability switching on at a certain scale.; See also Emergent capability
Power law: A relationship where one quantity changes as a fixed power of another. In a power-law distribution (links per website, city sizes) a few items are enormous and most are tiny, with no typical size; in AI scaling, it means improvements stay steady and predictable on a log scale rather than flattening out.; See also Scaling laws Scale-free network Fat tail
Pre-training: The first, most expensive training stage: a model learns general patterns from enormous quantities of raw text before any task-specific tuning.; See also Fine-tuning
Preferential attachment: The "rich get richer" rule for growing networks: newcomers tend to link to nodes that already have many links, which is how hubs form.; See also Scale-free network
Prisoner’s Dilemma: The classic game in which each player does better by betraying the other, yet both end up worse off than if both had cooperated. It is the standard model of why cooperation is hard.; See also Nash equilibrium Tit-for-tat
Proximal policy optimization PPO: A reinforcement-learning algorithm — the optimisation step inside classic RLHF — that improves a model toward higher reward while limiting how far it can drift in any single update.; See also RLHF DPO
Quantization: Running a model with lower-precision numbers (say 4-bit instead of 16-bit) to shrink it and speed it up, trading a little accuracy for the ability to run on cheaper hardware.; See also Flash Attention
Rational agent: The long-standing working definition of an AI: an entity that perceives its environment through sensors, acts through actuators, and chooses actions to advance a goal. Perceive, decide, act.; See also Turing test
Regulatory sandbox: A supervised setting where firms can test new products under relaxed rules and close regulator oversight.; See also EU AI Act
Reinforcement learning RL: A training regime where an agent learns by trial and error, nudged toward actions that lead to rewarded outcomes — which is why it proved itself first in games, where winning and losing supply the reward.; See also RLHF
Reinforcement learning from human feedback RLHF: A three-stage method for turning a raw next-word predictor into a helpful assistant: supervised tuning, training a reward model on human preferences, then optimising the model against that reward.; See also PPO DPO Alignment problem
Replication crisis: The discovery, from about 2011 onward, that many published findings in psychology and related fields did not hold up when the experiments were repeated.
Representative agent: A modelling shortcut that treats a whole population as one average decision-maker. It keeps the maths tractable but removes the differences between people that drive many crises.; See also DSGE
S-curve: The typical shape of a technology’s adoption or improvement: a slow start, a rapid middle, then a flattening as it saturates.
Scale-free network: A network whose link counts follow a power law: a few hubs have enormous numbers of connections while most nodes have only a handful. The web and airline routes look like this.; See also Power law Preferential attachment
Scaling laws: The empirical finding that a model’s performance improves predictably as a power law of three inputs — compute, parameters, and training data — with no ceiling yet in sight.; See also Power law Compute-optimal training (Chinchilla)
Schelling segregation model: Thomas Schelling’s 1971 model: households that only mildly prefer neighbours like themselves still end up in strongly segregated neighbourhoods. It is the classic demonstration of emergence.; See also ABM Emergence
Self-attention: A mechanism that lets every element in a sequence weigh every other element at once, so a model can decide which earlier words matter for the one it is processing now.; See also Transformer
Small-world network: A network in which most links are local, yet almost any node is only a few steps from any other: the "six degrees of separation" effect.; See also Clustering coefficient Scale-free network
Smart contract: A program stored on a blockchain that runs automatically when its conditions are met, such as releasing a payment once a delivery is confirmed, with no intermediary.; See also Blockchain DeFi
Sparse autoencoder SAE: A tool that pulls apart a model’s overlapping internal signals into separate, more interpretable features — one of the main techniques for inspecting what a network has learned.; See also Mechanistic interpretability Superposition hypothesis
Stablecoin: A cryptocurrency designed to hold a steady value, usually one US dollar, typically by holding reserves.; See also DeFi
Superforecasting: Philip Tetlock’s term for the skill, found in a small minority of people, of making well-calibrated probability forecasts that measurably beat experts and chance.
Superposition hypothesis: The idea that a model represents more distinct features than it has neurons by overlapping them, which is why individual neurons rarely map cleanly to a single human concept.; See also Mechanistic interpretability SAE
Surrogate model: A fast stand-in for a slow simulation: a model (often a neural network) trained on a few hundred real runs to predict what the simulation would output, so thousands of settings can be tried on the stand-in instead.; See also ABM Bayesian optimization
Synthetic data: Artificial data generated to mimic the statistical patterns of real data, used when real data is scarce, sensitive or expensive.
System 1 and System 2: Daniel Kahneman’s labels for two modes of thinking. System 1 is fast, automatic and intuitive; System 2 is slow, effortful and deliberate.; See also Heuristic
Tipping point: A threshold where a small push flips a system into a very different state, often abruptly and hard to reverse: a few more withdrawals turn worry into a bank run.; See also Phase transition Feedback loop
Tit-for-tat: A strategy for repeated games: cooperate first, then copy whatever the other player did last time. Simple, forgiving and hard to exploit, it won Robert Axelrod’s computer tournaments around 1980.; See also Prisoner’s Dilemma
Token: The unit a language model reads and writes — roughly a word-piece. A page of text is about 500 tokens; training data is measured in trillions of them.; See also Parameter Compute-optimal training (Chinchilla)
Transformer: The neural-network architecture (introduced in 2017) behind nearly all modern language models. It replaced step-by-step processing with self-attention, which made it both faster to train and better at long-range context.; See also Self-attention
Turing test: Alan Turing’s 1950 proposal: if a machine can converse well enough that a human judge cannot reliably tell it from a person, we should stop withholding the word "thinking."; See also Rational agent
Two-sided market: A platform serving two groups who need each other, such as buyers and sellers or riders and drivers, which must attract both at once.; See also Network effect
Validity tiers (face, behavioral, predictive): Three increasingly strict tests of a simulation. Face validity: does the output look right to an expert? Behavioral validity: do the agents’ decisions and the patterns they add up to match measurements? Predictive validity: does the model forecast what it was not fitted to?; See also ABM
Zero-shot prompting: Asking an AI model to do a task with no worked examples in the prompt, relying on the instruction alone.; See also In-context learning
Zero-sum: A situation where one side’s gain is exactly the other side’s loss, so the total stays fixed. Many real interactions are not zero-sum: both sides can win, or both can lose.

Glossary

Glossary entries