Humans and AI
AI as cognitive prosthesis, deskilling, algorithmic bias, and the governance choice ahead — how AI reshapes the way people think.
The previous module followed bounded minds into an environment built to capture their attention, and into the crowds that can make them wise or foolish. This one follows them one step further — into a partnership with machines that increasingly do their thinking with them. The agent that decided alone (Module 3), then strategically (Module 4), then under the pressure of the attention economy (Module 5), now decides alongside an artificial one — a cognitive prosthesis that brings new biases of its own. The question is what that partnership does to the human side of the pair — and who gets to set its terms.
Human-AI Futures: Augmentation, Deskilling, and the Choice Ahead
Module 3 drew the mind as two boxes: System 1, always on and effortless; System 2, asleep until called, slow and tiring. The 2016 edition of this book already compared machine search — the AI technique — to System 2 of the human brain; naming that machinery an external system is a short step from there. A decade on, the research literature has taken it: “System 3”, a cognitive prosthesis outside the skull, augmenting memory, attention, and reasoning. A clinician reading scans with a diagnostic assistant is the canonical case. Chess centaurs — human-AI teams — outperformed both humans and AI alone, leading Garry Kasparov to observe that “the process matters more than the components.”
But augmentation carries a shadow. When GPS replaced mental maps, spatial reasoning skills measurably declined; medical residents who lean on diagnostic assistants develop clinical intuition more slowly. When AI handles first-draft reasoning, the capacity to evaluate, challenge, and override weakens through disuse. Researchers call it cognitive surrender, and it comes with a paradox: the better AI performs, the less practice humans get at the skills needed to oversee it.
Now go back to the clinician’s assistant, and commit to a prediction. It was trained mostly on images of light skin; a patient with dark skin walks in — will it be as accurate? Stanford researchers answered in 2025: AI skin cancer detection systems were half as accurate for Black patients. Not because anyone made the algorithm prejudiced — in everyday language “bias” implies intent, but here it means a systematic skew in outcomes, whatever anyone intended: algorithmic bias. The training data underrepresented darker skin tones, and the skew scaled to every clinic running the model. The Workday hiring lawsuit alleged the same shape at hiring scale — an AI screening tool discriminating against applicants over 40 and people with disabilities, at a volume no human hiring team could match.
The pattern is consistent: AI amplifies rather than transcends. It inherits bias from training data (which reflects historical discrimination), scales it (one algorithm applied to millions of decisions), and automates it (humans defer to algorithmic authority). The same architecture that enables life-saving diagnosis enables systematic discrimination. The difference is not technology — it is governance.
Consider the two big governance answers now on the table. The United States backs Stargate, a national-scale build-out of AI infrastructure, on the bet that optimization at scale solves coordination problems and the gains diffuse broadly. The European Union passed the AI Act, which classifies systems like the diagnostic assistant as high-risk and requires transparency, human oversight, and bias audits. Two sincere answers to the same technology — and they are incompatible. Thomas Sowell traced incompatibilities of exactly this kind back to two underlying visions of human nature. In the unconstrained vision, human beings are perfectible — and now, perfectible through technology. In the constrained vision, human nature has permanent limits; concentrated power corrupts, so institutions must check it regardless of technological capability. William Godwin, the unconstrained vision’s founding philosopher, sorted human actions by two criteria — intentional or not, good or bad — and his table is worth a minute of counting:
Intentional good is virtue; intentional bad is vice; unintentional bad is merely bad. But for Godwin there was no unintentional good — he left the fourth cell empty. That empty cell is exactly what Module 2 called emergence: good that arises bottom-up, unplanned, from people seeking win-win exchanges. Whether a vision can see that cell is a fair test of it — and it is the cell this whole site lives in.
Neither vision fully prevails; the US, the EU, and China are running the experiment in parallel. The outcome depends not on which technology wins, but on which governance structures societies choose — and for whom they are designed. (For the full governance story, see Politics and Governance.)
AI amplifies rather than transcends human nature. Whether algorithms become tools of liberation or concentration depends on governance choices, not technological capability.
Adjustable Depth
Overview restates the section in four short paragraphs; Detailed adds the research anchors — calibration, delegation games, generative bubbles — and can be skipped without losing the argument.
AI as cognitive prosthesis (System 3) augments human memory, attention, and reasoning. Chess centaurs — human+AI teams — outperform either alone. But augmentation creates dependency: when AI handles first-draft thinking, the human capacity to evaluate and override weakens through disuse.
Algorithmic bias is not a technical glitch but a structural problem. AI inherits bias from historical training data, scales it to millions of decisions, and automates it so that humans defer to algorithmic authority. The same architecture enables both medical breakthroughs and systematic discrimination.
Two visions compete: the unconstrained vision (build massive AI systems, trust emergent optimization) and the constrained vision (regulate, require transparency, keep humans in the loop). Neither fully prevails globally — the US, EU, and China are pursuing different governance models.
The core insight: AI amplifies human nature rather than transcending it. The outcome depends on governance design — who sets the rules, who is accountable, and whose interests the system serves.
The cognitive surrender phenomenon has historical precedent but new urgency. London taxi drivers who use GPS show measurable hippocampal changes compared to those who navigate mentally (Woollett & Maguire, 2011). Medical residents who rely on diagnostic AI show slower development of clinical intuition. The calibration problem is severe: users tend to either over-trust AI (automation bias — accepting incorrect AI recommendations) or under-trust it (algorithm aversion — rejecting correct AI recommendations), with few achieving the nuanced calibration that centaur models require.
Stanford’s 2025 AI Index documented persistent accuracy gaps across demographic groups in medical imaging, hiring algorithms, and credit scoring. The Workday lawsuit (filed 2023) alleged that AI screening tools produced disparate impact against older workers and disabled applicants — discrimination at a scale impossible for human recruiters. The EU AI Act classifies such systems as “high-risk” and requires conformity assessments, human oversight, and bias auditing before deployment.
The delegation games framework (Riedl & De Cremer, 2025) identifies three independent dimensions of human-AI collaboration: cooperation (willingness to work together), alignment (shared objectives), and calibration (appropriate trust levels). Most failures stem from miscalibration rather than misalignment — users trust AI in domains where it fails and distrust it in domains where it excels.
Generative bubbles (LSE research, 2025) represent a new epistemic threat: AI systems that personalize not just content selection but content generation, creating individualized information environments that are increasingly difficult to verify against shared reality. Unlike filter bubbles, which select from existing content, generative bubbles create content tailored to individual belief systems — a qualitative shift in epistemic risk.
Sowell’s constrained/unconstrained framework, originally from A Conflict of Visions (1987), maps onto AI governance with striking precision. The unconstrained vision (techno-optimism, move-fast-and-break-things, Stargate Project) assumes that human nature is perfectible through technology. The constrained vision (EU AI Act, algorithmic accountability) assumes that concentrated power inevitably corrupts and that institutional checks are essential regardless of technological capability.
These four modules have traced a single thread: from individual bounded rationality, through strategic interaction and institutional design, through the algorithmic reshaping of attention and collective intelligence, to the partnership between human and machine cognition. The pattern is consistent — constraints become structures become power. The cognitive limits that make nudging possible also make manipulation possible. The network structures that enable cooperation also enable cascading failure. The AI systems that augment human capability also concentrate control.
The question is not whether these dynamics exist — they are features of complex systems. The question is whether we design institutions that channel them toward resilience or allow them to drift toward fragility. The remaining modules explore this question in specific domains: history, economics, AI, the digital economy, and governance.