Dave Schumaker

The Alignment Problem, (released in 2020 but still highly relevant today, especially in the age of generative AI hype), is a fascinating exploration of one of the most interesting issues in artificial intelligence: how to ensure AI systems safely align with human values and intentions. The book is based on four years of research and over 100 interviews with experts. Despite the technical depth, I feel that this book is written to be accessible to both newcomers and seasoned AI enthusiasts alike. A word of warning though: this book is has A LOT of info.

Before we get too deep into this review, let’s talk about safety and what it means in the context of AI. When we talk about AI safety, we’re referring to systems that can reliably achieve their goals without causing unintended harm. This includes:

• The AI must be predictable, behaving as expected even in novel situations.
• It must be fair, avoiding the amplification of existing societal biases.
• It needs transparency, allowing users and developers to understand its decision-making process.
• It must be resilient against failures and misuse.

Creating safe AI tools is both a technical challenge, as well as a psychological challenge: it requires understanding human cognition, ethics, and social systems, as these elements become encoded in AI behavior.

The book is divided into three main sections: Prophecy, Agency, and Normativity, each tackling different areas of aligning artificial intelligence with human values.

Prophecy explores the historical and technical roots of AI and highlights examples of unintended outcomes, such as the biased COMPAS recidivism prediction tool. COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) is a risk assessment algorithm used in the criminal justice system to predict the likelihood of a defendant reoffending. However, investigations revealed that the tool disproportionately flagged Black defendants as higher risk compared to white defendants, raising critical questions about fairness and bias in that AI system.

Agency delves into reinforcement learning and the parallels of reward-seeking behavior in human, showcasing innovations like AlphaGo and AlphaZero. His explanation of reinforcement learning, and its connection to dopamine studies, is particularly insightful. Christian dives into psychological experiments from the 1950s that revealed the brain’s pleasure centers and their connection to dopamine. Rats in these studies would press a lever to stimulate these areas thousands of times per hour, foregoing food and rest. Later research established that dopamine serves as the brain’s “reward scalar,” which helps influence decision-making and learning. This biological mechanism has parallels in reinforcement learning, where AI agents maximize reward signals to learn optimal behaviors.

Normativity examines philosophical debates and techniques like inverse reinforcement learning, which enables AI to infer human objectives by observing behavior. Christian connects these discussions to ethical challenges, such as defining fairness mathematically and balancing accuracy with equity in predictive systems. He also highlights key societal case studies, including biases in word embeddings and historical medical treatment patterns that skew AI decisions.

Christian interweaves these sections with interviews, anecdotes, and historical case studies that breathe life into the technical and ethical complexities of AI alignment.

He also delivers numerous warnings, such as:

“As we’re on the cusp of using machine learning for rendering basically all kinds of consequential decisions about human beings in domains such as education, employment, advertising, health care and policing, it is important to understand why machine learning is not, by default, fair or just in any meaningful way.”

This observation underscores the important implications of deploying machine learning systems in critical areas of human life. When algorithms are used to make decisions about education, employment, or policing, the stakes are insanely high. These systems, often trained on historical data, can perpetuate or amplify societal biases, leading to unfair outcomes. This calls for deliberate oversight and careful design to ensure these technologies promote equity and justice rather than exacerbate existing inequalities. (Boy, oh boy — fat chance of that in light of current events in January 2025)

Christian also highlights some of the strengths of machine learning. These systems can detect patterns in data that are invisible to human eyes, uncovering insights that were previously thought impossible. For example:

“They (doctors) were in for an enormous shock. The network could almost perfectly tell a patient’s age and sex from nothing but an image of their retina. The doctors on the team didn’t believe the results were genuine. ‘You show that to someone,’ says Poplin, ‘and they say to you, “You must have a bug in your model. ‘Cause there’s no way you can predict that with such high accuracy.” . . . As we dug more and more into it, we discovered that this wasn’t a bug in the model. It was actually a real prediction.”

Examples like this show the real-world potential of machine learning to revolutionize fields such as healthcare by identifying patterns that humans might overlook. However, these benefits are accompanied by significant challenges, such as the “black box” nature of AI decision-making, where it remains difficult to determine what features a model is actually using.

Christian shows how understanding these technical challenges, alongside ethical frameworks, can lead to more robust and equitable AI systems. These considerations emphasize the nature of AI safety, which requires combining insights from cognitive science, social systems, and technical innovations to address both immediate and long-term risks.

While the book is dense (very dense!) and information-rich, this strength can also be a drawback. Some sections felt overly detailed, and the pacing, especially in the latter half, left me feeling fatigued.

Despite this, The Alignment Problem remains a compelling and optimistic exploration of how researchers are tackling AI safety challenges. I think this book is an insightful read for anyone interested in AI and will leave you thinking about our future AI overlords long after you’ve turned the last page.