DeepMind AGI Safety Paper Explained: Part 2 — The Four Risks of AGI

What could go wrong with AGI? Alex and Thuy walk through the paper's four risk categories — misuse, misalignment, mistakes, and structural risks — and why DeepMind chose to focus on just two.

In this episode:

• Misuse — When bad actors weaponize AI for cyberattacks, fraud, and disinformation
• Misalignment — When the AI itself pursues goals contrary to developer intent, including deceptive alignment
• Mistakes — Why standard engineering practices handle these and the paper deprioritizes them
• Structural risks — Race dynamics, collective action problems, and why these need governance not just tech
• Four background assumptions — No capability ceiling, 2030 timelines, recursive improvement, and approximate continuity
• The paper's philosophy — Proactive risk management and defense in depth

Paper

• Title: An Approach to Technical AGI Safety and Security
• Authors: Rohin Shah et al. (30 authors), Google DeepMind
• Published: April 2025
• Link: arxiv.org/abs/2504.01849

Series

This is Part 2 of an 8-part series covering the full paper.

---

Hosted by Alex (PM at a fintech scale-up) and Thuy (AI researcher)