Could an international agreement protect us from dangerous AI? (with Malo Bourgon)

Read the full transcript here. • What are the world’s leading AI companies actually trying to build when they talk about superintelligence? Is the goal merely better chatbots, or systems that could outperform all humans across every cognitive task? Why would such a system be so alluring if it could accelerate medicine, science, education, abundance, and human flourishing? Why would it also create an unprecedented concentration of power for whoever controlled it? If intelligence includes not only abstract reasoning but persuasion, strategy, manipulation, planning, and technological invention, what happens when those capacities are automated at superhuman scale? How seriously should we take AI CEOs when they say the technology could go catastrophically wrong, and how should we interpret the tension between their public concern and their continued participation in the race? If we cannot reliably inspect their goals, motives, reasoning, or learned objectives, how could we know whether apparent obedience is real safety or just surface behavior? Even if alignment were solved, who should be trusted to steer a superintelligence? Could compute governance, chip tracking, training thresholds, inspections, and a US-China agreement buy time before the frontier moves further? What do nuclear weapons, nuclear power, chemical weapons, and germline engineering teach us about the possibility and limits of technological restraint? Is resignation itself part of the danger, and could a credible movement for coordination make a saner future more possible? averages? And when injustice affects both men and women differently, what framework avoids turning that into a zero-sum argument? • Links: • MIRI (https://intelligence.org/) • Malo Bourgon leads the Machine Intelligence Research Institute. Before becoming CEO, Malo served as a program management analyst and then as COO, helping implement many of MIRI’s current systems, processes, and program activities. Malo joined MIRI in 2012 shortly after completing a master’s degree in engineering at the University of Guelph. • Staff • Spencer Greenberg (https://www.spencergreenberg.com/) — Host + Director • Ryan Kessler (https://tone.support/) — Producer + Technical Lead • WeAmplify (https://www.weamplify.info/) — Transcriptionists • Igor Scaldini (https://www.linkedin.com/in/igorscaldini/) — Marketing Consultant • Music • Broke for Free (https://freemusicarchive.org/music/Broke_For_Free/Something_EP/Broke_For_Free_-_Something_EP_-_05_Something_Elated) • Josh Woodward (https://www.joshwoodward.com/song/AlreadyThere) • Lee Rosevere (https://archive.org/details/MusicForPodcasts04/Lee+Rosevere+-+Music+for+Podcasts+4+-+11+Keeping+Stuff+Together.flac) • Quiet Music for Tiny Robots (https://www.freemusicarchive.org/music/Quiet_Music_for_Tiny_Robots/The_February_Album/05_Tiny_Robot_Armies) • wowamusic (https://gamesounds.xyz/?dir=wowamusic) • zapsplat.com (https://www.zapsplat.com/music/summer-haze-slow-chill-out-house-track-with-a-modern-pop-feel-warm-piano-chords-underpin-the-track-with-warm-pads-and-a-repetitive-synth-arpeggio/) • Affiliates • Clearer Thinking (https://www.clearerthinking.org/) • GuidedTrack (https://guidedtrack.com/) • Mind Ease (https://mindease.io/) • Positly (https://positly.com/) • UpLift (https://www.uplift.app/)