Google DeepMind 145-page paper predicts AGI could match human skills by 2030, and warns of existential threats ‘that permanently destroy humanity’

• DeepMind’s latest 145-page safety paper warns AGI could arrive by 2030 and cause “severe harm." However, some experts say the concept of AGI is still too vague and the timeline too uncertain to be properly evaluated.

Google DeepMind says in a new research paper that human-level AI could plausibly arrive by 2030 and "permanently destroy humanity."

In a discussion of the spectrum of risks posed by Artificial General Intelligence, or AGI, the paper states, "existential risks ... that permanently destroy humanity are clear examples of severe harm. In between these ends of the spectrum, the question of whether a given harm is severe isn’t a matter for Google DeepMind to decide; instead it is the purview of society, guided by its collective risk tolerance and conceptualisation of harm. Given the massive potential impact of AGI, we expect that it too could pose potential risk of severe harm."

The statements are contained in a 145-page paper outlining Google DeepMind's approach to AI safety as it attempts to build advanced systems that may one day surpass human intelligence.

The papers' co-authors, who include DeepMind co-founder Shane Legg, did not specifically say how AGI might result in human extinction. And most of the paper is focused on the steps Google DeepMind thinks it and other AI labs should take to reduce the threat that AGI results in what the researchers called "severe harm."

Legg has for decades said that his "median forecast" for AGI's arrival is 2028. Last month, Legg's cofounder, DeepMind CEO Demis Hassabis told NBC News that he thought AGI would likely arrive in the next "five to 10 years," putting 2030 at the earlier end of that range.

The paper separates the risks of advanced AI into four major categories: misuse, which refers to people intentionally using AI for harm; misalignment, meaning systems developing unintended harmful behavior; mistakes, categorized as unexpected failures due to design or training flaws; and structural risks, which refers to conflicting incentives between multiple parties, including different groups of people, such as countries or companies, and possibly multiple AI systems.

The researchers also outline DeepMind's risk mitigation strategy, which is focused on misuse prevention and emphasizes the importance of identifying dangerous capabilities early.

DeepMind also throws some subtle jabs at the AGI safety approaches of fellow AI labs Anthropic and OpenAI. It critiques Anthropic for placing comparatively limited focus on rigorous training, oversight, and security protocols, while accusing OpenAI as being overly focused on alignment research.

Google did not respond to Fortune's request for comment.

The paper has failed to win over some AI safety experts.

Anthony Aguirre, Co-Founder and Executive Director at the AI-safety-focused Future of Life Institute, told Fortune that while the DeepMind team was making an "admirable effort to address the risks of AGI, much more is needed."

“Superhuman artificial intelligence threatens social and political upheaval unmatched in human history," he said. "As the authors themselves indicate, AGI could arrive soon—indeed at almost any time—and could rapidly self-improve and vastly surpass human capability. Such systems are inherently unpredictable, and we are far closer to building them than to understanding how to control them, if it is even possible.” 

There are also questions about the timeline, plausibility, and definition of AGI itself. The concept is not clearly defined, Heidy Khlaaf, chief AI scientist at the nonprofit AI Now Institute, told TechCrunch. She said AGI was still too loosely defined to be “rigorously evaluated scientifically.” 

As of publication time, Google DeepMind had not responded to a request to comment for this article.

While researchers at top AI labs have predicted AGI could arrive in the next five years, many other computer and cognitive scientists remain skeptical that the standard is even achievable with current methods.

For instance, Gary Marcus, an emeritus professor of cognitive science at New York University who has emerged as a leading skeptic of today's approaches to AI, has written that today's AI based on large language models is incapable of matching human-level intelligence across all domains, especially when one considers aspects of human intelligence such as the ability to learn from relatively few examples and common sense reasoning.

Uncertain timelines

In the paper, the Google researchers also say that they "are highly uncertain about the timelines until powerful AI systems are developed," but that "crucially, we find it plausible that they will be developed by 2030."

Google's statement that AGI is "plausible" by 2030 is a slightly lengthier timeline than that adopted by those affiliated with other leading AI labs. Anthropic CEO Dario Amodei, while saying he increasingly finds the term AGI problematic, has said publicly that he expects AI that will surpass human capabilities "in almost everything" will arrive in the next "two to three years."

OpenAI CEO Sam Altman has been more circumspect, but written that "systems that start to point to AGI are coming into view" and also that OpenAI is now "confident we know how to build AGI as we have traditionally understood it."

Meanwhile, former OpenAI policy researcher Daniel Kokotajlo—who resigned from OpenAI last year, claiming the company was being "reckless" in pursuit of AGI—and Scott Alexander, whose Astral Codex Ten blog is widely-read among AI safety researchers, published a scenario online this week that foresees
AI surpassing human intelligence in 2027. Their scenario has received widespread attention on social media and among people working on AI.

With reporting assistance from Jeremy Kahn.

This story was originally featured on Fortune.com