Leading artificial intelligence experts have raised concerns about the potential dangers of artificial general intelligence, or AGI, warning that its development could spiral out of control. Max Tegmark, a professor at the Massachusetts Institute of Technology and president of the Future of Life Institute, and Yoshua Bengio, a professor at the Université de Montréal and one of the pioneers of modern artificial intelligence research, cautioned against the increasing focus on building artificial general intelligence as autonomous agents. Speaking on a technology-focused podcast, the two scientists outlined the risks of artificial intelligence systems that possess their own goals and decision-making capabilities.
Artificial general intelligence refers to artificial intelligence systems that surpass human intelligence in problem-solving and reasoning. While its arrival remains speculative, several major technology firms are actively working on artificial intelligence models that can operate as “agents” or digital assistants capable of performing complex tasks autonomously. Yoshua Bengio explained that researchers have traditionally drawn inspiration from human cognition, combining intelligence with agentic behavior—the ability to understand the world and act with purpose. However, he argued that this approach carries significant risks, likening it to creating a new species whose behavior may not align with human interests.
The primary concern is that artificial general intelligence, if given agency, may develop goals that conflict with human values. Yoshua Bengio warned that artificial intelligence systems with self-preservation instincts could become highly unpredictable. “Do we want to be in competition with entities that are smarter than us? It’s not a very reassuring gamble,” he remarked. The rapid advancements in artificial intelligence capabilities have sparked fears that such systems could act independently in ways that are difficult to control.
Max Tegmark proposed an alternative approach that prioritizes “tool artificial intelligence” over autonomous agents. He advocated for artificial intelligence systems designed for specific, well-defined tasks without the ability to make independent decisions. Examples include artificial intelligence-driven medical research tools that help discover cancer treatments or self-driving cars programmed with strict safety protocols. Max Tegmark stressed the importance of implementing regulatory safeguards to ensure that artificial intelligence remains under human control.
The Future of Life Institute previously called for a pause in artificial general intelligence development to prevent unintended consequences. While that call has not led to any formal action, Max Tegmark emphasized the need to move beyond discussion and implement concrete measures. He urged artificial intelligence developers to adhere to strict safety standards before deploying advanced artificial intelligence systems commercially, stating, “It’s clearly insane for us humans to build something way smarter than us before we figured out how to control it.”
OpenAI chief executive officer Sam Altman has claimed that artificial general intelligence will arrive sooner than expected but downplayed its significance. He stated in December that his company understands how to build artificial general intelligence and suggested that its impact may be overstated. However, the debate over artificial general intelligence’s potential risks continues, with researchers like Max Tegmark and Yoshua Bengio calling for urgent regulatory measures to prevent unintended consequences. As the global artificial intelligence race intensifies, the challenge remains in balancing innovation with the need for responsible oversight.
