On March 22, an open letter published by the Future of Life Institute and signed by prominent tech CEOs and researchers (including Turing Prize winner Yoshua Bengio and leading AI risks expert Stuart Russell) called for a 6-month pause on giant Large Language Model (LLM) experiments. This comes a few weeks after the release of GPT-4, and a few months after the release of ChatGPT. LLMs are being widely adopted in different sectors of the economy. Major publishers like Time in the US and The Guardian in the UK are now discussing risks linked to transformative AI. Fox News’ Peter Doocy asks questions about existential AI risks to the White House Press Secretary. US president Joe Biden says AI systems should be safe before release.
What do people want to slow down?
Most advocates of a pause want to set basic guardrails to ensure LLMs are safely developed, understandable, and secure before deployment, while continuing progress on other AI systems (as long as they don’t present risks).
Why Slow Down?
- Essentially, to avoid a race to the bottom, i.e. a race where companies try to be the first to deploy a technology cheaply, at the expense of ensuring its safety. That’s a classic problem in safety-critical industries, such as healthcare or aviation. Given the economic incentives to develop advanced AI, corner-cutting seems poised to define the relationship between AI labs (and between the states in which they operate).
- A pause in large language model development would allow society to adapt and decide on the extent of its deployment, considering its exponential growth and impact on up to 300 million full-time jobs, including white-collar roles. This break would also provide time for policymakers to evaluate and update laws related to intellectual property, liability, discrimination, and privacy, enabling the establishment of basic guardrails for safer future technology and ensuring legal certainty for AI development and deployment.
- A substantial fraction of AI experts worry about existential accidental risks. Among those expressing concern is Geoffrey Hinton, godfather of the deep learning revolution. Another, Dan Hendrycks, is a leading Machine Learning researcher, expert in evaluating AI systems, and the inventor of GeLU, an important component of most frontier AI systems. He has said publicly that the chances that we go extinct are as high as 80%. He defends his position more thoroughly in a paper which states that “Natural Selection Favors AI over Humans”. Zooming out, a poll on US public opinion reveals that 46% of the population is “somewhat” to “very concerned” about AI existential risks.
- Concerns about misuse, including high-consequence risks, also lend support to a pause in LLM development. Europol warns that enhanced disinformation campaigns and the empowerment of criminal actors are growing threats. LLMs are also disrupting the global cybersecurity landscape by facilitating large-scale search of security failures and the creation of rapidly mutating viruses. Without countermeasures, frequent and large-scale cyberattacks could occur, and advanced AI systems might make chemical and biological weapons more powerful and easier to create. We’ll discuss those risks more in future newsletters.
Proposals & Criticisms
As a result of growing awareness of those issues, calls for slowing down AI development are on the rise. But proposals vary a lot on their strength.
Most arguments against a slowdown come down to a) it wouldn’t work and b) it delays too much the benefits from AI. The second argument illustrates the risk-benefit trade-off at the center of most policy debates. Thus, a pause should be as short and targeted as possible. Some of the people worried about AI risks have indeed proposed to accelerate AI safety research to avoid having to excessively slow down AI development, and thus postponing its benefits too much.
The Open Letter itself calls for a 6-month pause on large AI training runs to develop guardrails and auditing procedures. Some view it as a step towards establishing basic safety requirements in the industry. Others think the plan would be counterproductive, and increase AI risks: If the training of AI systems is restrained while research to improve the training process continues, it could result in sudden capability jumps and increased safety risks when training resumes (at the end of the proposed pause). As AI governance analyst Matthew Barnett puts it, “continuous progress is more predictable, and better allows us to cope with challenges as they arise, compared to the alternative in which powerful AI suddenly arrives.”
Independently of this factor, implementing robust guardrails will be challenging, in part due to the complex and opaque nature of LLMs. OpenAI supports some measures advocated by the Open Letter, such as "independent audits," "independent review before training future systems," and limiting compute growth for advanced models. However, the key area of disagreement is about when the best time to pause is. Because we know very little about the current risks of models and the future pace of AI progress, there’s no authoritative answer to this question.
On the other end of the spectrum lies Eliezer Yudkowsky, founder of the Machine Intelligence Research Institute and precursor in the study of Artificial General Intelligence (a hypothetical AI system that can do anything a human brain can do). He believes the Open Letter doesn't go far enough, and suggests waiting until the "alignment problem" (ensuring AI systems do exactly what their designers want) is confidently solved before proceeding with LLMs development. If we don’t, Eliezer claims, “literally everyone on Earth will die”. Other critics argue that a full shutdown is inadequate due to uncertainty around existential risk.
Between those two ends of the slow down proposals spectrum lies conditional slowdown, which roughly consists in conditioning the slowdown to specific dangerous capabilities bars being hit. So for instance, if a model becomes better than the top 10% hackers, as measured by an independent auditor, the regulator would prevent AI labs from dedicating R&D to more powerful capabilities until they have managed to implement adequate countermeasures (such as, in this case, removing or offsetting those hacking abilities by enhancing the state of global cybersecurity). Conditional slowdown could also go the other way, allowing companies to develop models only if they fulfill specific conditions (such as “benchmarks of explainability/transparency” or firm-level safeguards). This approach incentivizes labs to prioritize safety, differentially accelerating safer actors if properly enforced.
Difficulties and Countermeasures
As you can see, proposals are vastly different from each other, ranging from a gradual approach to a full and complete ban on AI capabilities research. But for all of those, challenges have been said to stand in the way:
- “Coordination is hard. Those proposals will never see the light”. This argument seems to understate a few factors that are crucial to the situation we’re in:
- Most importantly, key AI leaders are US-based and have at least a few years of technological lead, making a unilateral slowdown feasible without harming US AI leadership. Unlike climate change, responsibility lies with few actors, and other countries, not benefiting from the technology race, may support an international slowdown plan.
- Because of its enormous economic and security benefits, it seems hard to imagine a world where public authorities or even companies strive to slow down AI progress. But history shows several examples of technologies that were deliberately not pursued despite their benefits. And both political elites and the general public in the US are increasingly in favor of regulating AI. If this trend persists, the idea of slowing down AI development may not seem as far-fetched as it currently appears.
- “A slow down will allow China to surge ahead of the US in AI development”.
- If we take the warning of existential risks from experts seriously, whether advanced AIs are developed by the US or China, humanity risks becoming extinct (as long as we don’t have the technology to control those systems, and we don’t seem to be on track to getting it anytime soon).
- China is very likely to be a few years late in terms of technology given the information available on the current state of its technology industry. It’s not even clear they’re matching the capabilities of second tier US players like Adept.ai or Cohere.ai (which are already lagging a few years behind OpenAI and probably Anthropic). Given that, China may be willing to agree to a slow down.
- China's AI industry has been severely affected by US export controls and is expected to fall significantly behind the US in the next few years. The gap between the two countries' capabilities may widen even more with the upcoming introduction of NVIDIA's state-of-the-art high-end computing chips (H100), which won’t be available in China (due, again, to US export controls). Moreover, the US is very likely to continue trying to slow China's tech industry, especially AI (for example by preventing certain American companies from investing in China or further restricting exports). As the US Undersecretary of Commerce, responsible for export controls, puts it, “If I was a betting person I would put down money on [additional export controls, including on AI]”.
- Even just a unilateral US pause on AI development could be beneficial. It would signal to China that the US is not willing to do “whatever it takes” to get ahead (i.e. sacrifice safety in favor of capability), thus reducing dangerous racing dynamics and potentially leading China to be less scared of slowing down AI development as well.
Overall, many agree that a slowdown would help with navigating AI risks successfully. Everyone agrees that slowing down won’t be easy to implement in the right way. But everyone is also surprised by how much more people than expected seem sympathetic to it. Which means that coordination to make transformative AI development go well might be much easier than many thought. That's great news for AI risk management!
