Anthropic's recent update to their Responsible Scaling Policy resulted in a downgrade of their safety rating from 2.2 to 1.9, placing them in the "weak" category alongside OpenAI and DeepMind. The downgrade is due to the new policy's shift away from precise, quantitative benchmarks and mitigation measures toward more qualitative descriptions, which reduces transparency and accountability.

We produced the first risk management rating on artificial intelligence risk management practices. Our analysis documents significant shortcomings across the industry.

Following a workshop focused on risk management frameworks and risk thresholds for frontier AI, which brought together top experts in AI risk management and key policymakers, we present six expert-driven claims that emerged from the discussions.

To succeed, the EU Code of Practice will have to rely on a fast evolving scientific literature. That's why we drafted "An Opinionated Literature Review to Inform the EU Codes of Practice on GPAI with Systemic Risks": to provide, in one single document, pointers to what we judge to be the most helpful references for each of the items that the GPAI Code of Practice will have to cover.

In late 2023, SaferAI ran a workshop on AI misuse to foster consensus amongst key experts in the field. As part of this workshop, we drafted a collaborative document providing a brief literature review on specific facets of AI risk, which we thought would be useful for the broader public.

In conversations about AI safety, you often hear people mention audits, red-teaming, and evaluation. While related, there are important differences. In short, evaluations (evals) and red-teaming can be a part of an auditing process. A full audit looks at both the AI model and the organization. Here's how we think about the relationships between the three concepts.

Reminiscent of the release of the ill-named Responsible Scaling Policies (RSPs) developed by their rival Anthropic, OpenAI has just released their Preparedness Framework (PF) which fulfills the same role. How do the two compare?

We compare Anthropic's "Responsible Scaling Policy" with the risk management standard ISO/IEC 31000, identify gaps and weaknesses, and propose some pragmatic improvements to the RSP.

There are three basic criteria I think will make AI risks manageable. For good risk management, models need to be interpretable, boundable and corrigible.

Our world is one where AI advances at breakneck speed, leaving society scrambling to catch up. This has sparked discussions about slowing AI development. We explore this idea, delving into the reasons why society might want to have slowdown in its policy toolbox. This includes preventing a race to the bottom, giving society a moment to adapt, and mitigating some of the more worrisome risks that AI poses.