The AI literature identifies numerous potential risks from general purpose AI models (Bengio et al., 2025). However, we still lack established methodologies for quantitatively assessing the actual risks these models pose to society. Most current AI risk assessment approaches don't measure risk directly, but instead focus on various imperfect proxies, particularly model capabilities.
At SaferAI, we therefore currently focus our research on developing rigorous, quantitative methods for assessing and analyzing AI risk. Our interactive visualization demonstrates this approach using a cyber risk scenario as an example.
What You'll See in This Visualization
The first panel outlines our cyber risk scenario, which unfolds in six sequential steps—from initial attack to economic damage. We provide quantitative estimations for each element in this risk model.
The second panel presents quantitative estimation of the scenario, using data collected in August 2024 through a structured Delphi process. This data was gathered from a group of Superforecasters from Good Judgment, Inc.—experts increasingly relied upon for AI risk assessments (Phuong et al., 2024).
Interactive Features
As you navigate through the visualization, you'll see for each step:
Step 4 includes an additional layer of detail: a mapping between the probability of successfully completing this step and an LLM's score on cybersecurity benchmarks. For more details on this methodology, see our recent paper (Murray et al., 2024).
The final view presents a complete probability distribution of potential economic damage from the risk scenario, comparing outcomes with and without LLM involvement.
This visualization aims to enhance understanding of AI risk among regulators and policymakers while demonstrating the value of quantitative risk assessment methods for AI companies and academic experts.
For optimal viewing experience, we recommend using a laptop display.