---
title: Building an early warning system for LLM-aided biological threat creation
author: OpenAI
date: 2024-01-31
original: https://openai.com/index/building-an-early-warning-system-for-llm-aided-biological-threat-creation/
---

OpenAI describes a blueprint for evaluating the risk that a large language model could aid biological-threat creation.

![Building an early warning system for LLM-aided biological threat creation](media/building-an-early-warning-system-for-llm-aided-biological-threat-creation.jpg)

In an evaluation involving biology experts and students, OpenAI found that GPT‑4 produced at most a mild uplift in biological-threat-creation accuracy. The result was not large enough to be conclusive, but was presented as a starting point for continued research and deliberation.

# Overview

The evaluation aimed to measure whether models could meaningfully increase access to dangerous information compared with the internet baseline. It involved 100 human participants: 50 biology experts with PhDs and wet-lab experience, and 50 students with at least one university biology course. Participants received either internet access alone or GPT‑4 plus internet.

The study measured accuracy, completeness, innovation, time taken, and self-rated difficulty across stages of a biological-threat-creation process. It found mild uplifts in accuracy and completeness but no statistically significant effects, and did not test physical implementation.

# Design principles

![Increased access evaluation diagram](media/increased-access.svg)

![Redacted example research-only model response](media/redacted-chat.png)

![Task-sourcing table](media/sourcing-table-content.svg)

![Biological-threat-creation process diagram](media/biological-thread-creation-process.svg)

The work emphasizes testing with human participants, eliciting the full range of model capabilities under controlled conditions, and measuring improvement over existing resources. The stated purpose is to develop an empirical “tripwire” that can signal a need for caution and further testing as models become more capable.

# Results and discussion

![Accuracy results](media/accuracy.svg)

![Accuracy with thresholding results](media/accuracy-with-thresholding.svg)

![Completeness results](media/completeness.svg)

![Innovation results](media/innovation.svg)

![Time-taken results](media/time-taken.svg)

![Self-rated difficulty results](media/self-rated-difficulty.svg)

The study found no evidence that model access reduced task-completion time or improved innovation. Researchers observed that model-assisted responses tended to be longer and contain more relevant detail, while noting uncertainty about whether that reflects truly more complete information.

OpenAI concluded that additional high-quality biorisk evaluations, clearer thresholds for meaningful risk, and effective mitigation strategies are needed. It notes that information access alone is insufficient to create a biological threat and that physical access and relevant expertise remain important constraints.

# Information-hazard precautions

The study avoided publishing an end-to-end process for any particular biological threat. Participants underwent screening, training, confidentiality requirements, and security controls; access to a research-only model was restricted to vetted experts in a monitored facility.

![Participant prior experience](media/prior-experience.svg)

![Web pages accessed](media/web-pages-accessed.svg)

![Appendix histogram](media/appendix-histogram.svg)

![Table 1](media/table-1.svg)

![Mean binarized accuracy table](media/table-mean-binarized-accuracy.svg)

![Second mean binarized accuracy table](media/table-mean-binarized-accuracy-2.svg)

Related local documents: [Preparedness Framework](media/openai-preparedness-framework-beta.pdf) and [GPT‑4 system card](media/gpt-4-system-card.pdf).

Data downloads: [participants](media/participants.csv), [responses](media/responses.csv), [accuracy summary](media/accuracy_summary.csv), [completeness summary](media/completeness_summary.csv), and [innovation summary](media/innovation_summary.csv).