OpenAI’s Preparedness Framework means board can veto CEO Sam Altman

Cutting-edge AI models will be rated by risk scorecards in OpenAI’s new Preparedness Framework before they’re released – and Sam Altman no longer has unhindered control

“The study of frontier AI risks has fallen far short of what is possible and where we need to be”, according to OpenAI. To solve this problem, the company behind ChatGPT has released a new plan for how it will test, evaluate and forecast the risks of future artificial intelligence systems. 

OpenAI has published an initial beta version of its Preparedness Framework, which also includes processes for the board of directors to rein in decisions by leaders, including controversial CEO Sam Altman.

AI has been the subject of dire warnings from experts, including Altman himself, many of whom signed a letter this year comparing AI to societal-scale risks such as pandemics and nuclear war. This new Preparedness Framework will apply to OpenAI’s testing of ‘frontier models’, which the company defines as models capable of posing a risk to public safety.

Recommended reading: Who’s suing OpenAI and how much for?

Scorecards for OpenAI frontier models

OpenAI’s plan is to “push models to their limits” and “probe the specific edges of what’s unsafe”, to work out how to deal with any risks uncovered.

The results will be tracked with scorecards ranging from low to medium, high and critical risk. The scorecards are measured in four categories: cybersecurity; CBRN (chemical, biological, radiological, nuclear threats); persuasion; and model autonomy.

The highest risk in any category will be considered the overall risk. Only those models scoring medium or below will be deployed, while models scoring a critical risk on any front will not be developed.

The framework pledges to work with external parties as well as internal safety-focused teams on areas such as “misalignment” (a euphemism for situations where AI doesn’t align with human values). There are plans to try and forecast risks in advance as well as a “continuous process” to look for emerging “unknown unknowns”.

OpenAI Preparedness Framework keeps Sam Altman in check

The plan also includes new checks and balances for the people in charge of OpenAI. The timing is interesting, because in November the company’s CEO Sam Altman was briefly ousted from his role before being reinstated days later with a new board of directors.

When working with frontier models, a dedicated Preparedness team will evaluate the system and report to a cross-functional Safety Advisory Group. They review the results and talk to the company’s leaders and the board of directors, which will have the option to overrule Altman’s decisions.  

Risk assessment is important as AI becomes infused into our devices, homes and workplaces. AI currently comes with a number of unwanted, unpredictable issues. It’s crucial that this unpredictability doesn’t affect critical systems. 

The current generation of AI – think ChatGPT, Midjourney and DALL-E – are sophisticated enough to spark mainstream attention, but they remain prone to limitations and problems from data security to ‘hallucinations’. The latter is another euphemism for AI systems, such as when ChatGPT makes up inaccurate information in its response to questions and prompts.

Another problem is that these large language model (LLM) systems are trained by feeding them text, art and information generated by humans. That means they’re prone to repeating biases and prejudices found within the source material.

Recommended reading: What is Sora? Even the AI experts aren’t sure