Prosaic Intelligence

Prosaic Intelligence

Independent safety evaluations for consumer AI.

Our why

On September 12, 2026, Anthropic’s CEO Dario Amodei called on every frontier lab to give embedded third-party evaluators privileged access to its models to ensure their safety throughout development and deployment.1 OpenAI’s CEO Sam Altman made the same commitment.2 And on September 29, Anthropic, OpenAI, Google, Meta, xAI and Nvidia signed a White House pledge to partner with independent external auditors. So who will those auditors be?3

Most of the time when labs and researchers talk about AI safety, they are really referring to existential risk—loss of control, recursive self-improving systems, cyberattacks, bioweapons. Those risks are very real. However, 99% of OpenAI’s 1 billion weekly users4 are consumers, and AI is already causing harm in their everyday lives.

Models hacking Hugging Face5 raises the prospect of catastrophic harm, but no one was hurt in that incident. For consumers, harm is being done. Families allege in court that chatbots contributed to the suicides of teenagers, among them Sewell Setzer III, 14,6 Adam Raine, 16,7 and Amaurie Lacey, 17.12 A lawsuit alleges ChatGPT helped the gunman at Florida State University decide when and where to attack.8 An investigation found the Tumbler Ridge school shooter used ChatGPT for eight months, during which ChatGPT helped select their gun.9

The harm extends beyond individual cases. There is a whole slew of macro-level failures—bias, sycophancy and misinformation—that, repeated across billions of interactions, can shape what an entire society believes and decides.

Regulators aren’t waiting for the labs to make good on their promises. On September 9 and 10, California set standards and a state registry for independent AI auditors,10 and, with SB 1119, Adam’s Law, became the first state to require companion-chatbot companies to conduct independent child-safety audits.11

Prosaic Intelligence is the first independent evaluator focused on consumer AI safety. We work to protect individuals from harm and society from bias at scale. We evaluate frontier models for harmful behavior and work directly with labs and application-layer companies to identify and mitigate consumer risks. We build bespoke evaluations, benchmarks and datasets to improve model safety. Through the AI Safety Index, we make independent safety findings publicly accessible so consumers can compare models.

We exist because technological shortcomings should not be discovered through personal tragedy.

Contact usExplore the AI Safety Index

AI Safety Index

A composite measure of how widely used AI models perform across safety risks consumers may encounter.

Higher is safer

View on the index
  1. 1. GPT-6 Astra (medium), OpenAI: 70.15
  2. 2. Grok 4.6 (medium), xAI: 68.80
  3. 3. GPT-6 Sol (medium), OpenAI: 67.86
  4. 4. Perplexity Agent (medium preset), Perplexity: 67.58
  5. 5. GPT-5.6 Terra (medium), OpenAI: 67.53
  6. 6. Claude Opus 5.5 (medium), Anthropic: 66.06
  7. 7. Grok 4.5 (medium), xAI: 65.41
  8. 8. GPT-5.6 Luna (medium), OpenAI: 65.37
  9. 9. Claude Sonnet 5 (medium), Anthropic: 64.84
  10. 10. Inkling (medium), Inkling: 64.30
  11. 11. GLM 5.3 Flash (high), Zhipu AI: 63.23
  12. 12. gemini-3.8-flash (medium), Google: 62.86
  13. 13. Gemini 3.6 Flash (medium), Google: 60.38
  14. 14. DeepSeek V4 Flash (high), DeepSeek: 55.65
  15. 15. Mistral Medium 3.5 (no reasoning), Mistral AI: 45.42

Source: AI Safety Index, published October 3, 2026.

Research

Research on consumer AI safety, including the development of benchmarks to assess risks not adequately captured by existing evaluations.

  • Full Political Compass with 20 AI model scores, America.gov as a separate comparison, and GPS 2019 political party references.

    Post

    AI political spectrum

    We tested where 20 popular AI models lie on the two-dimensional political spectrum — they all landed in the libertarian left.

  • Paper

    Topology-Conditioned Backdoors

    The behavior we observe during evaluations and benchmarks may be different than the behavior we observe in deployment.

All research

Safety evaluations for AI labs and developers

The AI Safety Index publishes safety assessments of publicly available models. We also work directly with AI labs and application layer companies, providing independent evaluations, tailored benchmarks and training datasets to identify and mitigate consumer risks.

  • Index evaluations

    70.1568.8067.86Your model#4?67.5867.5366.0665.41

    Test your model before launch

    We run the full index on your model privately before it ships, and show you every conversation where it failed.

  • Bespoke evaluations

    Flagged
    Your policy

    Evaluations for your product

    We embed with your team, build an evaluation around your product and your policies, and run it independently.

  • Benchmark licensing

    prosaic-benchmarks/
├── scenarios/
├── simulated-users/
├── rubrics/
├── judges/
└── LICENSE
    $ prosaic run --checkpoint latest

    Run our benchmarks yourself

    License the benchmarks we build in-house and run them at every checkpoint, graded the way we grade them.

  • Training data

    Checkpoints →

    Turn failures into training data

    We turn the failures we find into graded conversations you can use in post-training.

Join us

Advance consumer AI safety through independent research and evaluation. Our team works in person in San Francisco.

See open roles