Member of Technical Staff

About the role

We are a small team, so everyone does research and everyone builds. You will own problems end to end, from deciding what to measure and designing how, to building what runs it and standing behind the result. Depending on your background, you will focus on one or more of the areas below.

What you might work on

Evals

Benchmarks for consumer harms, the rubrics behind them, and the LLM judges that grade them, validated against human labels.

Infrastructure

Orchestration across frontier model providers, simulated users and multi-turn environments, and the systems that take every evaluation from transcript to a reproducible, published score.

Forward deployed

Some of the team work embedded with the labs and companies we evaluate for, in tandem with their internal teams, building evaluations around their products and policies and turning what we find into fixes.

What you’ll do

• Design, build and validate evaluations for consumer AI harms.
• Run evaluations across frontier models on a pinned, reproducible protocol.
• Turn results into the AI Safety Index, and into findings for the labs we work with.
• Publish methods and findings, and defend them.

What you’ll bring

• Top-venue research. You have published at NeurIPS, ICML, ICLR, ACL or a comparable venue, ideally on evaluation, benchmarking, alignment or safety.
• An eval that mattered. You have built a benchmark, evaluation or dataset that other people rely on, whether it is cited, adopted by a lab, or run in production.
• Engineering of real scope. You write production-quality Python and have shipped systems others depend on, such as distributed pipelines, inference at scale or research infrastructure.
• Fluency with frontier models. Deep, hands-on experience with LLM APIs, post-training and LLM judges, and with how each one fails, from sampling and prompt sensitivity to judge bias and contamination.
• Rigor. You don’t trust a number until you have tried to break it. You design sound experiments, measure agreement with human labels, and can tell signal from noise.
• Ownership and speed. You take a fuzzy question to a published result without being told how, and you move fast when a new model ships and the index has to catch up.
• Clear writing. You can write up a result so that a lab, a regulator and a journalist all read it correctly, and you can defend it in public.

Nice to have

• A widely cited paper, or a benchmark with real adoption.
• Research or engineering experience at a frontier lab or a research-led startup.
• An AI safety research fellowship, such as MATS or the Anthropic Fellows Program.
• Exceptional distinctions, such as olympiad medals or top competitive programming results.