CHRIS ROHLFS

AI safety & LLM evaluation researcher

I led Trust, Safety, and Quality research at MagicSchool AI, where I built systems that decide whether AI is safe and accurate enough to go in front of millions of kids and teachers. My team built dozens of evaluators and then rebuilt them to cut false positives by 99% while still catching all egregious failures in our curated data.

That work involved designing evaluator rubrics, refining them against human judgment, generating synthetic test cases stratified across user types and risk vectors, red-teaming student-facing chatbots to ensure sound, defensible handling of tough cases, running evals on multimodal data and agentic workflows, and monitoring system behavior on live production traffic. I co-built the company's internal LLM evaluation platform and used it extensively. A colleague and I chatted about it with Anthropic for a customer case study. I code primarily using AI harnesses like Claude Code and OpenCode. I also built research partnerships with the University of Chicago and Stanford to design impact evaluations with school districts.

I came to that work through a doctorate at Columbia on generalization in neural networks—whether a model's measured performance holds up outside the conditions it was measured in.

I spent 11 years building models of financial risk on Wall Street, including advising the CFO and CRO of a $94B firm. And before that, as an economics professor, I researched safety on the highway and the battlefield.

If you believe 360 reviews, word on me is that I understand users' needs, follow through, and stay calm under pressure.

As AI increasingly handles the rote details of everyday life, the premiums are rising on discernment, accountability, and human connection. I'm excited about my role in that transformation: proving safety and accuracy at scale.

I'm currently exploring research, data science, evaluation, and trust & safety leadership roles, and I love to talk about this sort of stuff. Hit me up at chris@rohlfs.ai.