Irene Solaiman, Hugging Face's chief policy officer, came to TechCrunch Disrupt last week with a message that undercuts a lot of AI industry marketing: model benchmarking is not yet the neutral, precise system it's often sold as. "Trust is such a weighted term," she said. The practical test is narrower and harder — can someone outside the lab reproduce the result? That depends on real access to the model, and open weights make all the difference.
Reproducibility is where trust breaks
Solaiman put reproducibility at the center of trust. A benchmark score is only meaningful if an independent team can run the same evaluation and get a comparable result. That has long been one of the stickiest problems in the field, and her framing makes clear it hasn't been solved. Without model access, especially to proprietary systems, verification is limited.
The comparison problem is just as severe. Evaluations are "an evolving science," Solaiman said, and the industry doesn't know how to meaningfully compare one benchmark to another. They also start to become ineffective relatively quickly, while the surrounding evaluation ecosystem hasn't grown thick enough to compensate. It's not just that individual tests can be gamed; it's that the whole science of cross-test comparison is still immature.
The whole system, not just the model
One of the largest shifts Solaiman described is that evaluations are moving from testing the model alone to testing the entire system around it. The harness — the tool a user actually interacts with to reach the model — has a lot to do with performance. She said a weaker model behind a better harness could easily outperform a frontier model. That's an uncomfortable fact for anyone reading leaderboards as product verdicts.
Agentic evals are even harder. "Agentic evals are really hard and really expensive," she said. There are plenty of evaluations for models, but comparatively few for harnesses, and the industry is still developing better evaluations for the holistic agent — the thing users are actually getting value from. The gap means current scores may not reflect the systems people deploy.
Embedded evaluators aren't a shortcut
The push for embedded evaluators — supposedly neutral third-party observers placed inside AI organizations — has gained steam as labs look for an alternative to active government regulation after a series of safety incidents, one of which hit Hugging Face directly. Solaiman is skeptical that the current plays will deliver neutrality or competence. Anthropic's choice of Accenture as its first evaluator drew immediate backlash, with critics questioning whether the consulting firm was qualified for the task.
She outlined the missing foundations:
- Reproducing a score still requires model access, and closed weights make independent verification hard.
- Benchmarks aren't comparable like-for-like; there's no shared method for saying what a cross-test difference means.
- Evaluations decay quickly, but the ecosystem hasn't matured to replace them or set standards.
Her bottom line: the evaluation space is a free-for-all. If benchmarking is going to remain useful, the industry needs a more uniform, shared approach — not an unstandardized gold rush of embedded observers.