Research: The gold standard for GenAI evaluation

How do we evaluate systems that evolve faster than our tools to measure them? Traditional machine learning evaluations, rooted in train-test splits, static datasets, and reproducible benchmarks, are no longer adequate for the open-ended, high-stakes capabilities of modern GenAI models. The core proposal of this position paper is bold but grounded: AI competitions, long used to crowdsource innovation, should be elevated to the default method for empirical evaluation in GenAI. These competitions are not just practical; they are structurally superior in ensuring robustness, novelty, and trustworthiness in results.

Why traditional ML evaluation no longer works

Most conventional LLM evaluation setups rely on the assumption that training and test data are drawn independently from the same distribution. This foundational idea has enabled the field to develop reproducible benchmarks such as MNIST or ImageNet, which in turn fueled decades of progress. But GenAI models do not operate in these narrow, well-bounded environments. They produce language, images, and code in open domains with no clear ground truth. Inputs can be ambiguous, and outputs vary in form and quality. These models often use prior outputs as context for future ones, creating feedback loops that undermine core statistical assumptions.

As a result, benchmark scores may say less about model quality and more about whether test data leaked into training. And once a benchmark is made public, the assumption must be that it has already been compromised. In such a landscape, reproducibility and robustness cannot be equally prioritized. Evaluations must now be viewed as processes rather than static objects.

The current environment demands a redefinition of generalization. Instead of asking whether a model performs well on new data from a known distribution, we must ask whether it succeeds at solving entirely unfamiliar tasks. This novelty-centric approach is more aligned with how humans assess intelligence. It places a premium on adaptability rather than memorization.

This shift comes with trade-offs. Benchmarks cannot be reused without risking contamination. Evaluation tasks must be generated dynamically or designed to be unreproducible by nature. These requirements make competitions, which excel at managing novelty and scale, the ideal framework.

Leakage and contamination

Leakage is not a fringe concern. It is a pervasive, often undetected problem that can invalidate entire evaluations. When evaluation data overlaps with training data, even unintentionally, scores are inflated. GenAI models are especially prone to this because their training data is often vast and poorly documented.

Competitions have shown how leakage arises through metadata, time-based artifacts, or subtle statistical cues. They have also pioneered solutions: hidden test sets, randomized sampling, and post-deadline evaluation. These practices, developed to prevent cheating, now double as scientific safeguards.

AI competitions enable parallelized, large-scale evaluation. Thousands of teams work independently to solve the same task, surfacing diverse strategies and approaches. This scale allows for empirical insight that static benchmarks cannot match. More importantly, it distributes the burden of validation and reveals weaknesses that isolated tests may miss.

By keeping evaluation data private and execution offline, competition platforms prevent leakage at a structural level. They create a trusted environment where results are both comparable and credible. Transparency also plays a role. Participants often share code, logs, and failure modes, creating a culture of openness that traditional research lacks.

Designing for leak resistance

Competitions also offer architectural blueprints for evaluation. Strategies include:

• Prospective ground truth: Labels are collected after model submissions. For example, protein annotation tasks have used future lab results as evaluation targets.
• Novel task generation: Challenges such as AI Mathematical Olympiad use fresh, human-designed problems to ensure models have not seen similar data.
• Post-deadline testing: Submissions are frozen and tested later on unseen data, avoiding any chance of prior exposure.

These methods are more than clever—they are necessary. As models improve, the evaluation standards must also become more robust and resistant to exploitation.

Other novel approaches are gaining traction. LiveBench continuously updates its test data from recent publications. Community platforms like LM Arena crowdsource head-to-head comparisons using real-time prompts. These formats are innovative and useful, but they come with their own risks. Public inputs can still lead to contamination, and crowd judgment may skew results in subtle ways. Competitions, by contrast, allow for curated control without sacrificing scale.

The paper ends with a call to action. To maintain credibility in GenAI research, the field must:

• Deprioritize static benchmarks in favor of repeatable, renewable evaluation pipelines.
• Treat AI competitions as core infrastructure for measuring model progress, not as side activities.
• Apply anti-cheating protocols developed in competitions as standard practice in evaluation design.
• Embrace meta-analyses of competition results to uncover broad insights across tasks and models.

These changes would align incentives across academia, industry, and open-source communities. More importantly, they would restore trust in empirical claims about model performance.

Featured image credit