AI benchmarks have a recurring problem: once leading models learn to ace them, the tests stop revealing much. A new preprint proposes a different kind of exam whose questions can be generated on demand and assigned a difficulty before humans or frontier models solve them. The method, called C-test ONE, uses smaller language models as reference machines to create and certify sequence puzzles for stronger systems. The paper is under review, and its first results should be treated as an early demonstration rather than a settled measurement standard.

The work comes from José Hernández-Orallo of the University of Cambridge's Leverhulme Centre for the Future of Intelligence and the Valencian Research Institute for Artificial Intelligence. It starts from inverse problems: tasks in which an examinee sees observations and must infer the hidden mechanism that produced them. Sequence completion is the paper's initial format. A problem might mix recalled facts with transformations, requiring the model to identify the rule and produce the next terms.

Abstract sequence puzzles move through composition, verification, rival checking and certification stages.
The pipeline rejects puzzles with competing low-cost explanations before assigning a difficulty and testing models.

C-test ONE borrows from Levin's Kt complexity, which balances the cost of describing a solution-generating program with the time needed to run it. Instead of relying on a minimal artificial computer language, the new method treats a language model as the reference machine and natural-language prompts as programs. Difficulty is expressed in bits. In the paper's interpretation, a linear increase in bits corresponds to exponential growth in the search computation needed to find a solution.

The pipeline has five stages. A composer model creates fresh sequence mechanisms across different subject areas. A verifier checks that a mechanism actually generates the sequence. The system then searches for rival explanations and rejects items if similarly economical rules predict different continuations. Surviving items are certified with a difficulty estimate and finally presented to the models being evaluated. This rejection step is meant to avoid a familiar weakness of pattern puzzles: multiple plausible answers that make the expected continuation debatable.

In the main experiment, the author evaluated 12 language models on 51 items certified with a 14-billion-parameter Qwen model as the reference machine, using three attempts per item. Accuracy declined as the proposed difficulty measure rose. Across the combined results, the rank correlation between measured difficulty and correct performance was negative 0.79. Ability estimates for the models also had a positive 0.76 rank correlation with Epoch's Capability Index for the ten systems included in both comparisons.

Several AI forms reach different thresholds before a sparse frontier of harder unsolved puzzles.
The first battery suggests an evaluable region above current model performance, but that region contains few items and remains preliminary.

The paper's central claim is not merely that hard questions can be produced. It is that a weaker system can help certify a sparse region of problems beyond the reliable reach of stronger examinees. With Qwen3-14B as the reference machine, the paper reports a high-difficulty window from roughly 190 to 380 bits for its strongest tested model. Only ten items were above 190 bits, however, and the author explicitly cautions that the sample is too small for a precise success-rate interval in that region.

Other limitations are equally important. The test currently covers only sequences, not open-ended agent tasks or broader real-world capabilities. Its generator produces English text dominated by Western culture, reflecting the knowledge and biases of the models used to compose and price the questions. The difficulty scale also depends on the chosen reference machine, even though the paper reports that different reference models ranked the examinees similarly.

C-test ONE therefore does not settle how to measure systems that exceed human test writers. It offers a technical route around benchmark saturation: generate problems from compact mechanisms, reject ambiguous items, estimate their search cost and reserve a portion that current models mostly fail. If the approach generalizes beyond text sequences and remains stable across languages and reference models, it could give evaluators a renewable test whose upper boundary moves ahead of the systems taking it instead of waiting for experts to invent another fixed exam.