LEVI: Can AI Image Generators Preserve the Details That Define Visual Identity?

Why have large language models made such rapid progress in coding and mathematics over the past two years?

Part of the answer lies in the infrastructure used to evaluate them. These fields illustrate a powerful principle: reliable verification provides a strong foundation for improving model capabilities.

Code and mathematics benefit from compilers, tests, and formal verifiers that can check whether an output satisfies specific requirements. Visual generation has long lacked an equivalent standard for many of its tasks.

Precise verification requires a clear definition of the target identity. Conventional aesthetic scores and image–text similarity metrics do not establish when images with very different appearances express the same underlying identity. Without that distinction, it is difficult to provide precise feedback for aligning visual generation models.

As an international community supporting open collaboration on AI data and evaluation, ZODA recognizes the need for this infrastructure. Jointly developed by ZODA and Imperial College London, LEVI: Benchmarking Visual Identity Fidelity with Verifiable Low-Entropy Constructs addresses this problem.

1. Why Do Existing Benchmarks Fall Short on Visual Identity Fidelity?

1.1 Limitations of Existing Visual Generation Benchmarks

Recent years have brought a growing number of benchmarks for text-to-image generation. Some incorporate executable checks, yet many still leave a fundamental question undefined: when do two images that look different express the same target identity? The comparison below summarizes these limitations across existing benchmarks.

Three limitations stand out:

• Limited support for verifiable structures with explicit semantics. Pixel-level or text-similarity scores alone cannot establish whether the structural rules defining an image’s content have been satisfied.

• Limited support for standalone and composed identity requirements. Evaluation needs to express both the identity of an individual component and the relationships that define a combination of components.

• Insufficient separation of appearance variation from identity errors. Changes in typography or material may preserve identity, but conventional similarity metrics do not explicitly treat all permitted visual realizations as equivalent.

1.2 What Is Low-Entropy Visual Identity?

Addressing these limitations requires distinguishing two aspects of controlled image generation: the identity-defining facts that must remain correct and the appearance that may vary. Pixel- or feature-level scores do not explicitly separate permissible creative variation from changes to the requested identity.

We introduce Low-Entropy Visual Identity (LEVI) to make this distinction clear. When an instruction fully specifies an identity, uncertainty about that identity should be zero, while its visual realization—including medium, lighting, and material—can remain diverse.

LEVI establishes three evaluation requirements:

• Support the construction of verifiable visual structures with explicit semantics.

• Express both standalone and composed identity constraints.

• Treat different permitted appearances of the same identity as semantically equivalent, such as printed and handwritten versions of the same text.

2. How Can We Define Rules for Visual Identity?

To define identity rigorously, we introduce Visual Identity Spaces (VIS).

• Equivalence classes and mappings. Given a set of identity rules, images expressing the same identity belong to the same equivalence class. These classes form a quotient space, and a mapping assigns each image realization to its corresponding identity.

• Composition and operations. VIS supports standalone identities involving strings, geometry, and topology, as well as compositions governed by shared constraints. Projections, transformations, and refinements provide ways to describe relationships within and across identity spaces.

VIS provides the mathematical semantics of visual identity. To turn those semantics into tasks that can be constructed and checked, we introduce Verifiable Visual Grammar (VVG). Through typed components, attributes, and compositional constraints, VVG describes visual structures with explicitly defined identities.

VIS and VVG connect task construction, image generation, and verification through a shared definition of identity. The pipeline has three steps:

• Task construction: VVG rules and text prompts specify the task’s identity constraints.

• Image generation: A model generates an image from the text prompt.

• Evidence readout and fixed-rule checking: A separate reader extracts visual evidence from the generated image. A fixed checker then evaluates that evidence under the same identity semantics used to construct the task. This separation preserves the distinction between a visual measurement failure and an observed identity violation.

3. Testing 1,375 Tasks: The Highest Whole-Task Pass Rate Is 58.69%

To put this framework into practice, LEVI provides a benchmark containing 1,375 tasks across six visual identity domains. The six domains cover a wide range of precise visual requirements:

• Symbolic: Latin letterforms, musical symbols, ordered strings, and shared strokes. For example, a task may require notes at specified staff positions with the correct open or filled noteheads, testing whether the symbolic content is preserved exactly.

• Geometric: Equally spaced spokes, height ordering, tick alignment, and mirrored contours. Tasks may require uniformly spaced wheel spokes or a level horizon within a rotated framed photograph.

• Topological: Interwoven five-ring structures, partitioned grids, node degrees, and specified over–under relationships between ribbons and letter strokes, including different crossing orders for the letters E and F.

• Layout: The repeating two–three grouping of piano black keys, occupancy of battery-compartment slots, and exact correspondence between stained-glass patterns related by a 90-degree clockwise rotation.

• Quantitative: Playing-card suit counts, counts across tiers of a fruit display, clock readings, and independent dot counts in the four wings of a butterfly-shaped cookie: two at the upper left, four at the upper right, one at the lower left, and three at the lower right. The asymmetric counts test whether models preserve explicit requirements rather than impose bilateral symmetry.

• Procedural: Successive stages of stroke accumulation, origami crease states, consumption of layered candles, and a partially piped icing heart with the nozzle at the bottom endpoint.

How do visual generation models perform on these tasks?

The evaluation covers ten model configurations, including the GPT Image series, Qwen Image 3 Pro, Seedream 5 Pro, and FLUX.2 Max. GPT Image 2.5 Sunburst achieves the highest whole-task pass rate at 58.69%. Even the leading configuration therefore receives no credit on more than two fifths of the tasks in the evaluated challenge set. On 243 tasks—17.67% of the total—none of the ten configurations receives a passing judgment.

Clearly defining the boundary between identity certainty and appearance diversity makes it possible to provide more targeted, verifiable feedback. Such feedback could support tighter generation–verification loops and future work on iterative model improvement and self-evolution.

4. Conclusion

Progress in model capabilities often depends on evaluation systems that define what a correct result looks like. LEVI exposes current limitations in visual generation and provides a foundation for more precise, verifiable evaluation. ZODA is committed to investigating the questions that shape the future capabilities of AI. We welcome researchers, experts, and industry partners to join us in advancing visual generation and its evaluation.