Visual Quality Intelligence, Part 1: Why Machines Need to Judge What They See

In the last 12 months, visual generation went industrial. Video models like Google's Veo and ByteDance's Seedance entered production. World models like DeepMind's Genie 3 and World Labs' Marble began generating explorable environments. And that wave landed on top of a layer that has been running for years: super-resolution, frame generation, and restoration models inside games, phones, and streaming pipelines, quietly redrawing most of the pixels people see. Generative models alone produce an estimated 80 million new images every single day.

This raises a question that gets far less attention than the generation itself: who judges all of it?

Visual AI creates far more than anyone inspects. The volume outgrew human review the moment generation became a background process.

Two research fields were built for exactly this gap. Image quality assessment (IQA) and video quality assessment (VQA) have studied machine judgment of visual quality since before deep learning. Both fields rest on the same premise: quality has no objective definition. The ground truth is human raters, averaged into mean opinion scores (MOS), and every metric since is a learned stand-in for asking people. Wang, Bovik, Sheikh, and Simoncelli's SSIM, published in 2004, still runs inside today's pipelines, and everything after it is a better attempt at the same prediction. Hidden in that lineage is a tension the field took years to formalize: faithfulness to a reference and quality to a human eye are not the same goal.

The stand-in metrics themselves evolved through several questions. The earliest metrics measured how far an image drifted from its original. Later ones checked whether an image still had the statistics of a natural photograph. Today's compare what a neural network's features see. Each question defined an era, and tools from every era are still running in production today.

Two recent shifts turn this from an academic question into a production problem.

The classic benchmarks saturated, but only where the field has looked longest. On the standard distortion datasets (clean photos degraded with controlled blur, noise, and compression), top methods agree with human rankings so closely that the datasets can no longer separate them. Step outside, into real-world degradations, generated images, rendered frames, and the agreement drops fast. The problem looks solved on the data the field has studied for twenty years, and wide open on the content the new world actually produces.

The improvement loops arrived. Modern visual systems learn by critiquing their own outputs: generators train against reward model scores (ImageReward made this standard practice in 2023), upscalers and restoration models optimize perceptual losses, and agents retry until a judge approves the output. Inside these loops, the judge is not a reporting tool. It is the teacher. A critic that cannot see a visual artifact teaches the model to keep producing it, and the system inherits every blind spot its judge has.

The judge sets the ceiling.

Machine-made images went from curiosity to infrastructure in under a decade. In a few years, every serious visual pipeline will have a learned judge in the loop, and the standards for those judges are being written now.

Next week, Part 2: why an upscaled frame can genuinely look better than its reference, and the 2018 theorem that explains how.

What do your eyes catch that your metrics miss? I am collecting war stories for this series, and the best ones will show up in later parts, credited. Leave a comment below or find me on LinkedIn.

References

1. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, April 2004.
2. J. Xu, X. Liu, Y. Wu, Y. Tong, Q. Li, M. Ding, J. Tang, and Y. Dong, "ImageReward: Learning and evaluating human preferences for text-to-image generation," NeurIPS 2023.
3. SQ Magazine, "AI Image Generation Statistics 2026: Market Size, Adoption & Risks," June 2026.

This is Part 1 of the Visual Quality Intelligence series. Follow along on LinkedIn or subscribe here for the full bibliography and extended notes with each part.