PerceptionBench
An open benchmark for multimodal model researchers and evaluators that helps produce more interpretable visual-perception scores instead of a single mixed leaderboard number.
Tool overview
My take: this is best understood as a diagnostic vision benchmark, not a general multimodal leaderboard and not an image-app framework. The evidence consistently frames its value as isolating visual perception from knowledge and complex reasoning, so teams can tell whether a model failed because it did not actually see the image, struggled with OCR, missed localization, or only failed at later reasoning. A better analogy is “a unit test suite for MLLM visual perception,” not a tool for directly building end-user image products.
Its practical use is in evaluation and research workflows. Posts in the evidence say PerceptionBench derives 10 atomic visual abilities from failure cases across 42 existing benchmarks, and releases downloadable data plus evaluation code. The public set is described as roughly 3,000 verified questions, including atomized questions decomposed from prior failures and newly written items. That makes it useful for model comparison, regression testing, weakness diagnosis, and paper analysis.