A new framework uses multiple-choice QA to evaluate video captions for VLLMs. The method redefines caption quality as information fidelity, measuring how well a caption covers the video content. It generates fine-grained questions that test specific visual elements, overcoming the one-to-many mismatch of traditional metrics, and offers developers a more nuanced benchmark for training and tuning VLLMs.
Opening Kapyn…