Large Language Models for Machine Translation Quality Annotation: Humans and Models Are Both Challenged

Large Language Models are tested as a cost‑effective alternative to human judges for evaluating machine translation quality. The study compares LLM agreement with humans on two schemes, Multidimensional Quality Metrics and Error Span Annotation, across many language pairs and domains. Results show both humans and models face challenges, highlighting limits of LLMs in nuanced MT assessment.

Opening Kapyn…