Can we trust LLMs to judge whether a research idea is novel?

Researchers evaluate whether large language models can reliably judge the novelty of research ideas. The study tests several LLMs on a benchmark of academic proposals, finding mixed accuracy and highlighting biases toward familiar topics. The results suggest caution for using LLMs as sole novelty reviewers in academia.

Opening Kapyn…