OpenAI analysis reveals flaws in the SWE-Bench Pro coding evaluation benchmark. The findings suggest reliability and accuracy issues, impacting how AI models are assessed for coding tasks. This raises questions about the validity of current benchmark results and the need for more robust evaluation methods.
Opening Kapyn…