news
307

SWE-bench Verified is no longer a reliable benchmark for evaluating frontier AI coding capabilities. Recent analysis reveals that the dataset is increasingly contaminated by flawed tests and training data leakage. Consequently, researchers are now recommending SWE-bench Pro as a more accurate alternative for measuring progress.
Comments
0Discuss on Hacker NewsOnly full members can comment
Log in to express your opinion.
No comments yet.