news
307

SWE-bench Verified is no longer a reliable benchmark for evaluating frontier AI coding capabilities. Recent analysis reveals that the dataset is increasingly contaminated by flawed tests and training data leakage. Consequently, researchers are now recommending SWE-bench Pro as a more accurate alternative for measuring progress.

Only full members can comment

Log in to express your opinion.

No comments yet.