Microsoft Research Study Shows AI Models Still Struggle with Software Debugging

Summary

A study from Microsoft Research found that leading AI models, including Anthropic's Claude 3.7 Sonnet and OpenAI's o3-mini, fail to debug many issues in the SWE-bench Lite software development benchmark. The findings highlight a significant gap between AI-assisted code generation — where adoption is accelerating, with Google reporting 25% of new code generated by AI and Meta pursuing broad deployment of AI coding models — and the more complex task of identifying and fixing bugs in existing codeb