CodeRabbit analyzed pull requests and found that code written by AI agents had about 40% more critical bugs after merging than human-written code. The bugs often went unnoticed during code review.
Chapters
But an analysis that CodeRabbit itself did was looking at those PRs and looking at the signals on which PRs were opened by code review tools, which had commits that were made by coding agents versus commits in open source PRs that did not show any signs of any coding agent. And we analyzed those human versus AI written code and we found the code written by AI coding agents had 40% more critical bugs after the PRs were merged, or, you know, while they were still being— people were still commenting on it, than PRs that didn't show sign of AI coding agent activity. So that really got us thinking into what's going on. Why is there— why are there more bugs in these PRs?