Code Review Agents
Code Review Agents is the reference implementation of "A Multi-Agent Framework for Evidence-Grounded Automated Code Review", which I published in the ICENSTED 2026 proceedings. The obvious design hands the whole diff to one reviewer and takes back whatever it says. I split the work across seven reviewing agents and gave each of them a single concern.
the agents
The Change & Context Analyst reads the diff against what the pull request claims to be for. A Security Reviewer works from Semgrep and Bandit output, and a Style & Format Reviewer consolidates Ruff and ESLint. Logic, performance, documentation and tests get a reviewer each. A Revision Proposer drafts the change itself. A Supervisor goes over everything that comes back. It merges duplicates, settles disagreements between agents and sets severity, and it cuts noise before anything is shown.
The paper described five agents and the implementation ended up with seven. Splitting them up costs more than one long prompt would. It also means an agent that only looks at security has nothing to trade a vulnerability against.
the evidence rule
Nothing reaches the review unless it comes with static-analysis output, or a file and line with the code attached. An agent that has neither drops the finding.
The rule keeps the framework from calling a function slow because the function looks like a slow one. It also cuts the vulnerability classes that nothing in the diff supports. Git, Ruff, ESLint, Semgrep, Bandit and Coverage.py supply the evidence. Ten languages are wired up. Where a fix was small enough to write out, the agent returns a patch for it.
how it is measured
The evaluation set is real pull requests taken from public repositories and sorted into bugfix, feature and refactor. I wrote the ground truth for each one by hand. Precision, recall, F1 and review efficiency are measured against it, and the harness runs statistical tests and exports the numbers as LaTeX.
Every agent's prompt is versioned in the repository. When the output changes I can go back and find the edit to the instruction that did it. Reviews of real pull requests are checked in as well, including ones on FastAPI.
- Python
- CrewAI
- Semgrep
- Bandit
- Ruff
- ESLint
- Coverage.py