Internal Agents Map

CodePal

Company
Snap
Approach type
Agent
Work
Code review
Human involvement
Drafts reviewed
Invocation
Event-driven
Interfaces
Github
Deployment stage
Scaled
Evidence strength
Detailed primary
Entry reviewed

Purpose

Snap built CodePal, an internal AI code reviewer that comments on pull requests in its GitHub Enterprise instance before a human reviewer reads them. It builds symbolic context for the diff without cloning the repository, runs a multi-pass review loop with a verifier that audits every finding, and posts bug findings, a semantic diff summary, and a generated pull request description. Snap reports that CodePal reviews 90% of all its pull requests today, and that every pull request still requires a final engineering approval.

Check the reviewed sources and claim details.

How it works

Representative workflow: Pull request diff through symbolic context build, the multi-pass review loop, and verified findings posted for the author and the human reviewer. Research details

Build symbolic context with a two-pass file picker

The first pass parses the repository with tree-sitter to build a symbol-to-file index; the second pass extracts the symbols the diff references, scores files by symbol overlap, and selects the top N files within the token budget

Share one context build across three child workflows

A parent workflow runs the symbol indexing and file selection once and writes the result to a shared store, which the code review, summary generation, and description generation child workflows all read

Run two bootstrap passes in parallel

Two passes start together on the same model with different sampling parameters, and comparing their findings shows which findings the model actually believes

Launch a speculative third pass behind an agreement gate

A third pass starts in a cancellable context as soon as one bootstrap pass finishes; the supervisor discards its work when the two bootstrap passes agree and counts it when they disagree

Hand off to the next pass as soon as a finding is new

From the third pass onward, a pass that surfaces a finding the supervisor has not seen launches the next pass without waiting for the current one to finish, and a pass with no net-new findings gets no successor

Detect bugs that compilation and tests miss

CodePal looks for logic errors, null pointer risks, race conditions, resource leaks, error handling gaps, type mismatches, edge cases, and state management problems, and Snap expanded the detection scope from 8 bug categories to 12

Track dependencies across repositories

For a growing share of reviews CodePal queries Code Search to identify the downstream callers a function signature change would break, including callers that live in a different repository than the pull request touches

Re-review each new commit incrementally

Each new commit triggers a focused re-review, with auto-resolution of findings whose files have left the diff

Where people stay involved

Each scope pairs its normal attention boundary with supporting evidence. See the supervision definitions for the level mapping and limits.

Reported: Findings are posted for the author and the human reviewer to judge and vote on, and every pull request still requires a final engineering approval.

  • pull request diff → posted review findings, semantic summary, and generated description that the author and the human reviewer act on

    Work-product review · Level 3

Catalog interpretation: Level 3 for pull request diff → posted review findings, semantic summary, and generated description that the author and the human reviewer act on; human attention boundary: work-product-review.

Observed in June 2026

Implementation details

Model
Not reportedThe article names no review model; it says only that the two bootstrap passes share one model with different sampling parameters and that Snap keeps testing which model balances quality, cost, and speed.
Harness
A parent workflow builds the review context once and writes it to a shared store that three child workflows for code review, summary generation, and description generation all read; the review loop runs concurrent model passes under a supervisor with an agreement gate and an eager hand-off, alongside a separate long-running verifier conversation
Sandbox
Not reportedThe article documents in-memory, no-clone repository access through the GitHub Enterprise API but no execution or isolation boundary; the unknown claim stays in the research details.
Tool access
Reads git tree diffs and only the required source blobs through the GitHub Enterprise API without cloning, and for a growing share of reviews queries Code Search, Snap's internal semantic search over the full codebase
Knowledge
Repository-level customization through a .codepal.yaml file and per-path instructions, and repository-specific review checks on top of the shared bug categories
Context management
A two-pass file picker scores repository files by symbol overlap with the diff and selects the top N within a token budget, so a typical review reads a few hundred KB of source whatever the repository size; reviews are also chunked into logical parts to avoid overwhelming the model
Credentials
Not reportedThe article states that every review runs against the GitHub Enterprise API but does not document how CodePal authenticates to it or to Code Search.
Interfaces
github

Mechanisms

Diff-to-review pipeline

Pull request diff → symbolic context build → multi-pass review loop → verified findings, a semantic diff summary, and a generated description posted on the pull request

Read the code without a clone

Every review runs in memory against the GitHub Enterprise API, using git tree diffs to identify what changed and fetching only the needed blobs, with no working copy written to disk and no long-lived repository mirror

Validation and failure handling

Verify each finding before it is posted

The Verifier, a separate long-running model conversation, consumes findings as they merge and audits each one against the supplied context, for example checking that every symbol a finding cites is present in that context

Turn engineer reactions into ground truth

The Finding Lifecycle records a thumbs up or thumbs down on each CodePal comment, together with findings that authors fix and findings merged without being addressed, and aggregates them into the ground truth dataset

AB test every change against the evaluation framework

An evaluation framework built on the ground truth dataset AB tests new CodePal changes, targeting higher true-positive recall and fewer false positives, with speed and cost kept as guardrails

Reported observations

The catalog records what the sources report, with the scope and the denominator of every figure. A qualification below limits the figure it sits under.

Adoption output · Reported measurement · CodePal review volume and pull request coverage over the last 4 months before publication, with the confirmed issues corrected before human review

Headline claim

“More than 200,000 reviews across 90% of all pull requests over the last 4 months, catching thousands of confirmed issues that were corrected before human review and before reaching production”

Reported by
Snap
Scope
CodePal reviews over the last 4 months before publication; the source dates the window relatively and gives no calendar range
Denominator
90% of all pull requests at Snap

Observed in June 2026

Adoption output · Reported measurement · Growth of CodePal pull request coverage from none to over 90% within a single unnamed quarter

Key observation

Adoption went from 0% to 90% of pull requests within a single quarter

Reported by
Snap
Scope
Share of pull requests receiving a CodePal review, from virtually no AI-reviewed pull requests to over 90%, within a single unnamed quarter
Denominator
All pull requests at Snap

Observed in June 2026

Adoption output · Reported measurement · The opt-in phase of the rollout, covering the 9% starting share, 300 voluntary repositories, and sentiment during that phase

Key observation

CodePal started as an opt-in experiment on 9% of pull requests and reached voluntary use across 300 repositories with more than 70% positive sentiment before teams were auto opted in

Reported by
Snap
Scope
The opt-in phase of the CodePal rollout, before teams were auto opted in
Denominator
Pull requests at Snap for the 9% figure; repositories for the 300-repository figure

Observed in June 2026

Effectiveness · Reported measurement · True-positive recall measured against the ground truth dataset

Key observation

The recall rate of true positives climbed from 30% to 80%

Reported by
Snap
Scope
Recall of true positives, reported as climbing during the same quarter in which adoption reached 90%; the source gives no calendar dates
Method
An evaluation framework with a ground truth dataset formed from real engineer feedback, used to AB test CodePal changes

The source does not report the denominator of this figure.

Observed in June 2026

Effectiveness · Reported measurement · False positive rate on the held-out golden dataset, explicitly not on live traffic

Key observation

The false positive rate on the golden dataset dropped to 0%, measured on the held-out golden dataset and not on live traffic

Reported by
Snap
Scope
False positive rate on the held-out golden dataset, explicitly not on live traffic
Method
Measurement against the held-out golden dataset in the evaluation framework

The source does not report the denominator of this figure.

Observed in June 2026

Effectiveness · Reported measurement · Relative increase in positively rated bug findings against an unnamed earlier baseline

Key observation

CodePal finds 75% more bugs with a positive rating than it did before the recall work

Reported by
Snap
Scope
Bugs found with a positive rating, compared with an unnamed earlier period

The source does not report the denominator of this figure.

Observed in June 2026

Effectiveness · Reported measurement · Engineer sentiment on CodePal bug findings

Key observation

Engineer sentiment on bug findings reached 80% positive

Reported by
Snap
Scope
Engineer sentiment on CodePal bug findings

The source does not report the denominator of this figure.

Observed in June 2026

Cost latency · Reported measurement · CodePal review completion time against the median wait for a first human review

Key observation

CodePal reviews complete within 10 minutes, while the median wait for a first human review is about 5 hours

Reported by
Snap
Scope
CodePal review completion time against the median wait for the first human review on a Snap pull request

The source does not report the denominator of this figure.

Observed in June 2026

Cost latency · Reported measurement · Average cost of one CodePal review

Key observation

Reviews cost on average about $0.40 each

Reported by
Snap
Scope
Average cost of one CodePal review

The source does not report the denominator of this figure.

Observed in June 2026

Effectiveness · Qualitative · Severity split of the CodePal findings that engineers accepted with a +1 vote

Key observation

The majority of accepted bugs, meaning findings that received a +1 vote in the pull request review, rank as Critical or High severity

Reported by
Snap
Scope
Severity of CodePal findings that engineers accepted with a +1 vote in the pull request review
Denominator
Accepted CodePal bug findings

Observed in June 2026

Adoption output · Reported measurement · Snap's company-wide merged pull request rate year-to-date, reported as context for building CodePal rather than as a CodePal result

Key observation

Snap's merged pull request rate is up 60% year-to-date, which the article attributes to daily engineer use of AI coding tools rather than to CodePal

Reported by
Snap
Scope
Snap's merged pull request rate year-to-date, reported as context for building CodePal rather than as a CodePal result

The source does not report the denominator of this figure.

Observed in June 2026

Lessons

Reported opinion: Snap concluded that for its own missed bugs the context supplied to the model mattered more than picking a top-tier model, and it chunks each review into logical parts so the model is not overwhelmed.

CodePal produces high-quality reviews with zero configuration in most Snap repositories, but Snap reports that its largest and most complex repositories generate noise until teams invest in .codepal.yaml configuration and per-path instructions.

Snap keeps a human approval gate: CodePal reviews code written by humans and AI alike, and every pull request still requires a final engineering approval.

Snap built CodePal in-house after evaluating vendor tools, citing integration depth with its internal systems and speed; a working end-to-end demo shipped in two weeks, before the procurement cycle had finished.

Reported opinion: Snap reports that voting on every CodePal comment creates a flywheel, because the recorded feedback influences future reviews and the engineers who engage most shape what CodePal surfaces for them.

Sources and research details

Citations link to the original publisher. Each source also keeps a preserved copy in the repository, so a changed or removed page stays checkable.

  1. CodePal: How Snap Built an AI Code Reviewer for the Age of AI-Written Codehttps://eng.snap.com/codepalEngineering blog · First party · Last source verification: 2026-09-21
    Preserved copy
Question coverage and scope
purpose
Reported
workflow
Reported
human involvement
Reported: Findings are posted for the author and the human reviewer to judge and vote on, and every pull request still requires a final engineering approval.
implementation
Reported
validation
Reported
observations
Reported
lessons
Reported
Research details for every claim on this page
  1. Summary
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    High
    Confidence reason
    A linked first-party source states the claim.
  2. Headline claim
    Statement type
    Metric
    Provenance
    Reported
    Confidence
    Medium
    Confidence reason
    Snap reports the review count and the coverage share in its own engineering blog; the thousands of confirmed issues carry no counting rule and none of the figures are independently reviewed.
    Reported by
    Snap
    Scope
    CodePal reviews over the last 4 months before publication; the source dates the window relatively and gives no calendar range
    Denominator
    90% of all pull requests at Snap
    Method
    Not reported
    Observation date
    2026-06
  3. Sandbox

    unknown

    Statement type
    Inference
    Provenance
    Catalog judgment
    Confidence
    Medium
    Confidence reason
    The article documents in-memory, no-clone repository access but no execution or isolation boundary; unknown does not mean absent.
  4. Harness
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    High
    Confidence reason
    A linked first-party source states the claim.
  5. Interfaces
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    High
    Confidence reason
    A linked first-party source states the claim.
  6. Tool access
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    High
    Confidence reason
    A linked first-party source states the claim.
  7. Knowledge
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    High
    Confidence reason
    A linked first-party source states the claim.
  8. Context management
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    High
    Confidence reason
    A linked first-party source states the claim.
  9. Diff-to-review pipeline
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    High
    Confidence reason
    A linked first-party source states the claim.
  10. Read the code without a clone
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    High
    Confidence reason
    A linked first-party source states the claim.
  11. Build symbolic context with a two-pass file picker
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    High
    Confidence reason
    A linked first-party source states the claim.
  12. Share one context build across three child workflows
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    High
    Confidence reason
    A linked first-party source states the claim.
  13. Run two bootstrap passes in parallel
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    High
    Confidence reason
    A linked first-party source states the claim.
  14. Launch a speculative third pass behind an agreement gate
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    High
    Confidence reason
    A linked first-party source states the claim.
  15. Hand off to the next pass as soon as a finding is new
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    High
    Confidence reason
    A linked first-party source states the claim.
  16. Detect bugs that compilation and tests miss
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    High
    Confidence reason
    A linked first-party source states the claim.
  17. Track dependencies across repositories
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    High
    Confidence reason
    A linked first-party source states the claim.
  18. Re-review each new commit incrementally
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    High
    Confidence reason
    A linked first-party source states the claim.
  19. Verify each finding before it is posted
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    High
    Confidence reason
    A linked first-party source states the claim.
  20. Turn engineer reactions into ground truth
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    High
    Confidence reason
    A linked first-party source states the claim.
  21. AB test every change against the evaluation framework
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    High
    Confidence reason
    A linked first-party source states the claim.
  22. Key observation
    Statement type
    Metric
    Provenance
    Reported
    Confidence
    Medium
    Confidence reason
    Snap states the start and end points of the ramp in its own blog; the quarter is named only as a single quarter and no rollout dates are given.
    Reported by
    Snap
    Scope
    Share of pull requests receiving a CodePal review, from virtually no AI-reviewed pull requests to over 90%, within a single unnamed quarter
    Denominator
    All pull requests at Snap
    Method
    Not reported
    Observation date
    2026-06
  23. Key observation
    Statement type
    Metric
    Provenance
    Reported
    Confidence
    Medium
    Confidence reason
    The adoption narrative gives the 9% starting point, the 300-repository figure, and the sentiment share, but does not date the stages or say how sentiment was collected at that point.
    Reported by
    Snap
    Scope
    The opt-in phase of the CodePal rollout, before teams were auto opted in
    Denominator
    Pull requests at Snap for the 9% figure; repositories for the 300-repository figure
    Method
    Not reported
    Observation date
    2026-06
  24. Key observation
    Statement type
    Metric
    Provenance
    Reported
    Confidence
    Medium
    Confidence reason
    Snap reports the recall change twice and names the evaluation framework behind it, but publishes neither the dataset size nor its definition of a recallable bug.
    Reported by
    Snap
    Scope
    Recall of true positives, reported as climbing during the same quarter in which adoption reached 90%; the source gives no calendar dates
    Denominator
    Not reported
    Method
    An evaluation framework with a ground truth dataset formed from real engineer feedback, used to AB test CodePal changes
    Observation date
    2026-06
  25. Key observation
    Statement type
    Metric
    Provenance
    Reported
    Confidence
    Medium
    Confidence reason
    Snap states the 0% rate and volunteers the limit that it was measured on a held-out golden dataset rather than on live traffic, which bounds what the number shows.
    Reported by
    Snap
    Scope
    False positive rate on the held-out golden dataset, explicitly not on live traffic
    Denominator
    Not reported
    Method
    Measurement against the held-out golden dataset in the evaluation framework
    Observation date
    2026-06
  26. Key observation
    Statement type
    Metric
    Provenance
    Reported
    Confidence
    Low
    Confidence reason
    Snap reports the relative increase without naming the baseline period, the absolute counts, or whether review volume grew over the same span.
    Reported by
    Snap
    Scope
    Bugs found with a positive rating, compared with an unnamed earlier period
    Denominator
    Not reported
    Method
    Not reported
    Observation date
    2026-06
  27. Key observation
    Statement type
    Metric
    Provenance
    Reported
    Confidence
    Medium
    Confidence reason
    The 80% sentiment figure appears in both the adoption section and the closing numbers; the source does not state how the sentiment share is collected or over how many findings.
    Reported by
    Snap
    Scope
    Engineer sentiment on CodePal bug findings
    Denominator
    Not reported
    Method
    Not reported
    Observation date
    2026-06
  28. Key observation
    Statement type
    Metric
    Provenance
    Reported
    Confidence
    Medium
    Confidence reason
    Snap gives a completion bound for CodePal and a median for the human comparison; the CodePal figure is stated as within 10 minutes rather than as a median, and neither figure is dated.
    Reported by
    Snap
    Scope
    CodePal review completion time against the median wait for the first human review on a Snap pull request
    Denominator
    Not reported
    Method
    Not reported
    Observation date
    2026-06
  29. Key observation
    Statement type
    Metric
    Provenance
    Reported
    Confidence
    Medium
    Confidence reason
    Snap reports an average unit cost without a measurement window or a statement of what the cost includes.
    Reported by
    Snap
    Scope
    Average cost of one CodePal review
    Denominator
    Not reported
    Method
    Not reported
    Observation date
    2026-06
  30. Key observation
    Statement type
    Metric
    Provenance
    Reported
    Confidence
    Low
    Confidence reason
    Snap describes the severity split as a majority and defines an accepted bug as one with a +1 vote, but gives no share, no count, and no severity-rating method.
    Reported by
    Snap
    Scope
    Severity of CodePal findings that engineers accepted with a +1 vote in the pull request review
    Denominator
    Accepted CodePal bug findings
    Method
    Not reported
    Observation date
    2026-06
  31. Key observation
    Statement type
    Metric
    Provenance
    Reported
    Confidence
    Medium
    Confidence reason
    Snap reports this as a company-wide velocity figure behind the review bottleneck that prompted CodePal; the article attributes it to AI coding tool use, not to CodePal, and gives no method.
    Reported by
    Snap
    Scope
    Snap's merged pull request rate year-to-date, reported as context for building CodePal rather than as a CodePal result
    Denominator
    Not reported
    Method
    Not reported
    Observation date
    2026-06
  32. Lesson
    Statement type
    Opinion
    Provenance
    Reported
    Confidence
    Medium
    Confidence reason
    The lessons section states this as Snap's own conclusion from the misses reported to the team; no counts of missed bugs or model comparisons are published to support it.
  33. Lesson
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    Medium
    Confidence reason
    Snap reports both the zero-configuration result for most repositories and the noise its biggest repositories produce without per-path instructions; the claim covers Snap's repositories only.
  34. Lesson
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    High
    Confidence reason
    The closing section states directly that CodePal reviews human-written and AI-written code and that every pull request still requires a final engineering approval.
  35. Lesson
    Statement type
    Fact
    Provenance
    Reported
    Confidence
    Medium
    Confidence reason
    Snap names both reasons it rejected vendor tools and dates the demo at two weeks; the procurement comparison is Snap's own account of its evaluation.
  36. Lesson
    Statement type
    Opinion
    Provenance
    Reported
    Confidence
    Medium
    Confidence reason
    Snap describes the voting mechanism as fact and the flywheel as its own reading of it; the article shows no measurement that engagement improved later reviews for the engineers who voted.
  37. Operating model assessment
    Statement type
    Inference
    Provenance
    Catalog judgment
    Confidence
    High
    Confidence reason
    The source states that engineers judge each posted finding, that their reactions are recorded as ground truth, and that every pull request still requires a final engineering approval, so attention returns on the review output rather than on an outcome or an exception.
    Observation date
    2026-06
Back to agents
Know an internal agent?

Share a resource or public mention, suggest an addition, or correct an existing entry. Pull requests are also welcome.

Suggest a resource or update