Deflaker

Deflaker investigates and repairs flaky tests for Dropbox engineers using Nova.

Company: Dropbox
Approach type: Agent
Work: Coding
Human involvement: Unknown
Invocation: Event-driven
Deployment stage: Deployed
Evidence strength: Detailed primary
Entry reviewed: 2026-09-17

Copy as MD Ask ChatGPT

How it works

Athena flaky-test detection → CI-validated fix or capped attempts.

Collect evidence

Athena identifies flakiness; Deflaker gathers passing and failing logs.^{[Source 1]}

Propose a repair

Nova diagnoses the likely cause and proposes a change.^{[Source 1]}

Retry or finish

Failed validation starts another attempt with new logs and prior notes; the workflow lands a fix or stops after five attempts.^{[Source 1]}

Where people stay involved

People review, approve, or guide the agent’s work here.

Not reported: The source describes an automated fix-and-validate loop and publication outside the agent, but does not state the required human approval or exception-handling policy.

Supervision level
Not classified
- Athena flaky-test detection → CI-validated fix or capped attempts^{[Source 1]}

How supervision levels work

Implementation details

Model: Not reported
Harness: Durable remediation workflow calling Nova; publication and CI remain outside the coding agent.^{[Source 1]}
Sandbox: Not reported
Tool access: Not reported
Knowledge: Not reported
Context management: Prior attempt notes and new test logs accompany retries.^{[Source 1]}
Credentials: Not reported
Interfaces: Not reported

Validation and failure handling

Validate repeatedly

CI repeats the test at least 100 times, depending on its failure rate.^{[Source 1]}

Reported observations

Not reported

Lessons

Not reported: The available sources do not report lessons from this work.

Back to agents

Improve this record

Give your coding agent the context and instructions to prepare a source-backed update.

Suggest on GitHub

Paste the prompt into your coding agent, then describe the correction or add a public source.

Preview prompt

Help me update the Internal Agents Map record for Deflaker at Dropbox.

Repository: https://github.com/steel-experiments/internal-agents-map
Record: data/agents/dropbox-deflaker.yaml
Published record: https://internal-agents.com/agents/dropbox-deflaker
Readable context: https://internal-agents.com/agents/dropbox-deflaker.md

Use the correction or public source I provide in this conversation. If I have not supplied either, ask what I want changed before editing.

Use an existing checkout of this repository, or clone it if needed. Read the current record from the repository. Follow AGENTS.md, CONTRIBUTING.md, and the relevant instructions in data/schema.md.

Verify that the evidence supports the requested change. Preserve uncertainty and distinguish reported facts from catalog judgments. Follow the repository process for capturing new sources. If evidence is insufficient, explain the gap instead of inventing details.

Make a focused change on a branch. Regenerate derived files and run the required checks. Do not edit generated content by hand.

Show me the change, supporting evidence, and check results, including any checks you could not run. Ask whether to submit a pull request unless I already requested one. Do not merge the pull request. If you cannot edit the repository, prepare a source-backed issue draft.

Sources and research details

Citations link to the original publisher. Each source also keeps a preserved copy in the repository, so a changed or removed page stays checkable.

Introducing Nova, our internal platform for coding agentshttps://dropbox.tech/machine-learning/introducing-nova-our-internal-platform-for-coding-agentsEngineering blog · First party · Last source verification: 2026-08-31
Preserved copy

Question coverage and scope

purpose: Reported
workflow: Reported
human involvement: Not reported: The source describes an automated fix-and-validate loop and publication outside the agent, but does not state the required human approval or exception-handling policy.
implementation: Reported
validation: Reported
observations: Not reported
lessons: Not reported

Research details for every claim on this page

Summary
Statement type
Fact
Provenance
Reported
Confidence
Not assessed
Confidence reason
No explicit confidence assessment is recorded for this claim.
- SupportsIntroducing Nova, our internal platform for coding agentsHow we’re using the platform: Flaky test remediation; What we learned
Harness
Statement type
Fact
Provenance
Reported
Confidence
Not assessed
Confidence reason
No explicit confidence assessment is recorded for this claim.
- SupportsIntroducing Nova, our internal platform for coding agentsHow we’re using the platform: Flaky test remediation; What we learned
Context management
Statement type
Fact
Provenance
Reported
Confidence
Not assessed
Confidence reason
No explicit confidence assessment is recorded for this claim.
- SupportsIntroducing Nova, our internal platform for coding agentsHow we’re using the platform: Flaky test remediation; What we learned
Collect evidence
Statement type
Fact
Provenance
Reported
Confidence
Not assessed
Confidence reason
No explicit confidence assessment is recorded for this claim.
- SupportsIntroducing Nova, our internal platform for coding agentsHow we’re using the platform: Flaky test remediation; What we learned
Propose a repair
Statement type
Fact
Provenance
Reported
Confidence
Not assessed
Confidence reason
No explicit confidence assessment is recorded for this claim.
- SupportsIntroducing Nova, our internal platform for coding agentsHow we’re using the platform: Flaky test remediation; What we learned
Validate repeatedly
Statement type
Fact
Provenance
Reported
Confidence
Not assessed
Confidence reason
No explicit confidence assessment is recorded for this claim.
- SupportsIntroducing Nova, our internal platform for coding agentsFlaky test remediation: CI runs the proposed change 100 or more times depending on failure rate
Retry or finish
Statement type
Fact
Provenance
Reported
Confidence
Not assessed
Confidence reason
No explicit confidence assessment is recorded for this claim.
- SupportsIntroducing Nova, our internal platform for coding agentsHow we’re using the platform: Flaky test remediation; What we learned
Operating model assessment
Statement type
Inference
Provenance
Catalog judgment
Confidence
Unverified
Confidence reason
The source describes an automated fix-and-validate loop and publication outside the agent, but does not state the required human approval or exception-handling policy.
Observation date
2026
- SupportsIntroducing Nova, our internal platform for coding agentsFlaky test remediation: workflow lands a working fix or reaches five attempts; diagram describes the automatic retry loop