Roast
Roast is the workflow orchestration framework Shopify's Augmented Engineering Developer Experience team extracted from its internal AI tools and open-sourced; Shopify runs Roast workflows internally to interleave AI steps with deterministic code.
Capabilities and architecture
- Model
- An OpenAI API key is a prerequisite, or OpenRouter for other models; the Raix library abstracts AI providers and adds retry, caching and structured output; the built-in CodingAgent tool is powered by Claude Code.[Source 1]The post names the API-key prerequisite, the Raix abstraction and Claude Code behind the CodingAgent; it names no model for a plain AI step.Observation date: 2025-06
- Harness
- As released in 2025: a Ruby command-line framework that reads a workflow.yml file and matching markdown prompt files, interprets each step from its structure, and can be used with any programming language.[Source 1]The claim describes the 2025 release; the repository captured in 2026 documents a Ruby DSL of cogs instead.Observation date: 2025-06
- Sandbox
- Not reportedThe reviewed sources document WriteFile security restrictions and configurable Cmd restrictions, not an execution or isolation boundary.
- Tool access
- Built-in ReadFile, WriteFile, UpdateFiles, Grep, SearchFile, Cmd, Bash and CodingAgent tools; WriteFile carries security restrictions and Cmd carries configurable restrictions.[Source 1]
- Knowledge
- Markdown prompt files use ERB templating to read the workflow context; complex logic goes in custom Ruby step classes that inherit from BaseStep.[Source 1]
- Context management
- Steps in a workflow share their conversation transcript, so a later step references earlier discoveries without extra configuration; every execution is saved so a run can resume from any step.[Source 1]
- Credentials
- An OpenAI or OpenRouter API key is required; Raix initializers in the .roast/initializers directory can add custom authentication schemes.[Source 1]The post names the provider API key and custom authentication schemes through Raix initializers; it names no employee sign-in or secret store.
- Interfaces
The post names a command line tool distributed as a Ruby gem; the Slack channels the SRE workflow reads are a data source, not an invocation interface.
- cli
Mechanisms
A workflow.yml file and matching prompt.md files define the steps, and Roast interprets each step from its structure.[Source 1]
Directory prompts, shell commands wrapped in $(), inline prompts, custom Ruby step classes and nested-array parallel steps run in one workflow.[Source 1]
Workflows iterate over collections, execute steps conditionally and branch with case statements.[Source 1]
ReadFile, WriteFile, UpdateFiles, Grep, SearchFile, Cmd and Bash are supplied out of the box.[Source 1]
A Claude Code integration runs as a step and iterates within the guardrails and objectives the workflow author sets.[Source 1]
Steps share their conversation transcript so later steps build on the context earlier steps produced.[Source 1]
Roast saves every workflow execution so development resumes from any step instead of rerunning expensive AI operations.[Source 1]
Documented uses
Boba workflow: untyped Ruby test file → deterministic cleanup and Sorbet autocorrect → coding-agent iteration until tests and type checking pass.
Deterministic sed steps clean up the code before any AI step runs.[Source 1]
The workflow raises the test file to strict Sorbet typing.[Source 1]
Sorbet's autocorrect resolves the type errors it can fix on its own.[Source 1]
The CodingAgent iteratively fixes the type errors the deterministic steps left behind.[Source 1]
Access and controls
See documented credential and access boundaries in the architecture above. Scoped human-review assessments for individual uses remain in research details.
Reliability and validation
The workflow runs the tests and ensures that both the tests and type checking pass.[Source 1]
Adoption and operating evidence
These results come from the sources. Notes below each result explain what it covers and any reported limits.
Shopify engineers analyzed thousands of test files with Roast, identifying and fixing common antipatterns and increasing test coverage; the post reports no count, period or baseline.[Source 1]
- Reported by
- Shopify
- Scope
- Test files analyzed with Roast workflows inside Shopify
The source does not report the denominator of this figure.
Observed in June 2025
About this result
- Category
- Implementation scale
- Evidence type
- Estimate
- What it covers
- Test files analyzed with Roast workflows inside Shopify
A dozen engineers had contributed features and workflows to the public Roast repository by publication.[Source 1]
- Reported by
- Shopify
- Scope
- Engineers who had contributed features and workflows to the public Roast repository as of publication
The source does not report the denominator of this figure.
Observed in June 2025
About this result
- Category
- Adoption output
- Evidence type
- Estimate
- What it covers
- Engineers who had contributed features and workflows to the public Roast repository
Lessons
Reported opinion: Shopify says AI agents need help staying on track and work much better when a complicated prompt is broken into discrete steps; letting AI roam free around millions of lines of code did not work well for its unit-test work.[Source 1]
For type annotation, Shopify runs deterministic cleanup and Sorbet autocorrect first and hands only the remaining issues to the coding agent.[Source 1]
Reported opinion: Shopify says saving every workflow execution and resuming from any step speeds up workflow development because expensive AI operations are not rerun.[Source 1]
Reported opinion: Shopify describes using an AI step as a placeholder for a part of a workflow the author does not yet understand, then replacing it with deterministic code once the problem is clearer.[Source 1]