Source: https://internal-agents.com/agents/shopify-roast # Shopify — Roast Roast is the workflow orchestration framework Shopify's Augmented Engineering Developer Experience team extracted from its internal AI tools and open-sourced; Shopify runs Roast workflows internally to interleave AI steps with deterministic code. - Company: [Shopify](https://internal-agents.com/organizations/shopify) - Collection: Infrastructure - Approach type: Platform - Deployment stage: Deployed - Evidence strength: Detailed primary - Status: Open sourced - First reported year: 2025 - Work: Coding, Code review, On-call, Research - Interfaces: Cli - Entry reviewed: 2026-09-21 ## Purpose ### Summary Roast is the workflow orchestration framework Shopify's Augmented Engineering Developer Experience team extracted from its internal AI tools and open-sourced; Shopify runs Roast workflows internally to interleave AI steps with deterministic code. Fact · Reported · High confidence · `shopify-roast--summary` Confidence reason: A linked first-party source states the claim. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 12, 16, 170: the DX team built and open-sourced Roast, extracted it from internal AI tools, and reports internal deployment ## Capabilities and architecture ### Harness As released in 2025: a Ruby command-line framework that reads a workflow.yml file and matching markdown prompt files, interprets each step from its structure, and can be used with any programming language. Fact · Reported · Medium confidence · `shopify-roast--architecture-harness` Confidence reason: The 2025 post describes the YAML-and-markdown design in detail; the repository captured in 2026 documents a Ruby DSL instead, so the claim is scoped to the release the post describes. Qualifications: - Observation date: 2025-06 Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 16, 26-64: convention-oriented framework, workflow.yml plus prompt files, command-line tool usable from any language - Contextualizes · [2] [Shopify/roast: Structured AI workflows made easy](https://github.com/Shopify/roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-2/content.md) · Preserved content.md, lines 12-66: the repository now documents a Ruby DSL built from cogs and run with bin/roast execute, a later redesign of the 2025 YAML configuration ### Model An OpenAI API key is a prerequisite, or OpenRouter for other models; the Raix library abstracts AI providers and adds retry, caching and structured output; the built-in CodingAgent tool is powered by Claude Code. Fact · Reported · Medium confidence · `shopify-roast--architecture-model` Confidence reason: The post names the API-key prerequisite, Raix, and Claude Code behind the CodingAgent, but never names the model a plain AI step uses; the repository captured in 2026 lists a different provider set. Qualifications: - Observation date: 2025-06 Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 50, 146-154: OpenAI API key or OpenRouter prerequisite, Raix provider abstraction, CodingAgent powered by Claude Code - Contextualizes · [2] [Shopify/roast: Structured AI workflows made easy](https://github.com/Shopify/roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-2/content.md) · Preserved content.md, lines 60-140: the repository now documents OpenAI, Anthropic, Perplexity and Gemini chat providers and Pi or Claude Code agent CLIs ### Tool access Built-in ReadFile, WriteFile, UpdateFiles, Grep, SearchFile, Cmd, Bash and CodingAgent tools; WriteFile carries security restrictions and Cmd carries configurable restrictions. Fact · Reported · High confidence · `shopify-roast--architecture-tool-access` Confidence reason: A linked first-party source states the claim. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 68-77: the built-in toolkit list with its restriction notes ### Knowledge Markdown prompt files use ERB templating to read the workflow context; complex logic goes in custom Ruby step classes that inherit from BaseStep. Fact · Reported · High confidence · `shopify-roast--architecture-knowledge` Confidence reason: A linked first-party source states the claim. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 34-40, 54-58: prompt.md files with ERB templating and BaseStep Ruby step classes ### Credentials An OpenAI or OpenRouter API key is required; Raix initializers in the .roast/initializers directory can add custom authentication schemes. Fact · Reported · High confidence · `shopify-roast--architecture-credentials` Confidence reason: A linked first-party source states the claim. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 146-150, 156-162: the API key prerequisite and Raix initializers adding custom authentication schemes ### Context management Steps in a workflow share their conversation transcript, so a later step references earlier discoveries without extra configuration; every execution is saved so a run can resume from any step. Fact · Reported · High confidence · `shopify-roast--architecture-context-mgmt` Confidence reason: A linked first-party source states the claim. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 96-98, 118-124: shared conversation transcript and saved executions that resume from any step ### Interfaces cli Fact · Reported · High confidence · `shopify-roast--architecture-interfaces` Confidence reason: A linked first-party source states the claim. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 16, 142-144: Roast is a command line tool distributed as a Ruby gem ### Declarative workflow definition A workflow.yml file and matching prompt.md files define the steps, and Roast interprets each step from its structure. Fact · Reported · High confidence · `shopify-roast--primitives-5` Confidence reason: A linked first-party source states the claim. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 24-36: convention over configuration, workflow.yml and directory-based prompt.md steps ### Mixed deterministic and AI steps Directory prompts, shell commands wrapped in $(), inline prompts, custom Ruby step classes and nested-array parallel steps run in one workflow. Fact · Reported · High confidence · `shopify-roast--primitives-6` Confidence reason: A linked first-party source states the claim. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 30-64: the five step formats Roast interprets by structure ### Advanced control flow Workflows iterate over collections, execute steps conditionally and branch with case statements. Fact · Reported · High confidence · `shopify-roast--primitives-7` Confidence reason: A linked first-party source states the claim. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 102-116: iteration over collections, conditional execution and case statements ### Built-in tool set ReadFile, WriteFile, UpdateFiles, Grep, SearchFile, Cmd and Bash are supplied out of the box. Fact · Reported · High confidence · `shopify-roast--primitives-8` Confidence reason: A linked first-party source states the claim. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 66-76: the built-in toolkit list ### CodingAgent step A Claude Code integration runs as a step and iterates within the guardrails and objectives the workflow author sets. Fact · Reported · High confidence · `shopify-roast--primitives-9` Confidence reason: A linked first-party source states the claim. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 77, 79-92: the CodingAgent integrates Claude Code and has autonomy within the guardrails the author sets ### Shared transcript Steps share their conversation transcript so later steps build on the context earlier steps produced. Fact · Reported · High confidence · `shopify-roast--primitives-10` Confidence reason: A linked first-party source states the claim. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 96-98: steps share their conversation transcript ### Session replay Roast saves every workflow execution so development resumes from any step instead of rerunning expensive AI operations. Fact · Reported · High confidence · `shopify-roast--primitives-11` Confidence reason: A linked first-party source states the claim. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 118-124: session replay saves every execution and resumes from any step - **Sandbox:** unreported — The reviewed sources document WriteFile security restrictions and configurable Cmd restrictions, not an execution or isolation boundary; the unknown claim stays in research details. ## Documented uses Documented use example: Boba workflow: untyped Ruby test file → deterministic cleanup and Sorbet autocorrect → coding-agent iteration until tests and type checking pass. ### Clean up the test file Deterministic sed steps clean up the code before any AI step runs. Fact · Reported · High confidence · `shopify-roast--primitives-0` Confidence reason: A linked first-party source states the claim. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, line 174: Boba performs cleanup with sed ### Bump the file to strict typing The workflow raises the test file to strict Sorbet typing. Fact · Reported · High confidence · `shopify-roast--primitives-1` Confidence reason: A linked first-party source states the claim. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, line 174: Boba bumps tests to strict typing ### Run Sorbet autocorrect Sorbet's autocorrect resolves the type errors it can fix on its own. Fact · Reported · High confidence · `shopify-roast--primitives-2` Confidence reason: A linked first-party source states the claim. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 94, 174: Boba runs Sorbet autocorrect ### Hand remaining issues to the coding agent The CodingAgent iteratively fixes the type errors the deterministic steps left behind. Fact · Reported · High confidence · `shopify-roast--primitives-3` Confidence reason: A linked first-party source states the claim. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 94, 174: remaining issues are fed to the coding agent, which iteratively fixes type errors ## Access and controls See the credential and access boundaries in architecture (`shopify-roast--architecture-credentials`). Scoped human-review assessments for individual uses remain in research details. ## Reliability and validation ### Confirm the file passes The workflow runs the tests and ensures that both the tests and type checking pass. Fact · Reported · High confidence · `shopify-roast--primitives-4` Confidence reason: A linked first-party source states the claim. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 94, 174: the agent runs tests and ensures both tests and type checking pass ## Adoption and operating evidence Observation: Implementation scale · Estimate · Test files analyzed with Roast workflows inside Shopify ### Key observation Shopify engineers analyzed thousands of test files with Roast, identifying and fixing common antipatterns and increasing test coverage; the post reports no count, period or baseline. Metric · Reported · Low confidence · `shopify-roast--key-metrics-0` Confidence reason: Shopify reported an approximate scale in its own engineering blog with no count, measurement window, coverage baseline or method. Qualifications: - The source does not report the denominator of this figure. - Reported by: Shopify - Scope: Test files analyzed with Roast workflows inside Shopify - Observation date: 2025-06 Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, line 172: engineers have analyzed thousands of test files Observation: Adoption output · Estimate · Engineers who had contributed features and workflows to the public Roast repository ### Key observation A dozen engineers had contributed features and workflows to the public Roast repository by publication. Metric · Reported · Low confidence · `shopify-roast--key-metrics-1` Confidence reason: Shopify reported an approximate contributor count for its public repository without a period, and without saying whether the contributors are Shopify employees or outside developers. Qualifications: - The source does not report the denominator of this figure. - Reported by: Shopify - Scope: Engineers who had contributed features and workflows to the public Roast repository as of publication - Observation date: 2025-06 Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, line 20: a dozen Engineers have already contributed features and workflows on GitHub ## Lessons ### Lesson Shopify says AI agents need help staying on track and work much better when a complicated prompt is broken into discrete steps; letting AI roam free around millions of lines of code did not work well for its unit-test work. Opinion · Reported · Medium confidence · `shopify-roast--lessons-learned-0` Confidence reason: The opening section states this as what the team learned while building AI workflows for unit-test grading; it is Shopify's account of its own experience, with no measurement behind it. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, line 10: AI agents need help staying on track; allowing AI to roam free around millions of lines of code did not work very well ### Lesson For type annotation, Shopify runs deterministic cleanup and Sorbet autocorrect first and hands only the remaining issues to the coding agent. Fact · Reported · Medium confidence · `shopify-roast--lessons-learned-1` Confidence reason: The CodingAgent section and the internal-impact section both describe this step order for Boba; the post does not compare it against an all-agent alternative, so it stays a description of Shopify's own workflow. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 94, 174: deterministic steps clean up and autocorrect, then remaining issues go to the CodingAgent ### Lesson Shopify says saving every workflow execution and resuming from any step speeds up workflow development because expensive AI operations are not rerun. Opinion · Reported · Medium confidence · `shopify-roast--lessons-learned-2` Confidence reason: The session-replay section asserts the speed-up for workflow authors; the post reports no development-time measurement. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 120-124: session replay eliminates the need to rerun expensive AI operations ### Lesson Shopify describes using an AI step as a placeholder for a part of a workflow the author does not yet understand, then replacing it with deterministic code once the problem is clearer. Opinion · Reported · Low confidence · `shopify-roast--lessons-learned-3` Confidence reason: The closing section presents this as the post author's reading of a quoted Shopify engineer; the Steel capture preserves the author's paragraph but drops the quotation it interprets, so the engineer's own wording was not reviewed from the capture. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, line 190: use AI as a placeholder to keep the workflow moving, then replace it with deterministic code ## Research details and scoped use assessments ### Sandbox unknown Inference · Catalog judgment · Medium confidence · `shopify-roast--architecture-sandbox` Confidence reason: The reviewed sources describe WriteFile security restrictions and configurable Cmd restrictions but no execution or isolation boundary; unknown does not mean absent. Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 70-77: the tool list names file and command restrictions, not an execution boundary ### Operating model assessment Unclassified for Boba workflow: untyped Ruby test file → deterministic cleanup and Sorbet autocorrect → coding-agent iteration until tests and type checking pass; human attention boundary: unknown. Inference · Catalog judgment · Unverified confidence · `shopify-roast--operating-models-0` Confidence reason: The post follows the Boba workflow to a file where tests and type checking pass and says the team pursued unit-test work with minimal human intervention, but it names no point at which a person reviews the result. Qualifications: - Observation date: 2025-06 Evidence: - Supports · [1] [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) · [Preserved copy](https://github.com/steel-experiments/internal-agents-map/blob/main/archive/sources/shopify-roast-source-1/content.md) · Preserved content.md, lines 94, 174: the Boba workflow runs deterministic cleanup and autocorrect, then the CodingAgent iterates until tests and type checking pass ## Question coverage and scope - **purpose:** Reported - **workflow:** Reported — The post breaks out one internal workflow step by step, the Boba type-annotation workflow; the other internal uses it names are described only by purpose. - **human involvement:** Not applicable — Roast is a framework, so no single review boundary covers its workflows; the scoped Boba operating-model claim stays in research details and its boundary is unknown. - **implementation:** Reported - **validation:** Reported - **observations:** Reported - **lessons:** Reported ## Sources 1. [Introducing Roast: Structured AI workflows made easy](https://shopify.engineering/introducing-roast) - Engineering blog · First party · Evidence - Original URL: - Publisher: Shopify · Published: 2025-06-18 · Accessed: 2026-09-21 · Last verified: 2026-09-21 - Preserved copy in the repository: 2. [Shopify/roast: Structured AI workflows made easy](https://github.com/Shopify/roast) - Repository · First party · Evidence - Original URL: - Publisher: Shopify · Accessed: 2026-09-21 · Last verified: 2026-09-21 - Preserved copy in the repository: