Boulder — The Self-Replicating AI Build Factory on AWS

Most “AI builds my app” demos are a single script calling an LLM in a loop. Nice on stage, useless in production. The moment you want concurrency, retries, idempotency, observability, cost control, or the ability for the system to fix its own mistakes, the demo collapses.

Boulder is the answer I built to that problem. It is a meta-orchestrator that spawns dedicated AI worker instances on AWS to autonomously design, code, deploy and — critically — repair full-stack applications on Amplify Gen 2. It has reached Phase 5 and now builds its own sub-agents. This post is a walkthrough of what it is, how it works, and why every serious AI agent system eventually converges on the same small set of AWS primitives.

The Problem with Monolithic “AI Builders”

A single agent trying to plan, research, code, deploy and verify in one context window is the same anti-pattern as a single monolithic process trying to handle every workload. It does not scale, it cannot retry, it cannot observe itself, and it cannot heal.

The agent world is rediscovering what the distributed systems world learned twenty years ago: decouple responsibilities, put queues between them, make each component idempotent, and you get reliability almost for free.

Boulder is built on exactly that premise.

Architecture in One Picture

User (CLI) → boulder-forge → SQS command queue
    → ECS Fargate Service picks up message
        → Creates GitHub repo (idempotent)
        → Generates full-stack Amplify Gen 2 app via Bedrock
        → Pushes to GitHub
        → Creates Amplify app + connects repo (idempotent)
        → Monitors Amplify build
        → On failure: fetches logs, asks Bedrock to fix, retries (max 3)
        → Posts result to SQS status queue
    → boulder-forge reads status → reports back to user

Two queues. One service. One DynamoDB table. One Bedrock inference profile. One Amplify target. That is the whole system.

The Component Map

Component	Choice	Why
Command center	Local CLI (boulder)	Start local, migrate to AWS later. No premature infra.
Workers	ECS Fargate Service, desiredCount=1..5	No cold start per build, auto-scalable, no server management.
Forge ↔ Worker comms	SQS — 2 queues (commands + status) + DLQ	Full decoupling. Components are hot-swappable.
LLM	Amazon Bedrock — Claude Sonnet via inference profile	Zero API keys. Access via IAM task role. Cross-region routing.
Deploy target	AWS Amplify Gen 2	Amplify handles CI/CD. Worker just writes code and git push.
Code repos	GitHub, one repo per app	Classic PAT with repo scope in SSM. Fine-grained tokens can’t create repos.
State	DynamoDB + GSI on (status, createdAt)	Efficient boulder list --status FAILED without table scans.
Secrets	SSM Parameter Store (SecureString)	GitHub token, never in env vars, never in code.
Alerting	CloudWatch alarm → SNS boulder-alarms	Subscriptions managed at runtime, not in IaC.
IaC	CDK TypeScript	Same language as the app. One mental model.
Region	us-east-1	Bedrock model availability dictates this.

Every choice above has an ADR behind it. Nothing is cargo-culted.

Why SQS Is the Spine

If you remember one design principle, remember this one: put a queue between your orchestrator and your workers.

SQS gives you, for free:

• Back-pressure — if builds pile up, the queue grows. Nothing crashes.
• Retries — visibility timeout means a failed worker’s message becomes available again automatically.
• At-least-once delivery — which forces you to build idempotent workers, which makes them reliable.
• DLQ as a bug tracker — after maxReceiveCount, poison messages end up in the DLQ where you can study them. Set retention to 14 days, wire a CloudWatch alarm on depth, route that alarm to SNS. Done.
• Visibility heartbeat — for long builds, call ChangeMessageVisibility every 3 minutes. No premature redelivery, no absurdly long default timeouts.

A direct HTTP call from CLI to worker would give you exactly none of those properties. That is why every serious agent factory eventually adds a queue.

Why IAM Beats API Keys for LLM Access

Boulder workers never see a Bedrock API key. They do not need one. The Fargate task role has bedrock:InvokeModel on the inference profile ARN, and the SDK handles authentication via the instance metadata service.

This matters more than it sounds:

• You cannot leak a key that does not exist.
• Rotation is automatic (IAM role credentials rotate every few hours).
• Audit is automatic (CloudTrail logs every invocation with the role).
• Cost attribution is automatic (tag the role, filter Cost Explorer by tag).

Any time you see “put your API key in an env var” in an AI tutorial, ask yourself: could this be an IAM role instead? On AWS, the answer is almost always yes.

Error Handling — The Part Demos Always Skip

Phase 2 of Boulder was entirely about hardening. Three ideas did most of the work:

1. Classify errors as retryable or permanent.

class RetryableError extends Error {}   // let SQS redeliver
class PermanentError extends Error {}   // delete message, mark FAILED

Bedrock throttling, DynamoDB timeouts, transient network errors → retryable. Invalid project name, GitHub repo already exists with a conflicting owner, Amplify rejected the app → permanent. Permanent errors are deleted from the queue immediately and marked FAILED in DynamoDB. No poison message cycles endlessly.

2. Retry everything with exponential backoff and jitter.

Not just the Bedrock call. Every AWS SDK call — DynamoDB writes, SSM GetParameter, SQS send/delete, Amplify StartJob. Three retries, 1 second base, ±25% jitter. This one pattern absorbs probably 80% of real-world transient failures you will ever see.

3. Make workers idempotent.

If a message is redelivered — and at-least-once delivery guarantees it eventually will be — the worker must not break on duplicate resources. Check before you create:

• GitHub repo? List first, create if missing, reuse if it’s ours.
• Amplify app? Paginated ListApps, reuse the existing one, trigger a fresh deploy.
• DynamoDB status? Conditional writes — BUILDING only accepted from PENDING or QUEUED.

Without idempotency, at-least-once delivery becomes at-least-once corruption. With idempotency, it becomes free reliability.

Self-Healing — When the Worker Repairs Its Own Output

This is the part that made Boulder feel alive. Bedrock-generated code is fast but not perfect — compile errors, missing imports, a misnamed export. Before Phase 4, those ended the build. Now:

1. Amplify build fails.
2. Worker calls the Amplify GetJob API, pulls the build step logs.
3. Worker sends the original files + the exact error output back to Bedrock with a fix prompt.
4. Worker pushes the corrected code to GitHub.
5. Amplify auto-triggers a new build.
6. Repeat up to 3 times, then give up and mark FAILED.

In practice this turns a roughly 40% first-attempt success rate into something close to 95%. The worker is now an autonomous build-and-repair loop, not a single-shot code generator.

The same loop applies to updates: boulder update <name> -d "<changes>" fetches current code via the GitHub tree API, sends it to Bedrock with the change request, pushes modifications. Amplify deploys. If the update breaks the build, self-healing kicks in on the updated code too.

ECS Service vs Standalone Tasks — The Phase 3 Inflection Point

Phase 1 launched a Fargate task per build via RunTaskCommand. Fine for the end-to-end proof, wrong for real use. Two bad properties:

• Every build paid a cold-start tax (container pull, runtime init, SDK handshake).
• The CLI held infrastructure responsibility (knowing subnets, launching tasks).

Phase 3 switched to an ECS Fargate Service with desiredCount=1 that continuously polls SQS. The CLI now just does two things: send an SQS message and write a PENDING row to DynamoDB. That’s it.

The worker restructured from one-shot to while(!shuttingDown) loop. Each iteration: 20-second long poll, process one message, clean up, loop. SIGTERM finishes the current message then exits gracefully. Errors no longer crash the worker — they log, clean up, and the loop continues.

Tradeoff: ~$0.05/hour idle cost for the always-on task. Worth every penny. You can scale to zero later with a queue-depth-based scaling policy. In the meantime, cold starts are gone and the design is dramatically simpler.

The Flywheel — Agents Building Agents

This is where Boulder stops being a build tool and starts being something weirder.

Phase 5 added an Agent Builder. The logic:

• If a needed sub-agent does not exist, Boulder builds it.
• Each new agent extends Boulder’s own capabilities.
• The system becomes more capable with every cycle — a flywheel.

Concretely, two new DynamoDB tables (boulder-agents, boulder-agent-executions), an agent-invoke handler in the worker, and five CLI commands: agent create, list, invoke, logs, disable. Agents are first-class objects now, not hardcoded roles.

The planned next layer is a Scout Agent that analyzes Product Hunt, Hacker News and Reddit trends, surfaces monetizable app ideas, and when I validate one, Boulder builds it autonomously. The factory finds its own work. I am the approval gate; Boulder does the rest.

Say what you want about the ambition — the primitives are all there. SQS for the work, Bedrock for the reasoning, Fargate for the compute, Amplify for the delivery, DynamoDB for the memory, CDK to tie it together. Everything else is orchestration.

Lessons for Anyone Building Agent Systems on AWS

If you are designing something similar, this is what I would tell you, compressed:

• Put a queue between every logical boundary. SQS is almost free and solves more problems than any framework.
• Use IAM, not API keys, for Bedrock. Every time. No exceptions.
• Make every worker idempotent. At-least-once delivery is a feature, not a bug — but only if you earn it.
• Classify errors, don’t just catch them. Retryable versus permanent is the single most useful distinction in any distributed system.
• Retry every AWS call, not just the LLM call. Exponential backoff with jitter. Three tries. Done.
• Default VPC public subnets are fine for Phase 1. NAT gateways cost real money. Don’t pay for a production network when you’re proving a pipeline.
• ECS Service over one-shot tasks. Once the pipeline is real.
• Alarms without subscriptions are decoration. Wire your CloudWatch alarms to SNS from day one.
• Write ADRs. Every non-obvious choice. Future-you will bless past-you. An LLM reading your repo will too.

Closing Thought

Every time the AI industry proposes a new orchestration framework, the implementation turns out to be “a queue, a worker pool, a store, and a retry policy.” Boulder is the honest version of that: the framework is AWS, the primitives are the ones that have carried production workloads for a decade, and the AI is just one well-bounded component inside it.

The interesting question was never “can an AI write an app?” It was “can we build infrastructure reliable enough that an AI writing apps is the least impressive part of the system?”

With Boulder, the answer is yes.

---

Boulder source: github.com/agiusalexandre/boulder. Built with AWS CDK, ECS Fargate, Amazon Bedrock, Amplify Gen 2, DynamoDB, SQS, SSM Parameter Store — and a lot of ADRs.