AWS + Cerebras: wafer-scale inference is coming to Bedrock

In March, AWS and Cerebras announced a collaboration that deserves more attention than a press release news cycle gives it: Cerebras CS-3 systems are being deployed inside AWS data centers, paired with Trainium servers over Elastic Fabric Adapter networking, and exposed through Amazon Bedrock. AWS is the first cloud provider to host Cerebras’s disaggregated inference solution, with leading open-source LLMs and Amazon Nova slated to run on the hardware later this year.

The headline promise — inference “an order of magnitude faster” than what’s available today, in David Brown’s words — is bold. What makes it credible is that the architecture underneath is not marketing: it’s a clean mapping of two very different computational problems onto two very different processors.

Inference is two workloads pretending to be one

Every LLM request goes through two phases with opposite characteristics:

• Prefill (processing your prompt) is embarrassingly parallel and compute-intensive, with moderate memory-bandwidth needs. Throw matrix-multiply hardware at it.
• Decode (generating the answer) is inherently serial — each token depends on the previous one — computationally light, and brutally memory-bandwidth-bound. For every single token, the hardware must stream the model’s weights through the compute units. The bottleneck isn’t FLOPS; it’s bytes per second.

Serving both phases on the same hardware means sizing for one and wasting the other. That waste got dramatically more expensive with reasoning models, which now represent the majority of inference compute and generate long “thinking” traces — thousands of decode tokens per request. Decode, already the dominant share of inference time, keeps growing.

Inference disaggregation splits the two phases across specialized hardware: Trainium — Amazon’s purpose-built AI chip, which Anthropic uses to train and deploy its models and OpenAI will consume at 2-gigawatt scale — handles prefill; the Cerebras CS-3 handles decode; EFA provides the low-latency, high-bandwidth link to hand off the KV state between them. Each processor runs the part of the problem it’s actually shaped for. The whole assembly sits on the AWS Nitro System, so it inherits the same isolation and security model as everything else in an AWS data center.

Why a wafer-sized chip wins at decode

Cerebras’s Wafer Scale Engine 3 is the answer to a question most chip designers never dared ask: what if you didn’t cut the wafer? Instead of dicing a 300mm wafer into dozens of dies, Cerebras keeps the largest square TSMC can produce as one chip: 46,225 mm², about 56× larger than the largest GPU die.

The numbers that follow are the point:

WSE-3	Spec
Transistors	4 trillion (TSMC 5nm)
AI-optimized cores	900,000
On-chip SRAM	44 GB
Memory bandwidth	21 PB/s
Peak AI compute	125 petaflops (FP16)

The spec that matters for decode is the third and fourth line. A GPU stores model weights in HBM — off-chip memory connected through an interface measured in single-digit TB/s. The WSE-3 keeps 44 GB on the silicon itself, as SRAM distributed next to the cores, and moves data across the wafer at 21 petabytes per second — roughly three orders of magnitude more bandwidth than the fastest GPU.

Recall what decode needs: stream the weights past the compute for every token, sequentially. On a GPU, that’s a memory-interface problem you can’t buy your way out of. On a wafer where the weights already live next to the cores, the serial token loop stops being bandwidth-starved. That’s how Cerebras serves models at up to 3,000 tokens per second for customers like OpenAI, Cognition, and Mistral — and why the CS-3 in the AWS solution is dedicated entirely to decode, where its architecture pays for itself on every generated token.

There’s also an efficiency story: wafer-scale integration keeps communication on-silicon instead of crossing chip boundaries, board connectors, and network switches — a fraction of the power per unit of compute compared to wiring together the equivalent GPU cluster.

Why this matters for builders

I spend my days helping enterprises put agents into production, and the constraint that keeps surfacing isn’t intelligence — it’s latency. Three patterns change materially when decode gets an order of magnitude faster:

1. Agentic coding and tool loops. An agent’s wall-clock time is dominated by sequential model calls: think, call a tool, read the result, think again. Every one of those turns pays the decode tax. Faster decode doesn’t make an agent marginally better; it compresses the entire loop, which is why Cerebras’s existing customer list is heavy on agentic coding workloads.
2. Reasoning at interactive speed. Long thinking traces are worth the tokens when the answer is better — but only if users don’t abandon the wait. Fast decode turns “reasoning model” from a batch tool into an interactive one.
3. Real-time experiences. Voice, live coding assistance, in-session document work — anything conversational lives or dies on time-to-token and tokens-per-second.

The delivery mechanism matters as much as the silicon: this lands inside Bedrock, not as a separate endpoint with separate auth, billing, and compliance reviews. If your stack already talks to Bedrock — mine does, from serverless photo agents to enterprise RAG — exotic hardware shows up as a faster model choice, not as a migration project. That’s the quiet significance of this partnership: wafer-scale inference stops being a specialty-cloud curiosity and becomes a checkbox in the same API you already use.

Announced in March as “launching in the next couple of months,” the solution hasn’t hit general availability at the time of writing — I’ll be watching for the Bedrock launch and will follow up with real latency numbers when I can run my own agents against it.

Sources: AWS press release (March 13, 2026), Cerebras, Cerebras chip specifications. Content from sources was paraphrased for compliance with licensing restrictions. I work as a Solutions Architect at AWS; opinions here are my own.