Disaggregated Inference: Why AWS and Cerebras Are Splitting with the GPU
Cloud inference is being broken down: AWS and Cerebras are separating prefill and decode onto their own chips.
AWS and Cerebras are breaking AI inference apart. Instead of a single GPU handling everything, one chip takes the input and a second handles the output. The move sounds like a hardware detail – but it shifts the question every cloud team asks as AI workloads grow: which compute do I buy for what, and where am I paying for capacity I never use?
Key Takeaways
- Divided work, divided hardware: AWS and Cerebras split inference into two phases. AWS Trainium chips handle prefill – processing the input – while Cerebras systems handle decode, generating the response.
- Speed is the promise: According to the vendors, the architecture runs significantly faster than existing inference on Amazon Bedrock. The numbers come from manufacturer benchmarks and should be read accordingly.
- For cloud teams, the underlying logic is what matters: Specialized chips for distinct phases are a signal. Anyone serious about inference costs will plan workloads by phase in the future – not around a generic GPU.
Related:Apple splits AI inference: on-device vs. cloud / OpenTofu vs. Terraform: which IaC tool holds up
What AWS and Cerebras have announced
What is disaggregated inference? Disaggregated inference breaks the process of answering an AI request into separate phases and assigns each to its own purpose-built hardware. Rather than a single accelerator carrying the entire workload, specialized chips each handle the part they are best suited for.
In March 2026, AWS and Cerebras announced a collaboration that puts this principle into practice. According to the vendors, AWS Trainium chips handle prefill – processing the prompt – while Cerebras hardware handles decode, generating the response token by token. The two are connected via a high-speed network link. The combined system is to be offered as an additional inference tier within Amazon Bedrock, with no need for customers to provision new instance types or build custom interfaces.
On performance, the vendors cite concrete figures. According to published manufacturer benchmarks, the Cerebras system achieves a multiple of the token rate of current GPUs on a large open model, and the combined architecture is said to be several times faster than existing Bedrock inference. These numbers come from the vendor and should be validated against your own workloads before drawing conclusions.
Why Prefill and Decode Require Different Hardware
The two phases of a request place opposing demands on hardware. That is precisely what makes separating them worthwhile.
Prefill processes the entire input in one pass. It is compute-intensive but places relatively little pressure on memory bandwidth. Decode, by contrast, generates the response word by word, requiring the model to be fetched from memory for every single token. This phase is bandwidth-hungry and stresses a conventional GPU in an entirely different way. Using a single hardware platform for both always means compromise: what works well for prefill often sits idle during decode.
| Characteristic | Prefill | Decode |
|---|---|---|
| Task | Process input | Generate response token by token |
| Bottleneck | Compute | Memory bandwidth |
| Profile | Short, parallelizable | Sequential, memory-heavy |
| In the AWS–Cerebras model | AWS Trainium | Cerebras system |
What This Means for Cloud Teams’ GPU Strategy
Few DACH organizations will adopt this architecture directly. The underlying principle, however, is relevant to all of them. It demonstrates that a single general-purpose GPU is increasingly a compromise for every AI workload – at least wherever speed and cost truly matter.
In practical terms, this has three implications. First, it is worth analyzing inference workloads by phase rather than booking compute time in bulk. Second, the cost equation shifts: reserving an expensive GPU for a bandwidth-intensive decode phase means paying for compute that sits idle. Third, vendor dependency deserves attention. An inference tier that is exclusively tied to a single cloud platform delivers speed, but deepens lock-in to exactly that provider.
For teams running models today in their own environments or via Kubernetes, this points to where things are heading. Scheduling and resource planning are becoming more granular. Teams that understand early which phase of their workloads consumes which resource will be better positioned to make procurement decisions based on real utilization – not broad assumptions.
Frequently Asked Questions
What does disaggregated inference mean in plain terms?
Answering an AI request is split into phases that run on different hardware. In the case of AWS and Cerebras, one chip handles processing the input, another generates the response. Each phase runs on the hardware best suited to it.
What is the difference between prefill and decode?
Prefill processes the input and is primarily compute-intensive. Decode generates the response token by token, which demands high memory bandwidth because the model must be fetched from memory at every step. These opposing profiles are precisely why the two phases are separated.
How reliable are the speed figures cited?
The numbers come from vendor benchmarks published by AWS and Cerebras. They indicate the direction of travel, but are no substitute for your own testing. Anyone evaluating this architecture should validate it against their own models and load profiles before relying on those figures.
Does my cloud team need to act on this now?
Not immediately. The architecture is initially offered exclusively through Amazon Bedrock. The sensible response is to understand your own inference workloads by phase and bottleneck, so that future procurement decisions are grounded in real utilization data.
What risks come with a specialized inference tier?
Vendor lock-in, above all. An inference tier that runs only on a single cloud platform delivers speed gains, but binds your workloads to that provider. This dependency belongs in every cost-benefit calculation alongside the raw performance improvement.
Title image: AI-generated (June 2026)
Image source: AI-generated (Juli 2026)
