Cerebras Explains Trillion-Parameter Model Deployment

Cerebras says it can run multi-trillion-parameter models across WSE-3T wafer-scale chips at 1,000–2,000+ tokens per second. Its deployment guide uses the open-source Kimi K3, a model with about 3 trillion parameters, to explain how it distributes model weights and request data across wafers. A single WSE-3T wafer has 44 GB of on-chip memory, while a trillion-parameter model’s weights alone require terabytes of storage.

Kimi K3 has 69 linear-attention layers and 24 dense layers. Cerebras cites its scores of 88.3% on Terminal-Bench 2.1, compared with GPT-5.6 Sol’s 88.8%, and 91.2% on BrowseComp, compared with GPT-5.6 Sol’s 90.4%. For the worked example, the company assumes an average request size of 256k, 4-bit weight quantization (storing each weight in four bits) and an 8-bit KV cache (memory holding information used to generate later tokens). At 4 bits per parameter, Kimi K3’s 3 trillion parameters require about 16 GB per layer when divided across its 93 layers.

Cerebras places one dense layer on each wafer, reserving the remaining memory for its KV cache, and two linear-attention layers on a wafer because those layers keep a fixed-size state per request. The 69 linear layers therefore use 35 wafers, while the 24 dense layers use 24, for 59 wafers total. Its compiler maps layers across wafer regions, and its runtime routes tokens between them. Only activations (intermediate results passed between layers) cross wafers, using RDMA over Ethernet at roughly 2 microseconds per hop; the company says weights typically stay on their wafer and entire layers fit on-chip.

Cerebras describes the model as a pipeline: each wafer runs a stage of layers and passes its intermediate results to the next wafer. To keep wafers busy, different requests move through different stages at the same time, and each stage can process a micro-batch, or small group of requests. The number of requests in flight is roughly the number of stages multiplied by the micro-batch size. Smaller micro-batches make each request faster, while larger ones increase total throughput.

The memory left for KV cache on wafers holding dense layers sets the maximum number of concurrent requests, because each in-flight request needs its own cache on each such wafer. Cerebras says concurrency and batch size alone do not show inference efficiency; it identifies total token throughput per unit of power, such as tokens per second per megawatt at the same output generation speed, as a more meaningful measure. The company says the same method can extend to models larger than 3 trillion parameters by distributing weights and request state across additional wafers, each adding compute and on-chip memory.

Cerebras says it can run multi-trillion-parameter models across WSE-3T wafer-scale chips at 1,000–2,000+ tokens per second. Its deployment guide uses the open-source Kimi K3, a model with about 3 trillion parameters, to explain how it distributes model weights and request data across wafers. A single WSE-3T wafer has 44 GB of on-chip memory, while a trillion-parameter model’s weights alone require terabytes of storage.

Kimi K3 has 69 linear-attention layers and 24 dense layers. Cerebras cites its scores of 88.3% on Terminal-Bench 2.1, compared with GPT-5.6 Sol’s 88.8%, and 91.2% on BrowseComp, compared with GPT-5.6 Sol’s 90.4%. For the worked example, the company assumes an average request size of 256k, 4-bit weight quantization (storing each weight in four bits) and an 8-bit KV cache (memory holding information used to generate later tokens). At 4 bits per parameter, Kimi K3’s 3 trillion parameters require about 16 GB per layer when divided across its 93 layers.

Cerebras places one dense layer on each wafer, reserving the remaining memory for its KV cache, and two linear-attention layers on a wafer because those layers keep a fixed-size state per request. The 69 linear layers therefore use 35 wafers, while the 24 dense layers use 24, for 59 wafers total. Its compiler maps layers across wafer regions, and its runtime routes tokens between them. Only activations (intermediate results passed between layers) cross wafers, using RDMA over Ethernet at roughly 2 microseconds per hop; the company says weights typically stay on their wafer and entire layers fit on-chip.

Cerebras describes the model as a pipeline: each wafer runs a stage of layers and passes its intermediate results to the next wafer. To keep wafers busy, different requests move through different stages at the same time, and each stage can process a micro-batch, or small group of requests. The number of requests in flight is roughly the number of stages multiplied by the micro-batch size. Smaller micro-batches make each request faster, while larger ones increase total throughput.

The memory left for KV cache on wafers holding dense layers sets the maximum number of concurrent requests, because each in-flight request needs its own cache on each such wafer. Cerebras says concurrency and batch size alone do not show inference efficiency; it identifies total token throughput per unit of power, such as tokens per second per megawatt at the same output generation speed, as a more meaningful measure. The company says the same method can extend to models larger than 3 trillion parameters by distributing weights and request state across additional wafers, each adding compute and on-chip memory.