AWS published a deployment guide on July 30, 2026 for running Moonshot AI's Kimi K3, a 2.8 trillion parameter open-weight model released on July 27, 2026, on AWS infrastructure. The guide gives two deployment paths: Amazon SageMaker HyperPod with the Inference Operator, and a standalone Amazon Elastic Kubernetes Service cluster. AWS says hosting a multi-trillion parameter architecture requires purpose-built infrastructure, high-end GPU compute, and optimized serving frameworks because open-weight models now handle multi-step agentic workflows, advanced reasoning, and long-horizon coding.

Kimi K3 uses a Mixture of Experts architecture, Kimi Delta Attention, Gated Multi Head Latent Attention, and a Stable LatentMoE framework. The model has 2.8 trillion total parameters distributed across 896 specialist experts, but activates only 16 experts per token, leaving about 104 billion parameters active during a single forward pass. AWS says that design gives a 2.5x improvement in scaling efficiency over Kimi K2. The model also has a 1 Million Tokens context window, Native Multimodal support for Text + Vision, native tool calling, structured output, and an always-on thinking mode for multi-step problem solving.

The Kimi K3 weights are available on Hugging Face as moonshotai/Kimi-K3 and are distributed in MXFP4, a 4-bit numeric format for memory-efficient inference. AWS says serving the model currently requires a vLLM day-0 inference container for Kimi K3, with commits available in vllm/vllm-openai:kimi-k3, and says those commits are expected to move into the main vLLM container in upcoming releases. vLLM is recommended because it supports MoE architectures, tensor parallelism, and MXFP4 quantization.

AWS says Kimi K3 requires a p6-b300 instance, listed for SageMaker as ml.p6-b300.48xlarge, with 8 NVIDIA B300 Blackwell Ultra GPUs and high-bandwidth interconnects for tensor-parallel inference across the model's expert pool. AWS lists two capacity options: Flexible Training Plans for SageMaker HyperPod, which reserve committed capacity for sustained workloads, and EC2 Capacity Blocks, which reserve GPU instances for a defined period and can be consumed by Amazon EKS workloads.

For SageMaker HyperPod, the guide instructs users to create a HyperPod cluster orchestrated by Amazon EKS, choose Quick setup or Custom setup, enable default Helm charts and add-ons, and add a worker group using ml.p6-b300.48xlarge. Users then attach or create a Flexible Training Plan, set the Target Availability Zone to match the reserved capacity, and deploy an InferenceEndpointConfig manifest with kubectl apply -f kimi-k3.yaml. The Inference Operator downloads the model from Hugging Face, schedules the container, performs health checks, and exposes an OpenAI compatible API after the endpoint is ready.

For standalone Amazon EKS, the guide points to the AI on EKS Kimi K3 recipe, which uses Terraform modules to create a GPU-optimized EKS cluster with VPC networking, managed node groups, and IAM roles. The EKS flow reserves p6-b300.48xlarge Capacity Blocks, installs NVIDIA drivers and the Kubernetes device plugin, deploys the vLLM container with tensor-parallel size of 8, MXFP4 load format, and MoE backend settings, then exposes port 8000 through a LoadBalancer or Ingress controller for the OpenAI compatible /v1/chat/completions endpoint. The guide says deployments can be validated with the OpenAI Python SDK or curl, and cleanup requires deleting Kubernetes resources, clusters or node groups, and unused capacity reservations.

AWS published a deployment guide on July 30, 2026 for running Moonshot AI's Kimi K3, a 2.8 trillion parameter open-weight model released on July 27, 2026, on AWS infrastructure. The guide gives two deployment paths: Amazon SageMaker HyperPod with the Inference Operator, and a standalone Amazon Elastic Kubernetes Service cluster. AWS says hosting a multi-trillion parameter architecture requires purpose-built infrastructure, high-end GPU compute, and optimized serving frameworks because open-weight models now handle multi-step agentic workflows, advanced reasoning, and long-horizon coding.

Kimi K3 uses a Mixture of Experts architecture, Kimi Delta Attention, Gated Multi Head Latent Attention, and a Stable LatentMoE framework. The model has 2.8 trillion total parameters distributed across 896 specialist experts, but activates only 16 experts per token, leaving about 104 billion parameters active during a single forward pass. AWS says that design gives a 2.5x improvement in scaling efficiency over Kimi K2. The model also has a 1 Million Tokens context window, Native Multimodal support for Text + Vision, native tool calling, structured output, and an always-on thinking mode for multi-step problem solving.

The Kimi K3 weights are available on Hugging Face as moonshotai/Kimi-K3 and are distributed in MXFP4, a 4-bit numeric format for memory-efficient inference. AWS says serving the model currently requires a vLLM day-0 inference container for Kimi K3, with commits available in vllm/vllm-openai:kimi-k3, and says those commits are expected to move into the main vLLM container in upcoming releases. vLLM is recommended because it supports MoE architectures, tensor parallelism, and MXFP4 quantization.

AWS says Kimi K3 requires a p6-b300 instance, listed for SageMaker as ml.p6-b300.48xlarge, with 8 NVIDIA B300 Blackwell Ultra GPUs and high-bandwidth interconnects for tensor-parallel inference across the model's expert pool. AWS lists two capacity options: Flexible Training Plans for SageMaker HyperPod, which reserve committed capacity for sustained workloads, and EC2 Capacity Blocks, which reserve GPU instances for a defined period and can be consumed by Amazon EKS workloads.

For SageMaker HyperPod, the guide instructs users to create a HyperPod cluster orchestrated by Amazon EKS, choose Quick setup or Custom setup, enable default Helm charts and add-ons, and add a worker group using ml.p6-b300.48xlarge. Users then attach or create a Flexible Training Plan, set the Target Availability Zone to match the reserved capacity, and deploy an InferenceEndpointConfig manifest with kubectl apply -f kimi-k3.yaml. The Inference Operator downloads the model from Hugging Face, schedules the container, performs health checks, and exposes an OpenAI compatible API after the endpoint is ready.

For standalone Amazon EKS, the guide points to the AI on EKS Kimi K3 recipe, which uses Terraform modules to create a GPU-optimized EKS cluster with VPC networking, managed node groups, and IAM roles. The EKS flow reserves p6-b300.48xlarge Capacity Blocks, installs NVIDIA drivers and the Kubernetes device plugin, deploys the vLLM container with tensor-parallel size of 8, MXFP4 load format, and MoE backend settings, then exposes port 8000 through a LoadBalancer or Ingress controller for the OpenAI compatible /v1/chat/completions endpoint. The guide says deployments can be validated with the OpenAI Python SDK or curl, and cleanup requires deleting Kubernetes resources, clusters or node groups, and unused capacity reservations.