Kimi K3 Open Weights: Deployment, Cost, and Who's Hosting It

What did Moonshot actually release?

Kimi K3's model card and Moonshot's own announcement lay out the architecture:

Spec	Value
Total parameters	2.8T (2.7799T per repo metadata)
Active parameters per token	~104B (16 of 896 routed experts)
Layers	93 total — 1 dense, 69 Kimi Delta Attention (KDA), 24 Gated MLA
Context window	1,048,576 tokens
Vision encoder	MoonViT-V2, 401M params, native image/video input
Native quantization	MXFP4 weights, MXFP8 activations (quantization-aware training)
Weight format	Safetensors, 96 shards, ~1.56TB / ~1.42TiB
License	Custom "Kimi K3 License" — read the license file before commercial use

The two architectural pieces Moonshot is calling out — Kimi Delta Attention (KDA) and Attention Residuals (AttnRes) — are a hybrid linear-attention design meant to make the 1M-token context window cheaper to serve than standard full attention would be at this scale.

Officially recommended serving engines are vLLM, SGLang, and TokenSpeed. There's no official Ollama, llama.cpp, or LM Studio support — a point every deployment writeup on K3 has flagged, because those tools are built around single-node, consumer-hardware inference that this model's architecture doesn't fit.

How do you actually deploy Kimi K3?

If you have the hardware, the vLLM day-0 support post and the model card give a real path. This is the honest version of "how to deploy it" — most of it only applies once you already have a multi-GPU node:

Minimum hardware. vLLM lists at least one 8× B300 (or GB300 NVL72) node, with 16× B200 also supported. AMD's ROCm path supports 8× MI355X. Moonshot's own production guidance is a "supernode" of 64 or more accelerators — the 8-GPU minimum gets the model running, not running at production throughput.

Quickstart commands, straight from the model card:

pip install vllm
vllm serve "moonshotai/Kimi-K3"

or via SGLang:

python -m sglang.launch_server --model-path moonshotai/Kimi-K3

Moonshot also documents a Docker path: docker model run hf.co/moonshotai/Kimi-K3.

Configuration details that aren't optional. Prefix caching is disabled by default for K3 in vLLM — you have to pass the flag explicitly or you lose most of the benefit of the 1M-token context on multi-turn workloads. MoE backend choice depends on your setup (deep_gemm_mega_moe for disaggregated/expert-parallel, flashinfer_trtllm for tensor-parallel >1), and the all-to-all backend depends on your interconnect (flashinfer_nvlink_one_sided for NVLink, deepep_v2 for RDMA). The vision encoder needs data parallelism by default, because its head_size=12 can't be sharded evenly across TP=8.

What throughput actually looks like. vLLM reports a baseline of 111 tokens/sec per user on TP8 and 118 tok/s on TP16 at batch size 1. With speculative decoding (DSpark) that rises to 331–370 tok/s per user — a 3.14x speedup, but only after you've tuned the speculative decoding path on top of an already-provisioned cluster.

The real cost of self-hosting

This is where "can I deploy it" and "should I deploy it" split. The math, cross-referenced from vLLM's blog and independent hardware/cost breakdowns:

• VRAM floor: ~1,680GB. The theoretical minimum from 2.8T params at 4-bit is about 1.4TB; the real serving footprint (weights + KV cache + overhead) runs higher.
• Storage: 1.56TB of weights, so plan for 4TB of fast NVMe just to hold the checkpoint plus scratch space. Downloading it takes anywhere from ~2 minutes on a 100Gbps line to nearly 35 hours on a 100Mbps connection.
• GPU configurations that clear the floor: 8× B300/MI355X (2,304GB aggregate), 16× H200 (2,256GB), 16× B200 (2,880GB), or 32× H100 (2,560GB). Nothing smaller works.
• Cloud rental, one estimate as of July 2026: an 8× B300 node runs roughly $59–$142/hour depending on provider, which is $43,000–$104,000/month running continuously. A 16× H200 node runs $46,600–$116,800/month.
• Break-even against the official API (at Moonshot's $0.30/$3.00/$15.00 per-million pricing for cache-hit input, cache-miss input, and output): the cheapest node estimate above only pays for itself past roughly 8 billion tokens/month with no caching, or 12.5 billion tokens/month at a 90% cache-hit rate.

If your actual usage is nowhere near 8B tokens a month — and almost nobody's is — a rented cluster is a way to spend tens of thousands of dollars a month to get a worse deal than the API.

Who's already running it?

K3's weights are hours old as of this writing, but day-0 support moved fast because Moonshot coordinated the release with inference vendors in advance:

• vLLM shipped official day-0 support with the throughput numbers above, spanning NVIDIA (Hopper and Blackwell) and AMD (MI355X) with ROCm support at launch.
• Fireworks AI put K3 on its platform at launch, positioning it as ownable, hosted inference rather than a self-managed cluster.
• Baseten published a day-zero API build guide walking through their own hosting setup.
• AMD published its own Instinct GPU deployment writeup, which is normal practice when a new frontier open-weight model launches with hardware-vendor day-0 backing.
• Multiple infrastructure blogs — Northflank, Hyperstack — published deployment and cost breakdowns within the first day, which tells you how much demand vendors expect for self-hosting guidance even though almost none of that audience will end up running the actual cluster.

The pattern matches every large open-weight release: a handful of inference platforms and the hardware vendor ship support on day zero, a wave of hardware/cost writeups follows within 24–48 hours, and the vast majority of actual usage still ends up going through an API rather than a self-managed deployment.

The simpler path: use Kimi K3 through an API

Given the hardware floor above, self-hosting K3 makes sense in a narrow set of cases: you're already running a large, mostly-idle GPU fleet, you have sustained usage well past the multi-billion-token break-even point, or you have a specific compliance reason data can't leave your infrastructure. Outside those cases, the cluster math above works against you.

APIMaster.ai already has kimi-k3 live in its model marketplace, routed through an OpenAI-compatible API. The route draws on Moonshot's own API as well as outside cloud GPU infrastructure running the open weights, so pricing and availability reflect more than one path to the same model — check the live route card before moving production volume, since channel supply and pricing can shift.

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_APIMASTER_KEY",
    base_url="https://apimaster.ai/v1",
)

response = client.chat.completions.create(
    model="kimi-k3",
    reasoning_effort="max",
    messages=[
        {"role": "user", "content": "Summarize the tradeoffs of self-hosting a 2.8T MoE model."}
    ],
)

print(response.choices[0].message.content)

To get started:

1. Register for an APIMaster account.
2. Add wallet credit with a supported payment method.
3. Generate an API key from the APIMaster console.
4. Set model to kimi-k3 and the base URL to https://apimaster.ai/v1 in your existing OpenAI SDK integration.

Kimi K3 is also part of APIMaster's current 40%-off pricing on DeepSeek, Kimi K3, MiniMax M3, and GLM-5.2, and every route can be checked with the free AI model fingerprint tester before you rely on it in production.

FAQ

Can I run Kimi K3 on a single GPU or a normal workstation?

No. The realistic VRAM floor is around 1,680GB. Even an 18-card RTX PRO 6000 Blackwell workstation (96GB per card) only just clears that number on paper, with no practical topology to actually serve the model that way. A Mac Studio's maximum 512GB of unified memory is about a third of what's needed.

What's the cheapest way to legitimately self-host Kimi K3?

Renting an 8× B300 or 8× MI355X cloud node is the entry point, at roughly $43,000–$104,000/month depending on provider and instance pricing. That only becomes cost-competitive with the official API past several billion tokens of monthly usage.

Does Ollama or LM Studio support Kimi K3?

Not officially. Moonshot's recommended serving engines are vLLM, SGLang, and TokenSpeed — all built for multi-GPU, data-center deployment, not single-machine consumer inference.

Is Kimi K3 available through APIMaster?

Yes. It's live in the APIMaster marketplace as kimi-k3 behind an OpenAI-compatible endpoint, drawing on both Moonshot's own API and outside cloud GPU capacity running the open weights.

How does K3's context window compare to running it locally vs through an API?

The 1M-token window is identical either way — it's a property of the model, not the serving path. What changes is prefix caching behavior and cost: APIMaster's route and Moonshot's own API both support cache-hit pricing on repeated long prefixes, while a self-hosted vLLM deployment requires explicitly enabling prefix caching (it's off by default for K3) to get the same benefit.

Sources

• Kimi K3 model card, Hugging Face
• Moonshot AI, "Open Frontier Intelligence" launch post
• vLLM, "Kimi K3 Is Here: Efficient Day-0 Support on vLLM"
• Fireworks AI, Kimi K3 launch post
• Baseten, day-zero API build guide
• AMD, Kimi K3 on AMD Instinct GPUs
• Kingy.ai, hardware/VRAM/cost breakdown
• Northflank, benchmarks/pricing/self-hosting overview
• Kimi K3 official pricing

Kimi K3's weights are open, but for almost everyone the fastest path to using the model is still an API call, not a GPU cluster. Register on APIMaster to get an OpenAI-compatible key for kimi-k3 without provisioning any hardware.