Kimi K3 — the first open 3T-class model, explained

For most of the past two years, “open-weight” meant “a class below the frontier.” The largest open models topped out around a trillion parameters while the proprietary frontier moved on. On July 16, Moonshot AI shipped Kimi K3: 2.8 trillion parameters, a 1-million-token context window, native vision — and full weights promised by July 27.

That makes it the first open model in the 3-trillion-parameter class. Even Moonshot is careful about the claim that matters most: by their own admission, K3 still trails Claude Fable 5 and GPT 5.6 Sol overall. But “frontier-level performance with weights you can download” is a sentence that has never been true at this scale before.

If you want the fast version, this 13-minute explainer by Caleb Writes Code is a good use of your coffee break — it walks through the three architectural ideas that carry the model:

Here is my longer read on what’s actually new.

The architecture: three ideas, one goal

Everything in K3’s design serves a single objective: keep scaling efficient past the trillion-parameter mark. Moonshot claims roughly a 2.5× improvement in scaling efficiency over Kimi K2 — more intelligence per unit of compute — and attributes it to three structural changes.

Kimi Delta Attention (KDA) is the foundation — an attention variant built to stay cheap as sequence length grows. It’s what makes the 1M-token context economically servable rather than a demo number. One telling detail: KDA broke conventional prefix caching, so Moonshot wrote a new implementation and contributed it to vLLM rather than keeping it proprietary. Cache-hit input is priced at $0.30/MTok precisely because that caching works.

Attention Residuals (AttnRes) address information flow across depth: instead of every layer uniformly accumulating representations, the model selectively retrieves them from earlier blocks. Think of it as skip connections that learned what to skip.

Stable LatentMoE is the sparsity play: 896 experts, only 16 active per token. At that sparsity, routing stability becomes the hard problem, and the supporting cast matters — Quantile Balancing derives expert allocation from router-score quantiles instead of a fragile balancing hyperparameter, and Per-Head Muon optimizes attention heads independently. Whether those names survive the technical report, the direction is clear: extreme sparsity is now the only affordable way to buy parameters.

One more choice worth noting: K3 was trained quantization-aware from the SFT stage — MXFP4 weights, MXFP8 activations. Quantization isn’t a post-hoc compression trick here; it’s the native format. That’s a big deal for anyone planning to actually run the weights.

What the benchmarks say (and don’t)

Moonshot’s evaluation suite shows K3 consistently ahead of other open models and competitive with the frontier on long-horizon coding — the kernel-optimization case study (24-hour autonomous sessions profiling and rewriting GPU kernels) is the most interesting one, because an early K3 reportedly handled most of the team’s own kernel-optimization work during development.

The showcase projects are chosen to impress and they do: a Triton-like compiler built from scratch that beats Triton on some workloads, a chip designed and verified in a 48-hour autonomous run, a computational-astrophysics reproduction that compressed one to two weeks of expert work into about two hours.

The honest footnotes are more informative than the headline numbers, though. Moonshot openly documents harness differences per benchmark, fallback behavior in competitor evaluations, and — rare candor — a limitations section that admits K3 “exhibits a noticeable gap in user experience” versus Claude Fable 5 and GPT 5.6 Sol.

The limitation agentic builders should read twice

Two items in the limitations section deserve attention if you build agents:

• Thinking-history sensitivity. K3 was trained with preserved thinking history. If your harness drops historical reasoning content — or you swap K3 into a session started by another model — generation quality can become unstable. Model-agnostic agent frameworks, take note: “swap the model mid-session” is not a free operation here.
• Excessive proactiveness. Moonshot says K3’s long-horizon training makes it prone to making decisions on your behalf when intent is ambiguous, and recommends imposing explicit behavioral constraints in the system prompt or AGENTS.md. An official model card telling you to constrain the model via AGENTS.md is a sign of the times — guardrails are becoming part of the model’s documented interface, not an afterthought.

Why this matters

The pricing is aggressive ($0.30/MTok cache-hit input, $15/MTok output), but the API was never the point. The point is July 27, when the weights land. A 3T-class model with native MXFP4 weights, a vLLM-supported attention stack, and a documented preference for 64-accelerator supernodes is a model designed to be deployed by other people — inference providers, enterprises with sovereignty requirements, and every team whose data cannot leave their infrastructure.

The open-weight frontier isn’t trailing by a class anymore. It’s trailing by a margin — and margins close.

Sources: Moonshot AI’s Kimi K3 announcement, Caleb Writes Code’s video explainer. Content from sources was paraphrased and condensed.