Moonshot has released Kimi K3: a 2.8-trillion-parameter Mixture-of-Experts model with native vision and a 1-million-token context window — the world’s first open 3T-class model, positioned for long-horizon coding, knowledge work, and reasoning. The honest framing comes first: the company states plainly that overall performance still trails the strongest proprietary models, Claude Fable 5 and GPT 5.6 Sol, while reaching frontier level across its own evaluation suite and consistently beating the other tested models. Full weights are promised by July 27, 2026, with architecture and training details in a coming technical report.
Architecture: Delta Attention, Attention Residuals, and Stable LatentMoE
K3’s backbone pairs two updates: Kimi Delta Attention (KDA) provides an efficient foundation for scaling attention across sequence length, while Attention Residuals (AttnRes) selectively retrieve representations across depth rather than accumulating them uniformly. Sparsity comes from Stable LatentMoE — effectively activating 16 of 896 experts — with Quantile Balancing (deriving expert allocation directly from router-score quantiles, eliminating a sensitive balancing hyperparameter) and Per-Head Muon (optimizing attention heads independently) solving the first-order routing and stability challenges at this scale. The company claims roughly a 2.5× improvement in overall scaling efficiency over K2.
The engineering details matter too: quantization-aware training from the SFT stage onward (MXFP4 weights, MXFP8 activations) for hardware compatibility; a fully balanced expert-parallel training method with static shapes and no host synchronization on the critical path; and, because KDA complicates conventional prefix caching, a corresponding implementation contributed to the vLLM community, released alongside the model. Deployment guidance: supernode configurations with 64 or more accelerators.
Coding Cases: From GPU Kernels to a Chip
The official cases show off long-horizon range. In a GPU kernel-optimization test, each model worked independently in an identical sandbox with up to 24 hours, across four tasks on NVIDIA Hopper and an alternative vendor’s GPGPUs — K3 performed competitively with Fable 5 (evaluated by a third party, results may include fallback) and substantially outperformed Opus 4.8, GPT 5.6 Sol, and GPT 5.5.
Harder still is MiniTriton: K3 built a Triton-like compiler from scratch, with its own tile-level IR layer over MLIR, optimization passes, and a PTX code generation pipeline. On supported roofline benchmarks it matches or beats Triton and torch.compile — surpassing Triton on certain workloads — and sustains end-to-end nanoGPT training with stable convergence, the loss curve closely tracking the reference. The chip-design proof of concept is a single 48-hour autonomous run: using open-source EDA tools on the Nangate 45nm library, K3 built, optimized, and verified a chip for a nano model based on its own architecture — timing closed at 100 MHz within 4 mm², sustaining over 8,700 tokens/s decode in simulation, with 1.46M standard cells and an INT4 MAC array with fused dequantization.
Scientific computing gets a concrete number: reproducing the I–Love–Q relations in computational astrophysics, K3 finished in about two hours what would take an experienced researcher one to two weeks — cross-validating 20+ papers, implementing the full numerical pipeline, evaluating 300+ equations of state, identifying inconsistencies in published formulas, and producing 3,000+ lines of Python plus an interactive dashboard.
Knowledge Work: Interactive Research and Widgets
In Kimi Work, three cases lead: an interactive website covering 42 years of the ASIC industry (120+ rounds of recursive self-improvement, 2.8k+ web searches/fetches, 1.1k+ terminal data pulls, spanning 87 quarterly reports and 99 original PDFs); a consulting-grade fusion-industry report with timelines and Gantt charts; and an analysis of 391 gravitational-wave events using 20+ concurrent subagents. Two new features arrive alongside: Widgets (interactive components generated in-chat, connectable to local data or external plugins) and Dashboard (a persistent, personalized view organized around a topic or project). Video editing is called out too — K3 cut its own teaser from 56 source clips with motion-matched cuts and frame-accurate beat sync, work the company estimates at one to two days for an experienced editor.
Availability and Pricing
K3 is live on the Kimi app (iOS/Android/HarmonyOS), Kimi Work desktop (3.1.0+), Kimi Code (/model in the terminal), and the Kimi API platform (model name kimi-k3). API pricing: $0.30/MTok cache-hit input, $3.00/MTok cache-miss input, $15.00/MTok output — underpinned by Mooncake’s disaggregated inference architecture, with the official API reportedly achieving above 90% cache hit rates on coding workloads, which is what makes the low input price sustainable. Kimi Enterprise adds member management with complete separation between personal and organization accounts.
Reality Checks Before Adoption
Three things to weigh. First, the company itself says it still trails Fable 5 and GPT 5.6 Sol — “strongest open model” is not “strongest model”; compare benchmarks per task, and mind the evaluation conditions (max reasoning effort, temperature 1.0, varying harnesses per model — Fable 5 hit fallbacks on 35% of SWE Marathon tasks in their evaluation). Second, the deployment bar is real: 64+ accelerator supernodes are recommended, and KDA’s prefix caching needs their contributed vLLM implementation to be efficient. Third, launch defaults to max thinking effort, with low/high modes only in later updates — early cost estimates should assume max.
Sources
AI-assisted summary compiled from the sources above, reviewed by a human before publishing.