On April 29, 2026, IBM released the Granite 4.1 family: dense decoder-only language models in 3B, 8B, and 30B sizes, all shipped under Apache 2.0. The headline claim is that the 8B instruct model “consistently matches or outperforms” IBM’s own previous 32B Mixture-of-Experts model, Granite 4.0-H-Small — on the BFCL V3 tool-calling benchmark, the 8B scores 68.3 against the 32B MoE’s 64.7.
In a 2026 where reasoning models and ever-longer thinking chains dominate releases, IBM is swimming the other way: Granite 4.1 deliberately ships without a reasoning mode, trading “thinking” for predictable latency and cost — a direct aim at enterprise deployment pain. It is another sharp data point in the ongoing divergence of open-model strategies.
An 8B Dense Model Catches a 32B MoE
The whole line returns to dense architecture (GQA, RoPE, SwiGLU, RMSNorm, tied embeddings) — no MoE routing, no sparse layers. On IBM’s own evals: the 3B scores 60.8 on BFCL V3, edging Qwen3-8B at 60.2 despite being roughly half the size; the 30B’s 73.7 leads Gemma-4-31B’s 72.7 and tops IBM’s chart; on IFEval the 8B’s 87.1 is effectively tied with Qwen3.5-9B’s 87.2; GSM8K lands at 92.5 for the 8B and 94.2 for the 30B. Context stretches to 512K tokens for the 8B and 30B (the 3B caps at 128K). Dense has practical upsides: simpler architecture, easier fine-tuning, and compute per token you can predict.
Reasoning Removed on Purpose
There is no chain-of-thought mode in Granite 4.1 — not missing, removed. IBM’s argument: enterprise tasks need predictable latency, stable token usage, and lower operational cost, and long reasoning chains break all three at once. The twist is that Long CoT data did go into training (about 12.5% of phase three), but the model does not emit it at inference. IBM also ships FP8 quantized variants — linear layers only — cutting memory and disk roughly in half with most performance intact, optimized for vLLM.
15T Tokens, Four RL Stages
Pre-training ran on roughly 15T tokens across five phases: phase one (10T) led with CommonCrawl (~59%), code ~20%, math ~7%; phase two pushed math to 35% and code to 30%; phase five staged context extension (32K to 128K to 512K) with a model merge after each stage so long-context training would not eat short-context performance — a rare piece of honest engineering detail. Post-training starts with an LLM-as-Judge scoring six dimensions to curate about 4.1 million SFT samples, auto-rejecting hallucinations, false premises, and wrong arithmetic. Then come four RL stages (on-policy GRPO with DAPO loss): stage-two RLHF lifted AlpacaEval by ~18.9 points on average but damaged math; stage four existed specifically to repair it, recovering ~3.8 points on GSM8K and ~23.5 on DeepMind-Math. The training infrastructure: NVIDIA GB200 NVL72 on CoreWeave.
More Than Language Models
The 4.1 family extends beyond LLMs. Granite Speech 4.1 (2B) hits a 5.33% word-error rate, among the top of the OpenASR Leaderboard, with a non-autoregressive NAR variant for throughput. Granite Vision 4.1 targets document understanding — tables, charts, key-value extraction — and ships alongside ChartNet, a newly released million-scale chart-understanding dataset. Guardian 4.1, a moderator fine-tuned from the 8B, works with any LLM and flags bias, hallucination, jailbreaks, and agentic risks. Granite Embedding Multilingual R2 covers 200+ languages. Weights are on Hugging Face (ibm-granite), Ollama, LM Studio, watsonx, and OpenRouter, with day-one support in vLLM, SGLang, and llama.cpp. A full locally deployable enterprise family — a route that lines up with the open-source divergence we flagged in our opening-of-year outlook (2026 opening outlook).
How to Read Self-Reported Scores
A third-party analysis (Firethering) puts the necessary caveat on the table: every comparison above comes from IBM’s own evaluation harness. The numbers look plausible and directionally consistent with third-party reports, but the methodology deserves scrutiny. The other sober reading: an 8B beating the previous 32B MoE says both that 4.1’s data engineering is solid and that the 4.0 MoE was underbuilt — as that author put it, “probably both.” The advice for developers is simple: Apache 2.0, 512K context, and a 3B that runs on consumer hardware mean the barrier to trying it is near zero. Benchmarking on your own workload beats trusting anyone’s leaderboard.
Sources
- The IBM Granite 4.1 family of models — IBM Research
- Granite 4.1 LLMs: How They’re Built — Hugging Face
- Granite 4.1: IBM’s 8B Model Matching 32B MoE — Firethering
AI-assisted summary compiled from the sources above, reviewed by a human before publishing.