Skip to content

Granite 4.1 LLMs: How They’re Built

70
Academic
Chimera Difficulty Score
a synthesis of Flesch-Kincaid, Coleman-Liau, SMOG, and Dale-Chall readability metrics
Granite 4.1 LLMs: How They’re Built Authors: Granite Team, IBM TL;DR — Granite 4.1 is a family of dense, decoder‑only LLMs (3B, 8B, and 30B) trained on ~15T tokens using a multi‑stage pre‑training pipeline, including long‑context extension of up to 512K tokens. The models are further refined with supervised fine‑tuning on ~4.1M high‑quality curated samples and reinforcement learning via on‑policy ...
The Granite 4.1 models represent a significant advancement in open-source language models, emphasizing data quality and rigorous training pipelines over sheer scale. The methodology is robust, with a clear progression from broad pre-training to targeted reinforcement learning, ensuring strong performance across diverse tasks. The use of an LLM-as-Judge framework for SFT data curation is particularly noteworthy, as it addresses the critical challenge of maintaining high-quality training data. The...