LFM2.5-Encoders for Fast Long-Context Inference on CPU
Here's what you get:
- Strong for their size: match or beat larger encoders on GLUE, SuperGLUE, and multilingual tasks.
- 8,192-token context with latency that grows slowly as inputs get longer.
- Fast on CPU: about 3.7× faster than ModernBERT-base at long context.
With these, you can build intent routers, policy linters, PII detectors, and text classifiers that run cheaply, all day. See the live demos below.
Why we built a general-purpose encoder
Last month we released LFM2.5-Retrievers, built for multilingual search. LFM2.5-Encoders come from the same family but serve a broader purpose. They're pre-trained with a masked-language objective, so you can fine-tune them for classification, token-level tasks, and search alike. Search is just one thing an encoder enables. That's why we built a general-purpose model instead of reusing the retrievers.
Encoders power many modern production NLP applications: classifiers, intent routers, safety filters. These jobs run all day, usually on CPU, on ever-longer inputs. BERT established this class of model, and recently ModernBERT pushed its accuracy, speed, and context further. LFM2.5-Encoders take the next step on the LFM2 architecture, where cost grows slowly as inputs grow.
How the encoders are built
We initialize the encoders from their respective LFM2 decoder backbones: LFM2.5-230M and LFM2.5-350M. Then we turn each causal decoder into a bidirectional encoder with a few changes:
- Bidirectional attention mask: each token now sees the tokens on both sides, not just the ones before it.
- Non-causal short convolutions: we pad them symmetrically so each token's convolution mixes in its neighbors on both sides.
- Masked language modeling: we mask 30% of the tokens during training.
We train both models in two stages:
- General language competence: a short-context masked-language objective on a large web corpus at a 1,024-token context.
- Long-context adaptation: extending context to 8,192 tokens on the full data mix, strengthening factual, legal, and multilingual competence.
Benchmark Results
We fine-tune each model fully on every task and report the resulting score. Across the table, that's 14 models on 17 tasks pulled from GLUE, SuperGLUE, and multilingual classification.
We report the mean across five held-out seeds, so the numbers are stable run to run. The full framework and raw results are open-sourced.
LFM2.5-Encoder-350M ranks fourth of the 14 models. The three ahead of it are all larger, including a 3.5B model nearly 10 times its size. LFM2.5-Encoder-230M beats ModernBERT-base and every EuroBERT model, while being smaller than most of them. Both also score well above our own LFM2.5-Retrievers here.
Inference speed on CPU and GPU
Our encoders inherit the LFM2 backbone's fast inference. Since both our encoders and ModernBERT support an 8,192-token context, we measure speed across the full range.
Our encoders show their biggest edge on CPU. Here, LFM2.5-Encoder-230M is the fastest at every sequence length (even faster than the smaller ModernBERT-base for short inputs). With increasing input length, throughput decreases sharply for ModernBERT, while our LFM2.5-Encoders rise into the mid-range before tapering. At 8,192 tokens, ModernBERT-base takes over a minute and a half per forward pass versus about 28s for LFM2.5-Encoder-230M. This is about 3.7x faster. For developers, that means you can scan or classify a full contract, transcript, or long support thread in under 30 seconds on a laptop CPU.
On GPU, a similar pattern holds with a smaller margin: ModernBERT-base leads below ~1K tokens on the Apple GPU. Our encoders take the lead from about 2K tokens. This shows that for long inputs, LFM2.5-Encoders are the faster choice, and if you're running on CPU, dramatically so.
LFM2.5-Encoder demos
We built the demos below from fine-tuned LFM2.5-Encoders. Each one runs in a CPU-only Hugging Face space:
- Zero-shot prompt routing: define your own routing lanes as free text. The model scores the whole prompt against every lane in one pass.
- Zero-shot policy linting: check text against your company's rules, written as free text. It scores every token against every rule in one pass.
- Spell checking: correct misspellings token by token.
- PII detection: spot and remove 40 kinds of personal information across 16 languages.
- Masked-diffusion text generation (bonus): run the encoder as a chatbot that generates text by iteratively unmasking instead of left to right.
How to use and fine-tune LFM2.5-Encoders
Reach for an LFM2.5-Encoder when you have a high-volume understanding task, such as classification, routing, extraction, or scoring, that runs constantly and has to stay cheap and fast. For jobs like these, a fine-tuned encoder is smaller, faster, and far cheaper to run than a generative LLM, and it fits on the CPUs you already have.
Between the two encoder sizes:
- LFM2.5-Encoder-350M: choose it when accuracy matters most.
- LFM2.5-Encoder-230M: choose it for tighter hardware or higher throughput.
You can start in a few lines. Load a model with transformers
. Then run it directly for masked-token prediction, or attach your own head and fine-tune it for your task.
Load and run the model
Install the latest version of transformers
:
pip install -U transformers
Run masked-token prediction:
from transformers import AutoModelForMaskedLM, AutoTokenizer
import torch
model_id = "LiquidAI/LFM2.5-Encoder-230M" # or "LiquidAI/LFM2.5-Encoder-350M"
tok = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
mlm = AutoModelForMaskedLM.from_pretrained(model_id, trust_remote_code=True)
text = f"The capital of France is {tok.mask_token}."
enc = tok(text, return_tensors="pt")
with torch.no_grad():
logits = mlm(enc).logits
pos = (enc["input_ids"][0] == tok.mask_token_id).nonzero()[0].item()
print([tok.decode([t]).strip() for t in logits[0, pos].topk(5).indices.tolist()])
-> ['Paris', 'Strasbourg', 'Paris', 'Lyon', 'Versailles']
For downstream tasks, load the encoder body and attach your own head (classification, token classification, regression, retrieval):
from transformers import AutoModel
body = AutoModel.from_pretrained(model_id, trust_remote_code=True)
If your GPU supports it, use Flash Attention 2 for the highest efficiency:
pip install flash-attn
Fine-tuning for your task
A base encoder gives you general-purpose representations, not task outputs. So you fine-tune it for each task. Our fine-tuning tutorial walks through fine-tuning on long legal documents with an 8k context.
Get started with LFM2.5-Encoders
Both encoders are open-weight and available on Hugging Face today:
- Download: LFM2.5-Encoder-230M and LFM2.5-Encoder-350M on Hugging Face.
- Try: run the demos above in your browser, no setup needed.
- Fine-tune: adapt an encoder to your task with our fine-tuning tutorial.
We can't wait to see what you build.
Citation
If you use this work, please cite the release blog:
@article{liquidAI2026Encoders,
author = {Liquid AI},
title = {LFM2.5-Encoders: Fast at Long Context, Even on CPU},
journal = {Liquid AI Blog},
year = {2026},
note = {www.liquid.ai/blog/lfm2-5-encoders},
}
Facts Only
* LFM2.5-Encoders handle 8,192-token context with slow latency growth.
* Encoders match or beat larger encoders on GLUE, SuperGLUE, and multilingual tasks.
* Inference is fast on CPU, achieving about 3.7× faster performance than ModernBERT-base at long context.
* The encoder models are initialized from LFM2.5-230M and LFM2.5-350M decoder backbones.
* Training involved a general language competence stage (1,024 tokens) and a long-context adaptation stage (8,192 tokens).
* The LFM2.5-Encoder-350M ranks fourth among fourteen models on 17 tasks from GLUE, SuperGLUE, and multilingual classification.
* LFM2.5-Encoder-230M beats ModernBERT-base and EuroBERT models while being smaller than most of them.
* On CPU, LFM2.5-Encoder-230M is the fastest across all sequence lengths.
* On CPU, at 8,192 tokens, LFM2.5-Encoder-230M takes about 28 seconds, compared to over a minute and a half for ModernBERT-base.
* The models can be fine-tuned for classification, routing, extraction, and scoring tasks.
Executive Summary
LFM2.5-Encoders are models designed for fast, long-context inference on CPUs, capable of handling up to 8,192 tokens with latency that scales slowly with input length. These encoders perform strongly on GLUE, SuperGLUE, and multilingual tasks compared to larger encoders. The architecture is derived from the LFM2 family and involves converting causal decoders into bidirectional encoders using a bidirectional attention mask, non-causal short convolutions, and masked language modeling during training.
The models are trained in two stages: general language competence using a short context objective and long-context adaptation to 8,192 tokens to improve factual and multilingual skills. Performance benchmarks indicate that the LFM2.5-Encoder-350M ranks fourth among fourteen tested models across seventeen tasks from GLUE, SuperGLUE, and multilingual classification. Inference speed is significantly faster than ModernBERT-base on CPU, demonstrating approximately 3.7 times faster forward passes at 8,192 tokens.
The practical applications include building tools for high-volume understanding tasks like intent routers, policy linters, and PII detectors, running efficiently on standard CPU hardware. The models can be fine-tuned for specific downstream tasks by loading the encoder body and attaching custom heads.
Full Take
The innovation lies in decoupling the context length from computational cost by leveraging the LFM2 architecture's property where cost grows slowly with input size. This suggests a paradigm shift where the efficiency of long-context understanding is not just about scaling up parameter counts, but about architectural design—specifically transforming causal models into bidirectional encoders using lightweight modifications like non-causal convolutions and masked language modeling.
The pattern observed across benchmarks, particularly regarding CPU performance where the LFM2.5 models significantly outperform competitors while managing extended context, suggests a potential opportunity for deploying complex NLP tasks that require deep contextual reasoning on local hardware without relying solely on expensive GPU resources. The focus shifts from maximizing raw parameter count to optimizing the information flow within the model for efficiency in real-world, long-document applications.
The implication is that achieving high performance on extremely long contexts does not inherently require brute-force scaling, but rather a principled method for incorporating global context efficiently. This prompts questions about whether similar structural modifications can be systematically applied across other transformer families to unlock comparable efficiency gains for specific domains like legal or multilingual processing. What are the broader constraints on adapting this context-aware design principle to models where the cost scaling is less predictable?
Sentinel — Human
This appears to be a technically detailed announcement or blog post describing the architecture, performance, and usage of new NLP encoders, written in a style consistent with developer communication.