When should a company train a small language model?
When the task is narrow and repeated, when data cannot leave your environment, or when call volume makes per-request cost decisive. ACL 2023 research showed a fine-tuned 770M model beating a 540B model on the task it was trained for. Decide by benchmarking on your own task, not a leaderboard.
The default has become an API call to the largest available model. That default is right for exploration and wrong for a surprising share of production work. The question is not which model is best in general — it is the smallest model that does your task well, where your data must live, at a cost you can run every day.
When a small model wins
Three situations favour training or fine-tuning a small language model. First, the task is narrow and repeated: classifying documents, extracting fields, answering questions over one domain, drafting one kind of text. Research published at ACL 2023 by Hsieh and colleagues showed a fine-tuned 770-million-parameter model outperforming a few-shot-prompted 540-billion-parameter model on a benchmark task — roughly seven hundred times smaller, better on the job it was trained for. Second, the data cannot leave: fine-tuning methods such as QLoRA, presented at NeurIPS 2023, adapt a 65-billion-parameter model on a single 48GB GPU, which puts private training inside one machine an organisation controls. Third, volume: when a call runs millions of times, the gap between a hosted frontier model and a small self-hosted one becomes a budget line.
When it loses
A small model is the wrong answer for open-ended reasoning across many domains, for tasks that change faster than you can retrain, and for products still searching for what the task even is. Early on, the flexibility of a large general model is worth its price. The time to specialise is after the task has stabilised and the volume is known.
The deciding test
Do not decide on principle; decide on a benchmark. Take a few hundred real examples of your task, agree what a correct output looks like, and run the candidates — the large API, an adapted small model, and whatever you use today — against the same set. Domain adaptation is well-evidenced: ACL 2020 research by Gururangan and colleagues found that continued pretraining on domain text improved performance across every domain and task they tested. But the evidence that matters is performance on your task, measured before commitment, and re-measured on every release afterwards.
Related questions
Can a small model really match a frontier model?
On one narrow task it was trained for, yes — peer-reviewed work at ACL 2023 showed a fine-tuned 770M-parameter model outperforming a few-shot-prompted 540B-parameter model on a benchmark. On broad, open-ended work, no.
What hardware does private fine-tuning need?
Less than most teams assume. QLoRA, published at NeurIPS 2023, fine-tunes a 65B-parameter model on a single 48GB GPU while preserving full fine-tuning performance — hardware an organisation can own and keep inside its network.
How do we know when our task is ready for a small model?
When the task definition has stopped changing and you can assemble a few hundred real examples with agreed correct outputs. That set becomes both the training signal and the benchmark for the buy-or-train decision.
Model Training & SLMs
Train and fine-tune small language models on your data, for your domain and your infrastructure.
Discuss your requirement