A small language model is a transformer-based language model, generally under about 15 billion parameters, small enough to run on a single GPU, a workstation, or edge and mobile hardware rather than a multi-GPU cluster.
The contrast with LLMs is concrete. Large models (100B+ parameters, such as GPT-5 or Gemini) win on open-ended reasoning and broad general knowledge. SLMs win on latency, cost per query, and data privacy, because inference can happen on-device or on-prem without sending data to an external cloud. This is the heart of the slm vs llm trade-off.
The numbers make the gap tangible. Per NVIDIA, running a small Llama 3.1B can be 10-30 times cheaper than Llama 3.3 405B, depending on architecture and the typical query.
Small language models vs large language models
