Small Language Models: Why Smaller Can Be Better for Interpreting

When artificial intelligence is reported on, the focus is often on superlatives: ever more parameters, ever more training data, ever more computing power. This path has enabled impressive progress. For simultaneous interpreting at events, however, a different and very practical question arises: how much AI performance is actually needed? This is exactly where small language models come into play. In this article, we explain why specialised and specifically optimised models are of interest for interpreting systems, which techniques lie behind them – and where their limits are.

Bigger is not automatically better

The development of artificial intelligence was long shaped by a simple idea: more parameters, more training data and more computing power lead to more capable models. This relationship is not wrong – it has produced many of the capabilities that seem almost taken for granted today.

For many practical applications, however, it falls short. Anyone using a technology in everyday work does not only ask what a model can do in principle. What matters is whether it performs a specific task reliably – under the conditions that actually apply. At an event, that means: with the equipment available on site, and while the programme is running.

Specialisation instead of maximum size

A universal language model is a generalist. It is expected to write code, compose texts, solve mathematical problems and answer questions on any topic. This breadth comes at a price: a model that is supposed to do everything has to combine a great many capabilities.

A model for speech recognition or translation, by contrast, has a clearly defined job. It is meant to recognise spoken language or transfer a sentence from one language into another. Program code or mathematical tasks are not part of this. Such a specialised model therefore does not necessarily need the same range of capabilities as an all-rounder.

Through targeted optimisation, smaller models can be very powerful for specific tasks. An analogy from the workshop: for a particular screw, you do not need the entire toolbox, just the right spanner.

How small language models become more efficient

There are various methods for making models leaner. Two of them play a particularly important role in current development: quantisation and knowledge distillation.

Quantisation: less precision, less memory

At its core, a language model consists of a very large number of numerical values, known as model weights. In quantisation, these weights are stored and processed with lower numerical precision – put simply: with more coarsely graded numbers instead of very finely resolved ones.

This can significantly reduce a model's memory requirements and thus lower the demands on the hardware.

The method does have a downside, though. Depending on the method and the model, some accuracy may be lost in the process. How large this loss turns out to be cannot be stated in general terms. The rule is therefore: after every optimisation, quality must be checked again. A model that needs less memory but no longer reliably recognises technical terms would be no improvement for a specialist conference.

Knowledge distillation: from a large to a small model

A second approach is what is known as knowledge distillation. Here, a smaller model is trained with the help of a larger model. The large model takes on the role of the teacher, so to speak, and the smaller one that of the student.

The aim is to make certain capabilities of the large model usable in a more efficient architecture. The small model therefore does not have to master everything its role model can do – but above all what it will later be used for.

Why this is relevant for simultaneous interpreting

Three tasks sharing the resources

In professional interpreting systems, it is not a single model at work; instead, several tasks have to be carried out at the same time. We have described in more detail how these steps traditionally work together in our article Can AI translate emotions?:

• Speech recognition: what is spoken is recognised.
• Translation: the recognised content is transferred into the target language.
• Speech synthesis: a synthetic voice delivers the translation.

All three steps compete for the same resources: computing power and memory. And not just once, but continuously – for as long as people are speaking on stage.

What more efficient models can make possible

More efficient models can therefore offer a considerable practical advantage. If each individual task uses fewer resources, headroom is created. This headroom may make possible:

• more parallel language channels, for example when guests follow an event in several target languages,
• lower hardware requirements at the venue,
• shorter processing times, meaning less delay between the spoken original and the translation.

The word “may” has been chosen deliberately. Which advantage actually materialises in practice depends on the particular model, the optimisation method and the overall system – and, as with quantisation, must be checked every time.

Local AI: why efficiency is not a theoretical question for us

At MILA, AI simultaneous interpreting runs locally on our own equipment at the venue – without the cloud. What is said does not leave the event. For companies, associations and public institutions whose events also involve confidential topics, this can be an important point.

Local operation, however, also means that processing takes place on the equipment we bring with us. That is why the question of how much performance a model needs for its task is not merely of academic interest to us.

For you as an organiser, the process remains straightforward: each language is fed on its own channel into the existing interpretation system or conference equipment, and on request we provide receivers and headphones for the guests. Anyone looking for an interpreter for events therefore does not need to deal with model architectures – we take care of that.

The MILA perspective: size is not a quality criterion

For MILA, the size of a language model is not what matters. Four criteria are relevant:

• Translation quality: does the content arrive in the target language clearly and correctly?
• Latency: how much time passes between the original and the translation?
• Stability: does the system run reliably throughout the entire event?
• Resource consumption: how much computing power and memory is required?

We therefore pay particular attention to the development of efficient models and local inference technologies – that is, the methods by which models are run directly on one's own hardware.

Just as important is what can be planned independently of the model: we set up technical terms and proper names from the programme in advance, and a system technician is always on site. They plan with you, set up, look after the interpreting during the event and dismantle everything afterwards. And because even the best model can only translate what it hears, sound plays an equally important role – more on this in our article AI interpreting and audio technology: why everything starts at the microphone.

Conclusion: the right AI rather than the biggest

Small language models are not an end in themselves. Quantisation and knowledge distillation can make models leaner, but every optimisation must be measured against quality. For simultaneous interpreting at congresses, specialist conferences or corporate events, what ultimately counts is what reaches the audience in the room.

For professional applications, the most exciting AI is not always the biggest – but the one that reliably does its job.

If you are planning interpreting for your next event, you will find all the details on equipment, languages and process on our Services page. We have compiled answers to frequently asked questions in our FAQ.