Language Models

They are text synthesis engines

They are a mathematical model of the distribution of various text tokens—words, syllables, punctuation—across a large collection of writing. Their functionality is based on generating writing that’s a plausible representation of this model and the provided prompts.

They do not “learn” or get “inspired”

Every time a language model needs to learn something genuinely new it needs to be rebuilt from scratch, using the entirety of its training data set. The human equivalent would be if we had to regrow our brains and relive our entire lives every time we wanted to learn something or get inspired.

They do not reason

Their “thinking” is entirely based on finding correlations between texts. They only “reason” through replaying and combining records of reasoning as described in writing. This makes their “reasoning” extremely brittle and prone to breaking when the prompt is reworded or rephrased.

A language model only hallucinates

The output is all hallucination. But occasionally the hallucinations have factual elements. Fabricated text is the rule, not the exception.

They are great at pure textual tasks

These models lend themselves to tasks that require converting or modifying text, such as turning a casual email into a formal letter. That includes many software development tasks.

Language models are bad at tasks that require any reasoning or empathy

Plotting. Textual structure. Customer feedback. Therapy. Anything that requires a consistent structure or an understanding of people or the world is going to be an ineffective use of a language model.

Language models are unreliable

They can easily generate unsafe output. They are hard to secure. Their “reasoning” is very error-prone. Their advice can be harmful. All of their output is fabricated.

Larger models aren’t necessarily better

Language model “reasoning” and textual fluency seems to improve with model size, but many other factors degrade. Smaller, open source models may well suit your requirements better.

Larger training data sets are undocumented

This increases your risk as you have no reliable way of forming a clear picture of what sort of range of behaviours the language model offers. It makes it hard to assess its biases, tendency towards unsafe output, or your overall legal liability. Smaller, open source models may well suit your requirements better.

Language models are very labour-intensive

Modern language models require an enormous amount of cheap human labour to filter the training data and fine-tune the models themselves.

Language models are complex

Refer to the other cards in this set for further information and references on many of the specific issues with language models.

Further reading

• What Will Transformers Transform?
• The Fallacy of AI Functionality
• On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?
• For more, see, well, most of The Intelligence Illusion

References →