Analogies for a Large Language Model

How to think of them (not how they think)

Why Spend Time On Analogies?

Technology changes relentlessly. We who work with technology must bear the curse of new things on a daily (if not hourly) basis. So how are we, as technologists, supposed to keep up with the relentless pace of change? And, especially if one is a generalist, how does one keep abreast of all that's going on?

I've found analogy to be a useful -- if imperfect -- tool for understanding new technology and also for communicating to others what a new technology is and isn't. As an example, I'll share three analogies I returned to regularly during the Large Language Models (LLMs) craze of 2023-2025(?). No one analogy is perfect, but by choosing the right analogy at the right time, I've been able to short-circuit some otherwise sticky misunderstandings about the nature of a mysterious new technology.

Why are these important? I've found many professional miscommunications happen because of a divergence between mental models of an underlying problem or solution. Analogies provide a quick short-circuit to determining how aligned one is with one's counterpart before divining into nitty-gritty.

Three Analogies For a Large Language Model

Over-eager Sixth Grade Essay-writer

We've all been here, and many of us have the recurring nightmare: it's the night before a 1000-word essay on the economy of Saskatchewan is due, the library is closed and all you have is your parents' outdated Encyclopaedia Britannica. (If this example, sounds oddly specific, there's a reason.) So what's an ambitious sixth-grader to do? Got it in one -- pad the essay with as much fluff as you can.

Large language models behave similarly. They'll create as much verbiage as asked for to sound good, but really they're bs-ing based on an underlying (extremely complex) statistical model. So, the answer sounds good; it might even be right if the gods of probabilities are in our favor; but there's really no way to be sure other than independent factual verification. (More on that later...)

Lossy Compression Database

Remember the earlier stages of the Internet when bandwidth was important and so creators would spend time agonizing over the perfect compression ratio for a video stream or a JPEG image? Over-compress, and the playback of the image would be blotchy and pixelated. Under-compress, and you'd waste bandwidth for no significant gain in fidelity. (Or maybe you were listening to really raspy MP3s...)

In some cases, it's useful to think of LLMs has highly compressed databases of document data, but compressed with a lossy compression algorithm. The same way image or audio compression can replay a "close-enough" approximation of a picture or sound, the LLM replays a close-enough sequence of "tokens" (aka words) to be useful. The key is to remember the sequence is good, and depending on the application may be good enough. (A "gentleman's C" for our Sixth Grader.)

Markov Chain

This one's a bit nerdier. Way back when I was an undergraduate, one Artificial Intelligence implementation technique we looked at was the use of Markov Chains to create a predictive text engine. Our implementation was naively simple, just looking at a set of words that had come before and using it to predict the next word by estimating likelihood based on previously loaded text. But when I started reading up on Large Language Models, I recognized that they were essentially this implementation at scale and with an emergent algorithm for determining the next token. Essentially my old textbook exercise with a boatload of money, computing power and data (and several Ph.D.'s) thrown at it.

How RAG and Tools Change Things

"It's not what you know, it's knowing who to ask."

As a generalist, I can't know everything. What I can do is have enough context to understand what I do and don't know and how to find out more of what I need to. AI operationalization using external knowledge bases (accessed through a variety of mechanisms -- we won't get into that here) supports the eager sixth grader by providing more verified information to shape a reply. This information is usually conveyed to a final model in a chain via a context prompt, which is why "prompt engineering" and “context engineering” are recent hot topics in AI circles. Here, finally, the model has a cheat sheet of verified truth to work with, rather than a hazy statistical memory of what might have been.

The Near Future

LLM capabilities continue to mature, as does our operational knowledge about implementing LLM-based solutions. In-model filters for hallucination, specialized tasks (e.g. math, time, space) and traceability will continue to improve their usefulness, but at no point in the foreseeable future will we truly understand how a response to a particular prompt was arrived at. Thinking of LLMs imperfectly keeps us aware of their imperfections and creates a useful framework for managing thier risks.

Postscript: A Bonus Analogy About Managing GPUs

While planning GPU consumption for a project at my last engagement, I have a flashing insight that I was back in the 1990s planning mainframe processor consumption across a batch cycle, including optimizing for maximum utilization rather than just execution speed. There are only so many different resource-constraint problems out there in operations science.

Image Credit: AI SD3.5 running on Invoke locally.