Commonly Used AI Terms
Summary
Important conversations like the one about paying for AI, versus running it at home for (sort of) free, hinge on a handful of terms. You can’t assess the arguments for and against without understanding the jargon.
A large language model predicts the next word, and its weights are the billions of numbers that enable it to do just that. Open weight labs such as Alibaba, DeepSeek and Mistral publish those numbers, while Anthropic keep theirs private.
Quantisation squeezes a model onto home hardware at some cost in quality, and batching is why a shared data centre answers more cheaply than a desktop.
These are the words that get thrown about whenever the “I do it all at home for free” crowd argue with the people who pay Anthropic. They are explained in the order you need them, because each one leads to the next.
LLM
LLM stands for Large Language Model. It is a program that has read loads of text and, in doing this, has learned to predict the next word because it’s heard everything anyone has ever said before. From that single trick, an LLM can write, summarise, translate, deduce, solve problems and produce code.
When people say “model” in an AI conversation, an LLM is nearly always what they mean. But do note the difference between the model and the product built around it.
ChatGPT and Claude are both products: they are a chat window, a login, a subscription, some personalisation and a few safety rules, wrapped round the LLM.
The models underneath have their own names, and those names change as new versions are released: GPT-6 Astra at OpenAI and Claude Fable 5.1 and Opus 5.5 at Anthropic, for example. And of course the product can swap the model underneath without you really noticing.
Model Weights
A model has two main parts. The architecture is the design, a few lines of code that are much the same at every lab and freely published in research papers. But the weights are the numbers that complete it. There are literally billions of these numbers, each a small decimal, arrived at over months of the model pigging out on text. There is constant number tweaking too, until its guesses at the next word improve.
The learning program runs for ages on thousands of specialised chips which costs hundreds of millions of dollars. Everything the model knows ends up in those numbers, which is why people say weights when they mean model, and why a model is sometimes described as one enormous file. The architecture is the shell – or rather the biscuit tin. The weights are the biscuits. The biscuits are what you pay for.
Training and Inference
Training is how the weights get made. It is the months of computing time described above. But it happens only once, at the lab, and the resulting appearance of knowledge is far beyond anything an individual could ever hope to learn.
Inference is using the finished weights to answer a question. Every time you type into a chat window and get a reply, that’s inference. Outside the labs, using a model is all anyone ever does. So when people argue about what AI costs, or what hardware it needs, they mean the cost of running a model, not of making one.
Fine tuning sits in between. You take finished weights and train them a little further on a narrow set of material, to make a general model better at one job. It’s cheaper than training from scratch, but it still needs serious hardware and someone who knows what they’re doing.
Frontier Model
A frontier model is one of the best models anyone has made so far: the newest and most capable releases from the big labs, such as Claude Fable 5.1 from Anthropic, GPT-6 Astra from OpenAI, or Google’s latest Gemini.
The frontier is simply the furthest point anyone has reached, and it moves every few months as each lab tries to outdo the others.
Open and Closed Weights
When a lab publishes its weights so that anyone can download them, the model is called open weight.
The Chinese labs mostly do this: Kimi from Moonshot, Qwen from Alibaba, DeepSeek, and GLM from Zhipu. Mistral in Paris does it too, and so does Meta, with Llama.
Anthropic keeps its weights to itself. OpenAI has released a couple of smaller open weight models, but not the ones behind ChatGPT. So the only way to use Claude, or the models ChatGPT runs on, is through their servers, paid for by subscription or by the token.
The free models you can download and run yourself are never the frontier models that Anthropic and OpenAI sell.
The best open weight models tend to trail a year or so behind, so the “I built my own LLM” crowd are usually running something close to last year’s best, which of course they did not actually build. That’s what the whole “why pay for AI” argument comes down to.
It helps to notice what the open weight labs give away and what they keep. You get the finished numbers, which are the biscuits. You don’t get the training data, the training code or the process that produced the numbers, which is the recipe, and the recipe is what cost them hundreds of millions.
You can scoff as many biscuits as you like and still not be able to bake the next batch. People often call these models open source, but strictly that’s wrong. Open source would mean publishing the recipe too, and almost nobody does.
Why even share the biscuits?
They’re not giving away access out of kindness. It costs nothing to allow people to copy a file. It’s because the Chinese labs find they can’t win the AI race. This is mostly due to US export controls preventing the best chips from going to China. That made the Chinese go the free route in order to disrupt the AI market. Some European and American labs then followed for their own reasons.
A free model that the whole world runs eats into what the big labs like OpenAI and Anthropic can charge without the Chinese labs having to out-code them, and the free labs also benefit. Alibaba gives Qwen away and sells the cloud computing to run it on, a bit like Google gave search away to sell advertising.
For US cloud giants, open-weight models are loss leaders designed to drive consumption of paid cloud computing or industry intelligence. You see Google, Meta and Microsoft also give away small models. While the models are free to download, most developers and enterprises don’t have the hardware to deploy them at scale. So, they host them on Google Cloud Platform (GCP) or Microsoft Azure, generating massive, recurring cloud hosting revenue for the providers.
When Meta gives their AI away, thousands of smart programmers around the world start tweaking and improving it. Meta can then take all those free ideas and use them to make apps like Instagram and WhatsApp better.
The free access makes things a bit more challenging for rivals like OpenAI and Anthropic, who rely on charging people to use their offerings.
Parameters
Parameters is an indication of how clever a model is. It is an approximate count of weights. A small model has a few billion weights. The biggest open models you can actually download run to a trillion or more. The frontier labs keep their counts secret, but the best guesses put their models over a trillion. GPT-4, back in 2023, was reported to have about 1.8 trillion weights or as is said, parameters. More parameters usually means more knowledge and better judgement, as well as more memory required to run it.
Most large models are now built as a mixture of individual experts. The LLM is really a committee of smaller topic specialists, and only a few of them work on any given question. That saves effort while answering, but all experts have to be loaded into memory first.
So if you choose to run your own model, the idle experts cost you hardware whether they’re working or not. You’ll see an indication of this on the download pages of open weight models.
A model described simply as “27B” has 27 billion parameters, and all of them work on every word of every question. A mixture of experts gets two figures, such as “1T, 32B active”: a trillion parameters in total, of which only 32 billion wake up for each word.
Quantisation
Quantisation is a process of storing the weights less precisely, so the resulting file is smaller and fits on smaller hardware.
A weight stored in sixteen bits takes twice the memory of one stored in eight, and four times that of one stored in four, but the model still works because the numbers only need to be more or less correct.
Each step down does cost some quality, and it will show up more obviously during deep reasoning and long documents rather than in casual chat. Nearly all AI run on home hardware is quantised. The enthusiasts will tell you the loss is barely noticeable, and on easy tasks they’re probably right.
GPU and VRAM
A model runs on a chip called a GPU, and the memory attached to that chip, the VRAM, decides which models you can run and how fast. GPU stands for graphics processing unit, because graphic computations were the chip’s original job and the name has stuck.
Bringing a video game to life means doing the same tiny maths on millions of pixels at once, so graphics chips were built as thousands of small calculators working side by side.
Running a language model just happened to require the same sort of arithmetic featuring billions of little multiplications, except in the case of LLMs, it had nothing to do with gaming or graphics. That’s how the gaming chip became the AI chip by accident, and Nvidia, which made the best ones, became one of the most valuable companies on earth.
When it comes to graphics chips, two numbers matter. The first is how much VRAM there is, because the whole model has to fit into it. A good consumer GPU has 24 to 32 gigabytes. A data centre GPU has 80 or more and costs more than a car.
The second is how fast that memory can be read, because for one person the speed of the answer depends on how quickly the chip can pull every weight through memory for every word it writes. That’s why cheap hardware runs models slowly even when they fit, and why wiring several cheap boards together just gets you several slow machines taking turns.
A newer kind of machine, such as the Mac Studio, AMD’s Ryzen AI Max or Nvidia’s DGX Spark, shares one pool of memory between the main processor and the GPU, so you can have 128 gigabytes in a box on your desk for somewhere between two and ten thousand euros.
Those boxes are what made running a serious model at home possible at all. The biggest models are another matter. They need hundreds of thousands of euros of hardware and a three phase electricity supply. So it’s fair to say nobody runs those at home.
Tokens and Tokens Per Second
Tokens are the units a model reads and writes. A token is a piece of a word, on average about three quarters of one in English, so a thousand words is roughly 1,300 tokens. For example the word unbelievable will be three tokens, “un”, “believ” and “able”.
Everything is measured and priced in tokens. What you send the model counts as input tokens, what it sends back counts as output tokens. Output costs several times more than input because it takes more work to produce.
Tokens per second show how fast the AI can generate an answer. Anything under twenty tokens per second feels far too slow.
Hosted frontier cloud models usually output fifty to a hundred tokens per second or more.
People running models locally at home may give up and go back to cloud services when their own machine drops much below that speed.
Context Window
The context window is the amount of text a model can hold in its head at once, measured in tokens, counting both what you’ve sent and what it has sent back as a reply.
When a conversation or a document runs past the window, the oldest material falls out and the model no longer knows about it.
Modern hosted models have windows of hundreds of thousands of tokens, enough for several books.
Models run on home hardware usually make do with far less, because a big window needs a lot of memory. It’s one of the reasons a local model feels less intelligent than a hosted one on long pieces of work.
Batching
Data centres have a massive efficiency trick that home computers can’t match, known as batching.
Every time an AI generates a single word, the computer has to empty the AI’s entire knowledge base into its memory chips. A home computer can do all the heavy lifting for just one person.
But a data centre’s computing power can handle hundreds of different people’s questions in a single pass.
It’s a bit like a bus using slightly more fuel than a car, but carrying fifty people instead of one.
Because that huge workload is shared, a 700-watt data centre chip answering hundreds of people actually burns far less electricity per response than a 180-watt home computer serving just you.
Subscription and API
There are two ways of accessing the same models. The subscription is the flat rate method. You get to use the LLM for a fixed monthly fee with a usage limit that resets on a timer. Individuals and small teams use it, and it’s subsidised, in that you can use far more computing than the fee would buy at metered rates.
The API, (application programming interface), is the pay per token method. This tends to be used by companies and anyone building software that requires calls to the model. There’s no weekly allowance, only rate limits, and the bill grows with every token in and out.
There is in fact another way to access various models. You can use marketplaces like OpenRouter. While OpenRouter connects you to both closed and open models, the real budget trick is to use pay-as-you-go access to open-weight models. So you get access to them without having to host anything yourself, and you benefit from free models hosted on fast data-center GPUs.
Because there are no fees for the model itself, you only pay for the raw server time which is the cheapest way of all to get your software running using AI with fast response times.
Harness and Agent
A harness is the software built around an AI model. On its own, a raw model answers one prompt and stops working. The harness gives it a way to operate by supplying tools it can use, a memory it can read and write to, specific instructions to follow, and safety rules to obey.
An agent is what you get when you put a model inside a harness and give it a high level goal. Instead of waiting for a human to type out every single step, an agent can read files, run code, search the web, double-check its own work, and keep working away at a complex project until it determines the job is done. Tools like Claude Code and its competitors are prime examples of AI agents.
Why the Harness Matters
Much of the recent surge in AI capability comes from building better harnesses, not just bigger, smarter models. In practice, a modest model operating inside a great harness will often beat a top-tier model struggling in a poor one.
This is the strongest argument for running AI on home hardware. As long as the system can automatically check its own work, like running a script to test whether generated code actually works, a well-written harness allows lightweight, local models to punch far above their weight.
RAG
RAG stands for Retrieval Augmented Generation. It means finding the relevant passages in your own documents and handing them to the model along with your question, so that the model can answer using your material rather than from whatever it happens to remember or look up online.
RAG is how a model can answer questions about your invoices, your contracts or your teaching notes without ever having been trained on them. It’s also the usual reason a business wants a model that can see its private data, and that in turn is the usual reason it thinks about running one on its own premises.







