Skip to main content Skip to footer
Opens in a new tab
Building, Testing, Writing
  • Everything on the site, in one list. Tap a line to go there.
  • About
  • Stack

      What I build with

      Every tool behind this site, why it is here, and what I dropped to make room for it.

      Plugins

      The plugins I install on most sites

      WordPress & Bricks

      The platform and the builder behind every site I make.

      Claude

      My partner for specs, code and checking the work.

  • Projects

      Things I am building

      The tools and experiments I write about. Some are live, some are on the bench.

      OLL! Tools

      Ready-made French lessons for teachers with a class on Monday and no time on Sunday.

      This Website

      My own site, built semi-automatically with Bricks, ACF and Claude Code

      OLL! Works

      An engine built to produce high-value content for OLL! Tools on a variety of topics.

      Jam Web Design

      A prompt builder for Claude Design in 40 Flavours

  • Blog Latest Posts

      Writing and notes

      What I am building, what went wrong, and what I learned on the way. The newest pieces first.

      Commonly Used AI Terms

      AI Glossary

      These are the words that get thrown about whenever the "I do it all at home for free" crowd argue…

      Notebook

      How To Get Paid on Facebook

      Get Paid on Facebook

      Facebook pays in three ways, and you can plan for two of them. What to reach, in what order.

      Notebook

      Folders Matter To Claude Code

      Claude's Filing System

      Every rule written once, in a flat folder structure. How I set up my projects for Claude Code.

      Notebook

      The Magic Postie Elf

      Postal Elves

      If I said the words "Fire Saga", would you know what I was talking about? I'm referring to the film…

      Life

      Apparently It’s Not Called Vibe Coding

      Am I Vibe Coding?

      Did you know that if you do vibe coding properly, you are no longer allowed to call it vibe coding?…

      Notebook

      Should I Use Free LLMs?

      Should I Use Free LLMs?

      If you work alone and use AI every day, you may have wondered about the bloke with his home-grown AI,…

      Notebook
  • Contact Us
  • Search

    Search the site

Contact Us
Weights, tokens and the rest, in plain English
Last Modified: 23 October 2026
  • Notebook

Commonly Used AI Terms

Summary

Important conversations like the one about paying for AI, versus running it at home for (sort of) free, hinge on a handful of terms. You can’t assess the arguments for and against without understanding the jargon.

A large language model predicts the next word, and its weights are the billions of numbers that enable it to do just that. Open weight labs such as Alibaba, DeepSeek and Mistral publish those numbers, while Anthropic keep theirs private.

Quantisation squeezes a model onto home hardware at some cost in quality, and batching is why a shared data centre answers more cheaply than a desktop.

These are the words that get thrown about whenever the “I do it all at home for free” crowd argue with the people who pay Anthropic. They are explained in the order you need them, because each one leads to the next.

LLM

LLM stands for Large Language Model. It is a program that has read loads of text and, in doing this, has learned to predict the next word because it’s heard everything anyone has ever said before. From that single trick, an LLM can write, summarise, translate, deduce, solve problems and produce code.

When people say “model” in an AI conversation, an LLM is nearly always what they mean. But do note the difference between the model and the product built around it.

ChatGPT and Claude are both products: they are a chat window, a login, a subscription, some personalisation and a few safety rules, wrapped round the LLM.

The models underneath have their own names, and those names change as new versions are released: GPT-6 Astra at OpenAI and Claude Fable 5.1 and Opus 5.5 at Anthropic, for example. And of course the product can swap the model underneath without you really noticing.

Model Weights

Infographic explaining AI model weights value in four sections

A model has two main parts. The architecture is the design, a few lines of code that are much the same at every lab and freely published in research papers. But the weights are the numbers that complete it. There are literally billions of these numbers, each a small decimal, arrived at over months of the model pigging out on text. There is constant number tweaking too, until its guesses at the next word improve.

The learning program runs for ages on thousands of specialised chips which costs hundreds of millions of dollars. Everything the model knows ends up in those numbers, which is why people say weights when they mean model, and why a model is sometimes described as one enormous file. The architecture is the shell – or rather the biscuit tin. The weights are the biscuits. The biscuits are what you pay for.

Training and Inference

Training is how the weights get made. It is the months of computing time described above. But it happens only once, at the lab, and the resulting appearance of knowledge is far beyond anything an individual could ever hope to learn.

Inference is using the finished weights to answer a question. Every time you type into a chat window and get a reply, that’s inference. Outside the labs, using a model is all anyone ever does. So when people argue about what AI costs, or what hardware it needs, they mean the cost of running a model, not of making one.

Fine tuning sits in between. You take finished weights and train them a little further on a narrow set of material, to make a general model better at one job. It’s cheaper than training from scratch, but it still needs serious hardware and someone who knows what they’re doing.

Frontier Model

A frontier model is one of the best models anyone has made so far: the newest and most capable releases from the big labs, such as Claude Fable 5.1 from Anthropic, GPT-6 Astra from OpenAI, or Google’s latest Gemini.

The frontier is simply the furthest point anyone has reached, and it moves every few months as each lab tries to outdo the others.

Open and Closed Weights

Infographic comparing open weight vs closed weight AI models

When a lab publishes its weights so that anyone can download them, the model is called open weight.

The Chinese labs mostly do this: Kimi from Moonshot, Qwen from Alibaba, DeepSeek, and GLM from Zhipu. Mistral in Paris does it too, and so does Meta, with Llama.

Anthropic keeps its weights to itself. OpenAI has released a couple of smaller open weight models, but not the ones behind ChatGPT. So the only way to use Claude, or the models ChatGPT runs on, is through their servers, paid for by subscription or by the token.

The free models you can download and run yourself are never the frontier models that Anthropic and OpenAI sell.

The best open weight models tend to trail a year or so behind, so the “I built my own LLM” crowd are usually running something close to last year’s best, which of course they did not actually build. That’s what the whole “why pay for AI” argument comes down to.

It helps to notice what the open weight labs give away and what they keep. You get the finished numbers, which are the biscuits. You don’t get the training data, the training code or the process that produced the numbers, which is the recipe, and the recipe is what cost them hundreds of millions.

You can scoff as many biscuits as you like and still not be able to bake the next batch. People often call these models open source, but strictly that’s wrong. Open source would mean publishing the recipe too, and almost nobody does.

Why even share the biscuits?

They’re not giving away access out of kindness. It costs nothing to allow people to copy a file. It’s because the Chinese labs find they can’t win the AI race. This is mostly due to US export controls preventing the best chips from going to China. That made the Chinese go the free route in order to disrupt the AI market. Some European and American labs then followed for their own reasons.

A free model that the whole world runs eats into what the big labs like OpenAI and Anthropic can charge without the Chinese labs having to out-code them, and the free labs also benefit. Alibaba gives Qwen away and sells the cloud computing to run it on, a bit like Google gave search away to sell advertising.

For US cloud giants, open-weight models are loss leaders designed to drive consumption of paid cloud computing or industry intelligence. You see Google, Meta and Microsoft also give away small models. While the models are free to download, most developers and enterprises don’t have the hardware to deploy them at scale. So, they host them on Google Cloud Platform (GCP) or Microsoft Azure, generating massive, recurring cloud hosting revenue for the providers.

When Meta gives their AI away, thousands of smart programmers around the world start tweaking and improving it. Meta can then take all those free ideas and use them to make apps like Instagram and WhatsApp better.

The free access makes things a bit more challenging for rivals like OpenAI and Anthropic, who rely on charging people to use their offerings.

Parameters

Infographic explaining AI parameters, model sizing and impact

Parameters is an indication of how clever a model is. It is an approximate count of weights. A small model has a few billion weights. The biggest open models you can actually download run to a trillion or more. The frontier labs keep their counts secret, but the best guesses put their models over a trillion. GPT-4, back in 2023, was reported to have about 1.8 trillion weights or as is said, parameters. More parameters usually means more knowledge and better judgement, as well as more memory required to run it.

Most large models are now built as a mixture of individual experts. The LLM is really a committee of smaller topic specialists, and only a few of them work on any given question. That saves effort while answering, but all experts have to be loaded into memory first.

So if you choose to run your own model, the idle experts cost you hardware whether they’re working or not. You’ll see an indication of this on the download pages of open weight models.

A model described simply as “27B” has 27 billion parameters, and all of them work on every word of every question. A mixture of experts gets two figures, such as “1T, 32B active”: a trillion parameters in total, of which only 32 billion wake up for each word.

Quantisation

Infographic explaining AI quantisation for small hardware devices

Quantisation is a process of storing the weights less precisely, so the resulting file is smaller and fits on smaller hardware.

A weight stored in sixteen bits takes twice the memory of one stored in eight, and four times that of one stored in four, but the model still works because the numbers only need to be more or less correct.

Each step down does cost some quality, and it will show up more obviously during deep reasoning and long documents rather than in casual chat. Nearly all AI run on home hardware is quantised. The enthusiasts will tell you the loss is barely noticeable, and on easy tasks they’re probably right.

GPU and VRAM

A model runs on a chip called a GPU, and the memory attached to that chip, the VRAM, decides which models you can run and how fast. GPU stands for graphics processing unit, because graphic computations were the chip’s original job and the name has stuck.

Bringing a video game to life means doing the same tiny maths on millions of pixels at once, so graphics chips were built as thousands of small calculators working side by side.

Running a language model just happened to require the same sort of arithmetic featuring billions of little multiplications, except in the case of LLMs, it had nothing to do with gaming or graphics. That’s how the gaming chip became the AI chip by accident, and Nvidia, which made the best ones, became one of the most valuable companies on earth.

When it comes to graphics chips, two numbers matter. The first is how much VRAM there is, because the whole model has to fit into it. A good consumer GPU has 24 to 32 gigabytes. A data centre GPU has 80 or more and costs more than a car.

The second is how fast that memory can be read, because for one person the speed of the answer depends on how quickly the chip can pull every weight through memory for every word it writes. That’s why cheap hardware runs models slowly even when they fit, and why wiring several cheap boards together just gets you several slow machines taking turns.

A newer kind of machine, such as the Mac Studio, AMD’s Ryzen AI Max or Nvidia’s DGX Spark, shares one pool of memory between the main processor and the GPU, so you can have 128 gigabytes in a box on your desk for somewhere between two and ten thousand euros.

Those boxes are what made running a serious model at home possible at all. The biggest models are another matter. They need hundreds of thousands of euros of hardware and a three phase electricity supply. So it’s fair to say nobody runs those at home.

Tokens and Tokens Per Second

Infographic explaining AI tokens, pricing, and processing speed

Tokens are the units a model reads and writes. A token is a piece of a word, on average about three quarters of one in English, so a thousand words is roughly 1,300 tokens. For example the word unbelievable will be three tokens, “un”, “believ” and “able”.

Everything is measured and priced in tokens. What you send the model counts as input tokens, what it sends back counts as output tokens. Output costs several times more than input because it takes more work to produce.

Tokens per second show how fast the AI can generate an answer. Anything under twenty tokens per second feels far too slow.

Hosted frontier cloud models usually output fifty to a hundred tokens per second or more.

People running models locally at home may give up and go back to cloud services when their own machine drops much below that speed.

Context Window

Infographic explaining AI context window and memory limits

The context window is the amount of text a model can hold in its head at once, measured in tokens, counting both what you’ve sent and what it has sent back as a reply.

When a conversation or a document runs past the window, the oldest material falls out and the model no longer knows about it.

Modern hosted models have windows of hundreds of thousands of tokens, enough for several books.

Models run on home hardware usually make do with far less, because a big window needs a lot of memory. It’s one of the reasons a local model feels less intelligent than a hosted one on long pieces of work.

Batching

Infographic explaining AI batching efficiency in data centres

Data centres have a massive efficiency trick that home computers can’t match, known as batching.

Every time an AI generates a single word, the computer has to empty the AI’s entire knowledge base into its memory chips. A home computer can do all the heavy lifting for just one person.

But a data centre’s computing power can handle hundreds of different people’s questions in a single pass.

It’s a bit like a bus using slightly more fuel than a car, but carrying fifty people instead of one.

Because that huge workload is shared, a 700-watt data centre chip answering hundreds of people actually burns far less electricity per response than a 180-watt home computer serving just you.

Subscription and API

There are two ways of accessing the same models. The subscription is the flat rate method. You get to use the LLM for a fixed monthly fee with a usage limit that resets on a timer. Individuals and small teams use it, and it’s subsidised, in that you can use far more computing than the fee would buy at metered rates.

The API, (application programming interface), is the pay per token method. This tends to be used by companies and anyone building software that requires calls to the model. There’s no weekly allowance, only rate limits, and the bill grows with every token in and out.

There is in fact another way to access various models. You can use marketplaces like OpenRouter. While OpenRouter connects you to both closed and open models, the real budget trick is to use pay-as-you-go access to open-weight models. So you get access to them without having to host anything yourself, and you benefit from free models hosted on fast data-center GPUs.

Because there are no fees for the model itself, you only pay for the raw server time which is the cheapest way of all to get your software running using AI with fast response times.

Harness and Agent

A harness is the software built around an AI model. On its own, a raw model answers one prompt and stops working. The harness gives it a way to operate by supplying tools it can use, a memory it can read and write to, specific instructions to follow, and safety rules to obey.

An agent is what you get when you put a model inside a harness and give it a high level goal. Instead of waiting for a human to type out every single step, an agent can read files, run code, search the web, double-check its own work, and keep working away at a complex project until it determines the job is done. Tools like Claude Code and its competitors are prime examples of AI agents.

Why the Harness Matters

Much of the recent surge in AI capability comes from building better harnesses, not just bigger, smarter models. In practice, a modest model operating inside a great harness will often beat a top-tier model struggling in a poor one.

This is the strongest argument for running AI on home hardware. As long as the system can automatically check its own work, like running a script to test whether generated code actually works, a well-written harness allows lightweight, local models to punch far above their weight.

RAG

RAG stands for Retrieval Augmented Generation. It means finding the relevant passages in your own documents and handing them to the model along with your question, so that the model can answer using your material rather than from whatever it happens to remember or look up online.

RAG is how a model can answer questions about your invoices, your contracts or your teaching notes without ever having been trained on them. It’s also the usual reason a business wants a model that can see its private data, and that in turn is the usual reason it thinks about running one on its own premises.

Elizabeth Jamieson
Spec-Driven Developer

In this post

Related posts

  • Am I Vibe Coding?

About

My name is Elizabeth and I’m a software developer with over thirty years’ experience.

I rarely work with clients, preferring to collaborate with partners to build websites and web apps and explore new ideas.  But if you’re one of my old clients or if I really like you, I’ll work with you. Just ask.

I am located on the west coast of Scotland with Russell and my bilingual cockapoo, Raffa.

Navigate

Notebook

My blog for technical notes and opinion

Projects

What I’m working on now and what I’ve put live.

Life

About the events behind storylines.

Questions?

If you have questions, requests or a polite disagreement you are welcome to contact me.

Anything sent containing links, questionable cold outreach or dodgy niche vocabulary (you know what I’m talking about) will be spam filtered into oblivion.  I won’t even see it.

Contact Us
  • Legal Notice
  • Privacy Policy
  • Terms Of Use
  • Affiliate Disclosure
  • Legal Notice
  • Privacy Policy
  • Terms Of Use
  • Affiliate Disclosure
© Copyright 2026 Elizabeth Jamieson