Open-Weight and Local Models
When to run models yourself instead of calling an API: open-weight models, local inference, privacy, cost, and the capability trade-offs to expect.
Every example so far has sent your text to a model running in someone else's data center. For most work that's the right call: the frontier models are the most capable, and someone else handles the hardware. But sometimes you can't, or shouldn't, send the data away. Patient notes, legal documents under a strict agreement, source code your company won't let leave the building. Sometimes you're processing millions of documents and the API bill is the problem. Sometimes you're on a plane.
That's where open-weight models come in: models whose weights anyone can download and run on their own hardware. In this lesson you'll learn what "open-weight" actually means, the real reasons to run a model yourself and the real costs, how much hardware it takes, how to run one on your laptop and call it with the same SDK you already use, and where local models fit into an agent without dragging its quality down.
What "open-weight" means
A model is ultimately a very large file of numbers, the weights, learned during training. With a closed model like Claude, the weights stay with the provider and you reach the model through an API. With an open-weight model, the provider publishes the weights so you can download them and run the model anywhere.
Well-known open-weight families include Meta's Llama, Alibaba's Qwen, Mistral, DeepSeek, Google's Gemma, and OpenAI's gpt-oss. New versions arrive every few months, and the leaderboard among them shifts often, so treat any specific name as a starting point rather than a recommendation.
Three distinctions trip people up:
- Open-weight is not the same as open source. You usually get the weights, but not the training data or the full training code. You can run and fine-tune the model, but you can't rebuild it from scratch.
- Licenses vary a lot. Some models use permissive licenses such as Apache 2.0, which allow commercial use with few conditions. Others use custom licenses with restrictions on use cases, user counts, or attribution. Read the license for the exact model before you build a product on it.
- Local is not the only way to run them. You can run open-weight models on your laptop, on your own servers, inside your cloud account, or through hosting providers that serve them via an API. "Open-weight" describes what you're allowed to do, not where it has to run.