Few AI debates are as tribal, and as practical to resolve, as open-weight versus closed (API) models. The online version of the argument tends to be about ideology, openness and who gets to control the future. The version that matters for a team shipping a product is much more boring and much more answerable. With capability gaps narrowing and inference costs collapsing, it's a build decision, not a belief.
First, the terms. A closed model is one you reach over an API: a vendor hosts it, you send requests, you never see the weights. An open-weight model is one whose parameters you can download and run yourself, on your own hardware or a cloud you control. (Note: "open weight" isn't always "open source"; you get the model, not always the training data or licence to do anything you like with it.)
Closed (API) models
- Pros: frontier capability out of the box, zero ops, fast to start, and constant upgrades you get for free as the vendor improves the model.
- Cons: a per-token cost that never goes away, your data leaving your boundary on every call, vendor and pricing risk, and behaviour that can change underneath you when the provider ships a new version.
Open-weight models
- Pros: run on your own infrastructure or even on-device, full data control, no per-call tax once the hardware is paid for, and the ability to pin a version that never changes without your say-so.
- Cons: you now own the hosting, scaling, monitoring and tuning; reaching frontier-level quality needs real infrastructure and real expertise.
The two side by side
| Closed (API) | Open-weight (self-hosted) | |
|---|---|---|
| Time to first working call | Minutes | Days to weeks |
| Cost shape | Per token, forever, scales with success | Mostly fixed once hardware is paid for |
| Where your data goes | Leaves your boundary on every call | Never has to leave |
| Ops burden | None | Hosting, scaling, monitoring, tuning: yours |
| Upgrades | Free and automatic | Deliberate, and your job |
| Version stability | Can change under you | Pin it and it never moves |
| Ceiling on capability | Frontier | High, with real infrastructure and expertise |
| Who owns an outage | The vendor | You |
Read the last two rows together, because they are the trade people underestimate. Free upgrades and no ops sound purely good until the model behind your product changes behaviour in a week you were not planning to test anything.
Why this matters
The decision compounds. Per-token API costs are trivial in a pilot and can become one of your largest line items once a feature succeeds and traffic grows, the same trap covered in the AI energy bill. Data sensitivity can turn an easy API call into a compliance problem the moment regulated or confidential information is involved, and it's often the deciding factor for an open-weight deployment. And vendor lock-in quietly removes your leverage: if your whole product is wired to one provider's quirks, a price change or a behaviour change becomes your emergency, not theirs. The same is true one layer up the stack, in the RAG vs fine-tuning choice: whichever approach you pick, keep it reversible.
The honest answer is usually both: a closed frontier model for the hardest, lowest-volume requests, and a smaller open model self-hosted for the high-volume or sensitive ones. (Often the open one can be a small language model that's more than good enough for the narrow task.)
Model the bill at scale, not at pilot
The cost mistake is almost never a bad rate. It is comparing the wrong volumes.
Take a support-summarisation feature. In a pilot it handles 200 tickets a day and the API bill is small enough that nobody opens the invoice. It succeeds, so it goes to every ticket: 20,000 a day. The rate did not change, the volume moved two orders of magnitude, and a line item nobody was watching is now one of the larger ones in the engineering budget.
Do the arithmetic before you choose, with three numbers you can actually estimate: requests per day at full rollout, average tokens in and out per request, and the rate. Then compare that annual figure against what it costs to run a smaller open model on hardware you already pay for. Two things usually fall out of that comparison. High-volume, narrow tasks favour self-hosting far earlier than people expect. And low-volume, hard tasks almost never justify the ops burden, however appealing control sounds.
The pilot-scale bill is the single most misleading number in this decision, which is the same trap as the wider AI cost curve: cheap to start, expensive to keep.
How to actually decide
Decide on three axes, not on which camp you belong to:
- Control. Do you need to pin behaviour, run offline, or keep the model from changing under you?
- Cost at your volume. Model the bill at projected scale, not at pilot scale, where everything looks cheap.
- Data sensitivity. Does the input contain anything that can't cross your boundary?
The single most important move, whichever way you lean, is to abstract the model behind your own interface. If your code talks to a thin internal layer rather than directly to a vendor SDK, switching models (or running an open and a closed one side by side) stays a configuration change, not a rewrite.
In practice that layer is smaller than it sounds. One function that takes your own request shape and returns your own response shape, one place where the provider is named, one place where retries and timeouts live, and prompts stored as data rather than inlined at the call site. Teams that skip it usually do so because a single vendor SDK is faster on day one, which is true, and then discover that the vendor's field names have spread through the codebase by month three. The tell that you skipped it: you cannot answer how long it would take to route ten percent of traffic to a different model. That keeps the decision reversible, which matters because the landscape is moving fast: today's clear winner may be next quarter's expensive mistake. Treat it as an engineering trade-off you can revisit, not a flag you plant once.
Sources
- Stanford HAI: 2025 AI Index