Is self-hosting an AI model actually cheaper than paying for an API?

The “just self-host it, it’s free” advice shows up under every post about AI API bills, and the reply to it is always some version of “you clearly have never run a GPU box.” Both sides are arguing past each other, because they are talking about completely different workloads while using the same phrase.

I have an actual data point on this. The audio button on every post here, the one that reads a plain-English summary out loud, is not calling a text-to-speech API. It is a self-hosted neural model running on the same small VPS that serves this site. It has been running for months. So this is not a thought experiment about what self-hosting might cost, it is what one real case actually cost me, plus the arithmetic on the case where the answer flips the other way.

“Self-hosting a model” means three very different things

Before any cost comparison makes sense, the phrase has to be split up, because the economics are not remotely similar across the three:

1. A small, task-specific model on CPU. Text-to-speech, speech-to-text, embeddings, classification, OCR. Tens or hundreds of megabytes. No GPU. Runs next to your app on a box you already pay for.
2. A mid-size open-weight LLM on a rented GPU. Something in the 7B to 70B range. Needs real VRAM, which means a GPU instance, which means an hourly bill that runs whether or not anyone is using it.
3. A frontier model. You cannot self-host this at all. The weights are not available. There is no cost comparison to make, only a capability one.

Almost every “self-hosting is cheaper” argument on the internet is really about case 1, and almost every “self-hosting is a trap” argument is really about case 2. Both are correct about their own case.

Case 1: what my self-hosted model actually costs

The setup is deliberately small. A Go service wraps the Piper TTS binary, feeds it the post summary on stdin, pipes the resulting WAV through ffmpeg into Opus at 40 kbps, and writes the result to disk keyed by a hash of the text. The whole thing lives on a 2 GB / 1 vCPU Ubuntu VPS with no GPU, shared with other projects, behind one nginx location.

The cost accounting looks like this:

Line item	Cost
VPS	Already paid for, hosting several projects. Adding this claimed no new instance.
GPU	None. The model runs on CPU.
Model weights	Free, downloaded once, about 60 MB on disk.
Per-request cost	Zero. Each summary is synthesized once, then served from a disk cache forever.
Ongoing spend	Zero.

The marginal cost of adding it was genuinely zero dollars, and that is not a rounding trick. The interesting part is not that it was free, it is why it was free, because those reasons are the actual test for whether self-hosting will work for anything else.

The output is cacheable and effectively immutable. A published post’s summary never changes. So the model runs exactly once per post, ever. Everything after that is nginx serving a static Opus file. Amortized over a year of traffic, the compute cost per playback rounds to nothing:

// Cache key includes the tuning params, so changing the reading pace
// naturally busts stale audio instead of serving the old version.
hash := sha256.Sum256([]byte(s.lengthScale + "\x00" + s.sentenceSilence + "\x00" + req.Text))
cachePath := filepath.Join(s.cacheDir, hex.EncodeToString(hash[:])+".opus")

if _, err := os.Stat(cachePath); err == nil {
	serveAudio(w, cachePath)
	return
}

The workload tolerates a slow first request. Synthesizing a 200-word summary on one vCPU takes a few seconds. That is fine, because it happens once, for whoever clicks first on a new post, and they are already expecting a moment before audio starts. If every request had to be fast, one shared vCPU would not have been enough.

The concurrency is bounded by me, not by traffic. This is the part people skip. On a hosted API, capacity planning is the vendor’s problem. Self-hosting hands it back to you, and on a 1 vCPU box you have to say no to work explicitly, or the model will happily starve the web server:

select {
case s.sem <- struct{}{}: // maxConcurrent = 2
	defer func() { <-s.sem }()
case <-time.After(synthTimeout):
	http.Error(w, "server busy, try again shortly", http.StatusServiceUnavailable)
	return
}

That semaphore is capacity planning. It is three lines, but it is the difference between a self-hosted model being a feature and being an outage.

Case 2: where the math flips hard

Now run the same accounting for a mid-size open LLM, and the shape of the bill changes completely. The key difference is not that GPUs are expensive. It is that a GPU instance bills by wall-clock time, and an API bills by tokens. You are not comparing two prices, you are comparing a fixed cost to a variable one, and that always produces a break-even point rather than a winner.

Prices below are rough as of August 2026 and vary a lot by provider and GPU class. Check current rates yourself before planning around any of them. The point is the shape of the arithmetic, not the exact figures.

A single mid-tier inference GPU, an L4 or A10G class card, sits somewhere around $0.50 to $1.20 an hour on the cheaper GPU providers, and meaningfully more on the big clouds. Running one continuously:

Utilization	Effective monthly cost	What you get for it
24/7, one GPU at ~$0.70/hr	~$500/mo	Full capacity available at all times, paid for whether idle or not
8 hours a day	~$170/mo	Cheaper, but now you are managing start/stop and eating cold starts
Bursty, a few hundred requests a day	Still the full hourly bill	Almost entirely wasted spend

Against that, a hosted API bills you for exactly the tokens you send. At low or bursty volume that is not close, the API wins by an order of magnitude or more. The self-hosted GPU only starts winning once you are pushing enough sustained tokens per month that your API bill clears the fixed hourly cost, and critically, you have to be pushing them continuously, not in a spike at 9am and nothing after lunch.

So the honest rule for case 2 is: self-hosting an LLM is a high-volume, steady-load optimization. If your traffic is spiky or small, you are paying for an idle GPU, which is the single most expensive way to not use a model.

The costs the comparison usually leaves out

Even when the raw arithmetic favors self-hosting, these do not show up on either invoice:

• Your own time, twice. Once to get it running, then forever in small maintenance increments. My TTS service needed a systemd unit, a dedicated user, a cache directory with the right ownership, an nginx location, and a fix for the fact that the Piper binary resolves its espeak data directory relative to its own working directory. None of that was hard. All of it was time.
• A bigger operational surface. That is one more process that can die, one more disk that can fill, one more thing to patch. It is a real cost even when it is a small one, and it is the cost that grows fastest as you add self-hosted pieces.
• No burst headroom. An API absorbs a traffic spike as a larger bill. Your box absorbs it as a queue, then as timeouts. This is exactly the “hard wall versus slow queue” trade-off I keep running into on the vendor side of AI pricing too.
• Model updates are now your project. Nobody upgrades your weights for you. That cuts both ways: it is also why nobody can deprecate your model version out from under you.
• Cold starts, if you try to be clever about idle time. Shutting the GPU down between bursts is the obvious way to cut the hourly bill, and it converts your cost problem into a latency problem. Loading a large model into VRAM is not instant.

The reasons to self-host that have nothing to do with cost

Worth separating out, because these are often the real motivation and cost is just the argument people reach for:

• The data never leaves your machine. For anything you genuinely cannot send to a third party, this ends the discussion regardless of price.
• No per-request billing surprise. A fixed monthly cost is easier to reason about than a variable one, even when the variable one averages lower. There is no scenario where a bug in a retry loop turns into a five-figure invoice.
• Version stability. Your model behaves the same next quarter as it does today because it is the same file on the same disk.
• It works with no network at all. Rare, but when it matters it matters completely.

Notice that none of those four are cost arguments, and all four are legitimate. If one of them applies, do the self-hosting and treat the cost math as a constraint to work within rather than the reason for the decision.

How I would actually decide

The whole question collapses down to a few checks, in this order:

1. Is the model small enough to run on CPU? If yes, and you already pay for a server, self-host it. The marginal cost is close to zero and the decision is not interesting.
2. Is the output cacheable? Cacheable output turns per-request compute into a one-time cost. This is what made my case free, and it is the single biggest lever available.
3. If it needs a GPU, is the load steady and high? Do the arithmetic before anything else: fixed hourly cost against your projected token bill. Bursty or small load means the API wins, and it usually wins by a lot.
4. Is there a non-cost reason, privacy, offline use, version stability? If yes, that outranks the arithmetic. Just be honest that you are buying something other than savings.

For this blog, every one of those pointed the same way: a tiny CPU model, permanently cacheable output, no latency pressure, on a server already running. That is close to the ideal self-hosting case, which is exactly why it is a bad case to generalize from. Most AI workloads are not that shape. Mine happened to be, and the audio button on this page costs me nothing to keep running because of it, not because self-hosting is free.