When monthly spending on Claude or GPT starts to sting, plenty of businesses start looking at open-weight models instead. The logic that "the weights are public, so there's no usage fee" is half right and half wrong. Which half applies depends on the nature and scale of the work.

Get this judgment call wrong, and an attempt to cut API costs ends up burning more money and time on server rentals, technical upkeep, and error handling. Get it wrong in the other direction — sticking with the API when the numbers favor switching — and you keep paying for the same work unnecessarily. Figuring out which case you're in starts with a few numbers.

The Number That Matters More Than the Per-Token Price

API cost calculations usually start with the per-token price. GPT-4o runs about $5 per million input tokens and $15 per million output tokens (per OpenAI's official pricing in the second half of 2024). Claude 3.5 Sonnet is $3 per million input tokens and $15 per million output tokens.

The real question is: how many tokens are you burning a month?

Say drafting a single contract uses 2,000 tokens combined, input and output. Processing 500 contracts a month adds up to 1 million tokens total — about $8 a month on GPT-4o. At that scale, there's no reason to run your own server.

The math changes if that same business processes dozens of large documents a day, or repeatedly runs long analytical queries. Push past 50 million tokens a month and you're looking at more than $250 a month on GPT-4o — and Claude Sonnet lands in a similar range. That's the point where running the numbers on an open-weight model starts to matter.

The Infrastructure Open-Weight Models Demand

Running Llama 3.1 70B locally requires at least 40GB of GPU memory (with 4-bit quantization). Renting a cloud instance at that spec and keeping it running around the clock costs millions of Korean won a month. If your API bill runs around 300,000 won (roughly $220) a month, cloud-hosted local inference actually ends up pricier.

The much smaller Llama 3.1 8B, by contrast, runs on a consumer graphics card (around 16GB of VRAM). A one-time card purchase runs 600,000–800,000 won (roughly $450–600); amortized over 24 months and including electricity and upkeep, that works out to about 50,000–80,000 won ($40–60) a month. Once your API bill climbs past 100,000–150,000 won ($75–110) a month, the math flips in favor of running it yourself.

There's a real performance gap between an 8B-class model and GPT-4o, though. For clearly patterned tasks — filling out forms, simple classification, keyword extraction — an 8B model is often enough. The gap shows up in complex reasoning, holding context across long documents, and multi-step judgment calls.

Setting up and maintaining local-serving tools like Ollama or vLLM also takes time. Leave that time out of the cost calculation — converted into an hourly rate — and the math stops adding up.

When the Choice Gets Made Before the Cost Math Even Starts

One variable settles the choice faster than any cost calculation: the nature of the data you're handling.

Using a closed API means your input data travels to an outside server. Whether it's used for training varies by service and by enterprise contract terms, but the underlying fact is the same either way: the data physically leaves your control.

For businesses handling clients' personal information, financial data, or unannounced business plans, this is the first thing to check. Under Korea's Personal Information Protection Act (PIPA), outsourcing personal-data processing to a third party triggers a notification obligation, and medical or financial data carries its own separate regulations.

Run an open-weight model locally, and the data never leaves the building. Even when the cost math favors a closed API, local inference stays on the table wherever data sovereignty is a requirement.

When Fine-Tuning Enters the Picture

When a general-purpose model repeatedly falls short on domain-specific work, fine-tuning comes under consideration.

Closed-API providers offer fine-tuning too. But it means uploading training data to an outside server, comes with its own training costs on top, and runs straight into the same data-sensitivity issue raised above.

With open weights, you can adjust the model's parameters directly — tuning it to specific contract-language patterns, a particular medical-chart summary format, or a company's own data structures. Fine-tuning takes prep work on training data, GPU time, and validation, but the finished model then runs repeatedly with no further API charges. For a business running the same type of task at high volume, that upfront investment can pay for itself.

Meta is expanding its developer ecosystem by open-sourcing weights, while OpenAI is building its revenue model around a closed API. Regardless of which strategy wins that broader contest, the question an individual business needs to ask is a different one.

What's the monthly processing volume? Does the data include anything that shouldn't leave the building? Is there the technical skill and time to run a model in-house? Does the work demand precise reasoning quality? Answer those four first, and the question stops being "which AI should I use" and becomes "which setup fits my situation right now." That one is much easier to answer.