Open-Weight vs Closed Models: What Changes in Your Prompt

TL;DR: Closed-API prompting and open source model prompting diverge in four concrete ways: you own the exact chat template and special tokens, there's no vendor-enforced system-prompt hierarchy once weights leave the vendor's servers, "thinking mode" toggles vary by model card rather than by vendor, and licenses range from unrestricted (Apache 2.0, MIT) to usage-gated (Meta, Moonshot). Verify each one directly, per model.

What Does "Open-Weight" Actually Mean?

"Open-weight" means the trained parameters (the actual numbers a neural network learned) are published and downloadable, so you can run the model on your own hardware or a cloud GPU you control, instead of only reaching it through the vendor's API. That's a narrower claim than "open-source." The Open Source Initiative's definition, built for software, asks whether a license imposes no restrictions on who can use the code or for what. A lot of open-weight releases fail that test on purpose.

Hugging Face's own model registry makes the distinction visible without any editorializing: every model card carries a license field, and it is either a recognized SPDX identifier like apache-2.0 or mit, or the literal string other, meaning the vendor wrote a custom agreement instead of using a standard one. Checking that one field, per model, per version, is the fastest way to know which situation you're actually in before you build anything on top of it. This piece checked it for every model named below, via the Hugging Face API, on September 3, 2026.

Which Models Are Actually Open-Weight in September 2026?

The current landscape splits into three groups, not two. Closed models never leave the vendor's infrastructure: GPT-5.6, Claude Opus 4.8, Gemini 3.1 Pro, and Grok 4.6 have no downloadable weights at all. Every prompt goes through the vendor's API, full stop.

Open-weight models split further, by how restrictive the license actually is:

The middle column is genuinely unrestricted. Qwen3.8 (Alibaba), DeepSeek-V4-Pro (DeepSeek), and Gemma 4 (Google DeepMind) all carry apache-2.0 on their Hugging Face model cards, confirmed directly via the API on September 3, 2026. OpenAI's own gpt-oss-120b model card describes the release, in the vendor's own words, as "OpenAI’s open-weight models" designed for "powerful reasoning, agentic tasks, and versatile developer use cases", and lists the license as plain Apache 2.0: the same license text, word for word, that ships with countless ordinary open-source libraries. DeepSeek-V4-Pro's license file is even more direct: it's the standard MIT License, copyright DeepSeek, no modifications.

The right column carries genuine restrictions, and they differ from each other. Meta's Llama 4 models are tagged other on Hugging Face because the actual agreement is the Llama 4 Community License, not Apache 2.0 or MIT. Meta's own license text defines what you're receiving as "Meta’s proprietary Llama 4 and Documentation". Proprietary is Meta's own word, in a document that also grants broad reuse rights, which is exactly the tension that makes "open-weight" a more honest label than "open-source" here.

Does a Permissive License Change What You're Allowed to Prompt For?

Not directly, and this is where people conflate two different documents. A model's license governs redistribution, modification, and commercial deployment of the software. A separate acceptable use policy, where one exists, governs what you can use the model to do. Closed APIs fold both into one experience: the vendor's content-moderation layer sits between your prompt and the model, and a disallowed request gets refused or filtered automatically, every time, regardless of whether you've read any policy.

Open-weight models split the two documents apart, and only one of them travels with the file. Meta's license text folds a separate Acceptable Use Policy in by reference, and that policy opens by saying plainly, "We want everyone to use Llama 4 safely and responsibly." But once you've downloaded the weights, nothing in the software itself enforces that sentence: there's no vendor server in the loop to refuse anything. The policy still legally binds you; it just isn't a runtime filter anymore. If your deployment needs the refusal behavior a closed API gives you for free, you have to build it, buy it, or fine-tune it in yourself.

Google's Gemma 4 shows the same split even under a fully open license: the model card's license field reads apache-2.0, and the linked license page is confirmed, by its own title, to be the actual Apache License 2.0 text: genuinely permissive, no added clauses. But Google separately publishes a standalone Gemma Prohibited Use Policy at a different URL, layered alongside the license rather than inside it. Apache 2.0 tells you what you can do with the code. It was never going to tell you what Google would prefer you not do with the model, and checking only the license field would miss that second document entirely.

What Actually Changes in the Prompt Text Itself?

This is the part that affects you every time you write a prompt, not just once at deployment time. Four things move.

The chat template and special tokens. A closed API abstracts this away completely: you send a JSON array of {role, content} messages over HTTPS, and the vendor's infrastructure converts that into whatever raw token sequence the model actually needs. You never see it, and it can change between model versions without your code changing at all. Self-hosting an open-weight model removes that abstraction. Meta's own llama-models repository documents the literal format Llama 4 expects, and it looks nothing like a REST payload:

<|begin_of_text|><|header_start|>system<|header_end|>

You are a helpful assistant<|eot|><|header_start|>user<|header_end|>

Answer who are you in the form of jeopardy?<|eot|><|header_start|>assistant<|header_end|>

Every <|header_start|> / <|header_end|> pair wraps a role name; <|eot|> marks the end of a turn. Get one of these wrong (or double-wrap them, because a serving tool like Ollama or llama.cpp already applies the template automatically), and the model doesn't error out. It just reads a garbled conversation and answers accordingly, which is a much harder failure to diagnose than an HTTP 400. The format isn't even stable across one vendor's own model generations: Llama 3.x used <|start_header_id|>, and Llama 4 dropped the _id. Carrying a template forward from the previous generation breaks silently.

Some open-weight releases go further and require a wholesale format, not just special tokens. OpenAI's gpt-oss models are trained on what OpenAI calls its "harmony response format", and the model card states plainly that they "should only be used with the harmony format as it will not work correctly otherwise." That's a different, incompatible convention from Llama's: proof that "open-weight" doesn't imply one shared prompting grammar across vendors, or even across two labs that both happen to use the word "open."

System-prompt authority. Closed chat products increasingly document an explicit instruction hierarchy that the served model is trained to respect. OpenAI's current Model Spec (the August 18, 2026 revision, checked September 3, 2026) defines six levels of instruction authority, from Root (rules that cannot be overridden by any message) down through System, Developer, User, Guideline, to a bottom tier with no authority at all, covering the assistant's own prior output and any untrusted quoted text. Whether a user instruction beats a system instruction isn't a matter of taste; it's a documented, versioned ranking the vendor trained the model against.

A downloaded checkpoint has no equivalent contract unless the vendor's own training documented one for that specific model, and most model cards don't. Whatever priority a system role message gets in your self-hosted deployment is whatever priority the base model happened to learn during its own instruction tuning, which the model card usually doesn't spell out and the license definitely doesn't. If your workflow depends on a system prompt reliably outranking user input, that's a property you have to test empirically per model, not read off a spec page: it mostly doesn't exist for open-weight releases the way it does for the major closed chat products.

Reasoning and "thinking mode" toggles. These vary per model card within the same vendor's family, not per vendor. Alibaba documents prompt-level /think and /no_think directives only for specific open-source Qwen3 hybrid-reasoning checkpoints, not for Qwen3.8 generally. DeepSeek's newer models run thinking on by default at a fixed effort level, and at least one DeepSeek page documents a temperature parameter that silently has no effect while thinking mode is active. Neither of those facts generalizes to the other vendor, or even to a different-generation checkpoint from the same vendor. The instruction from earlier in this piece applies here at its most literal: read the specific model's own card before you assume a reasoning toggle works the way it did on the last model you deployed.

Sampling parameter exposure. A closed API curates what you're allowed to touch — a seed parameter might be documented as best-effort only, and the numeric range for temperature is whatever the vendor decided to expose. Local inference through something like llama.cpp, vLLM, or Ollama hands you the full sampler stack: temperature, top-p, top-k, repetition penalty, and often a genuinely deterministic seed, because there's no hosted infrastructure standing between your request and the raw generation loop. That's more control, and also more responsibility: nobody curated sane defaults for you, and a value that's safe on one model's context window or sampling curve isn't guaranteed to be safe on another.

None of this is a "local vs cloud" distinction, despite how often the two get merged. You can run an open-weight model in a cloud GPU instance you rent, and several vendors (DeepSeek, Alibaba, Moonshot) also sell their own hosted API access to essentially the same open weights, as a convenience layer on top of the same model. Open-weight versus closed is about who can download the parameters. Local versus cloud is about where the inference happens to run. A model can be open-weight and run entirely in someone else's data center; the license and the deployment location are independent choices.

How Do License Terms Constrain a Deployment, Not Just a Prompt?

This section is for whoever signs off on shipping the thing, and it's worth reading before a legal question becomes a production incident. Two of the licenses above attach a commercial scale threshold directly to the right to use the model at all.

Meta's Llama 4 Community License states that if the monthly active users of Licensee's products or services is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta before you're authorized to exercise any rights under the agreement. Redistributing Llama Materials, or a derivative model built from them, also carries a naming obligation: the license requires you to prominently display “Built with Llama” and to begin any derivative model's name with “Llama”.

Moonshot AI's Kimi K3 License is MIT-shaped but attaches two separate thresholds of its own. The license defines "Model as a Service" as giving third parties meaningful control over inputs, parameters, or training data via something like an API, and if a Model as a Service business built on Kimi K3 earns more than 20 million US dollars in aggregate revenue over any consecutive 12 months, the licensee must enter into a separate agreement with Moonshot AI before continuing commercial use. A second, lower threshold triggers a display requirement rather than a licensing gate: past 100 million monthly active users, or 20 million US dollars in monthly revenue, "Kimi K3" must be prominently displayed on the user interface. Internal use, and traffic routed through Moonshot's own hosted product or certified partners, is explicitly exempted from both.

Apache 2.0 and MIT carry no equivalent clauses. That's the entire practical difference between the middle and right columns of the table above, and it has nothing to do with prompt engineering: it's a question for whoever owns commercial and legal risk, decided per model, before the first API call.

So Which Should You Use — Open-Weight or Closed?

There isn't a universal answer, but the trade-off is at least concrete. A closed API gives you the vendor's own moderation and safety layer, an instruction hierarchy the model was actually trained against, a chat template you never have to think about, and someone else's infrastructure absorbing the operational load, in exchange for no ability to download the model, inspect it, or run it somewhere the vendor doesn't control.

An open-weight model gives you the parameters themselves: you can self-host for data residency or latency reasons, fine-tune on your own data, and inspect exactly what you're running. In exchange, you own getting the chat template exactly right, you own whatever content-safety behavior the base model happened to learn, and, per the license you land on, you may owe attribution, a display requirement, or a separate commercial agreement once you cross a defined threshold.

Neither path changes what Prompt Architects does for the prompt itself. The system prompt, the reusable variables, and the enhancement pass all happen before the text reaches either kind of model, on every plan, with no API key of your own required. Whether that finished prompt then travels to a closed API or a self-hosted checkpoint with its own template requirements is a decision you make on the other end. Reading the model's own license and model card first, rather than assuming it matches the last one you deployed, is the difference between a smooth rollout and a legal surprise six months later.

For a deeper look at how reasoning-tuned checkpoints specifically change what a prompt should look like, see our guide to prompting reasoning models versus chat models. If you're comparing prompt-management tooling itself rather than the underlying model, we've also covered open-source versus hosted prompt managers directly. And if your workflow leans on a closed, source-grounded product layer rather than a raw model, our guide to prompting NotebookLM shows what that closed-and-constrained end of the spectrum looks like in practice.