The 2026 Open-Weights LLM Playbook

A practical guide to running open models in production: which model for which job, how big, and on what hardware — CUDA and MLX both covered.

Updated May 2026 — reflects the April 2026 release wave (DeepSeek V4, Qwen 3.6, Kimi K2.6).

---

Why open weights only

Closed APIs are easy. You pay the bill, you get the answer. The interesting engineering — and most of the misunderstanding — is on the open-weights side, where you actually have to think about parameter counts, MoE architecture, quantization, VRAM, and whether your Mac Studio can really run that 1.6T model someone tweeted about.

This guide is open weights only. Every model below can be downloaded, run on your own hardware, and shipped in a product without paying per token. The trade-off is that you have to understand the hardware. That’s what most of this guide is about.

A note on freshness: the open-weights frontier moves fast — three of the most important models in this guide (DeepSeek V4, Qwen 3.6, Kimi K2.6) all shipped in a single 30-day window in April 2026. Specific version numbers will keep changing. The architectural patterns and hardware sizing won’t.

---

Part 1 — The Open Models That Matter in 2026

There are ~7 model families worth knowing for production work. Anything not in this list is either a research artifact or a smaller variant of one of these.

---

DeepSeek — V4 family (released April 24, 2026)

DeepSeek V4 is the current open-weight frontier. Two variants released simultaneously, both under MIT, both with 1M-token context. The headline architecture change is hybrid Compressed Sparse Attention + Heavily Compressed Attention, which cuts inference FLOPs to ~27% of V3.2 and KV cache occupancy to ~10% at 1M context.

Sizes you actually use:

• DeepSeek V4-Pro (MoE, 49B active / 1.6T total) — frontier-class, competes with Claude Opus and GPT-5 on coding/reasoning.
• DeepSeek V4-Flash (MoE, 13B active / 284B total) — fast, efficient, runnable on a multi-GPU setup most teams can afford.
• DeepSeek R1 (still maintained, MoE 37B/671B) — reasoning-tuned predecessor; still relevant if you’re already deployed on it or want a smaller frontier-reasoning option.

License: MIT. As clean as it gets.

Real use scenarios:

1. Self-hosted “private GPT-5” for a regulated enterprise — Brazilian bank, US defense contractor. V4-Pro on 8× H200 in a private datacenter is the standard 2026 answer when you can’t send data to closed APIs but need frontier quality.
2. High-volume coding pipeline at scale. V4-Flash chews through pull requests, code review, refactoring suggestions, automated migration tooling. At 13B active params, throughput per GPU is excellent and per-token cost is negligible after hardware.
3. Long-context document analysis at scales that were API-only six months ago. V4’s 1M context with the new attention mechanism actually works at long ranges (the KV cache doesn’t blow up). Useful for legal discovery, scientific literature review, full-codebase analysis.
4. Drop-in cheap inference via the DeepSeek API if you don’t want to self-host. V4-Flash at $0.14/M input is roughly 18× cheaper than GPT-5 flagship and good enough for most production work.
5. Frontier base for proprietary fine-tunes. MIT licensing makes commercial fine-tuning legally clean — important for vertical SaaS products that want to build a defensible model on top of an open base.

Hardware reality: V4-Pro at full precision needs an 8× H100/H200 cluster. V4-Flash fits comfortably on 2–4× H100 at FP8, or a high-end Mac Studio at heavy quantization for single-user inference. Most teams will use V4-Pro via the API and self-host V4-Flash if they need control.

---

Moonshot — Kimi K2.6 (released April 2026)

Kimi K2.6 is the strongest open-weight coding model as of mid-2026 — top of every relevant benchmark for autonomous long-horizon coding tasks. Native INT4 QAT (quantization-aware training), which means it’s specifically built to run quantized without quality loss. Includes “agent swarm” capability — can orchestrate up to 300 parallel sub-agents.

Sizes you actually use:

• Kimi K2.6 (MoE, 32B active / 1T total) — native INT4, vision-capable, 256K context.
• Kimi K2.5 (predecessor) — still widely deployed, cheaper to host.

License: Modified MIT (free for almost all commercial use; attribution required above 100M MAU or $20M monthly revenue).

Real use scenarios:

1. Production agentic coding products (open-source Cursor / Devin alternatives). K2.6 is the model behind several of these in 2026. Beat-the-API economics for VC-backed AI coding startups.
2. Self-hosted code review and PR analysis on enterprise codebases. Native INT4 quantization is critical here — you get frontier coding quality with materially less hardware than V4-Pro requires.
3. Long-horizon autonomous tasks — Moonshot demonstrated K2.6 running 4,000+ tool calls across 12+ hours to complete a real engineering project. Useful for overnight batch agentic work (codebase migrations, large-scale refactors, documentation generation).
4. Polyglot codebases (Rust + Go + Python + frontend + DevOps). K2.6 generalizes across languages better than most coding-specialized models, which tend to be Python-heavy.
5. Apps where deploying frontier coding capability on your own hardware is a competitive advantage — defense software, financial trading systems, healthcare device firmware. The code itself is the IP and can’t leave the building.

Hardware reality: Native INT4 means K2.6 is genuinely deployable on 4× H100 or 2× H200, which is much more accessible than V4-Pro. Heavy quantization runs on a 256GB Mac Studio for single-user inference.

---

Alibaba — Qwen 3.5 / 3.6 family

The most well-rounded open family. Spans every size class from sub-1B to MoE flagships at 1T. Qwen 3.5 (February 2026) was the major generational release; Qwen 3.6 (March-April 2026) is the agentic-coding-focused refresh on top of it. Both lines are actively maintained.

Sizes you actually use (Qwen 3.5 / 3.6 mix):

• Qwen 3.5 4B / 9B / 27B (dense) — strong all-rounders. The 9B specifically scores 81.7 on GPQA Diamond, unprecedented for sub-30B models.
• Qwen 3.6 27B (dense) — refresh of the 27B with better agentic coding.
• Qwen 3.6 35B-A3B (MoE, 3B active / 35B total) — the throughput sweet spot of the entire open ecosystem in 2026. 35B-quality output at 3B-class speed.
• Qwen 3.5 122B-A10B (MoE, 10B active / 122B total) — runs on a 64GB Mac.
• Qwen 3.5-397B-A17B flagship (MoE, 17B active / 397B total) — frontier-class.
• Qwen 3.6-Max-Preview — currently API-only, not open-weight; mentioned only because the open-weight 3.6 derivatives flow from it.

License: Apache 2.0 for sizes through ~30B; custom (commercially usable) for the larger flagships.

Real use scenarios:

1. Multilingual customer support for global products — Qwen handles Mandarin, Japanese, Korean, Indonesian, Vietnamese, Hindi, Arabic, Portuguese, Spanish at quality Llama can’t match. The default for any product with significant non-English traffic.
2. Cost-efficient high-throughput chat backend. Qwen 3.6 35B-A3B serves 3–5× more concurrent users per GPU than dense 30B alternatives, because only 3B params are active per token. Best price/performance for production serving in 2026.
3. Local agentic coding on Apple Silicon. Qwen 3.6 35B-A3B runs comfortably on a 64GB MacBook Pro M-series via MLX. This combo (MLX + 35B-A3B MoE) is becoming the standard solo-developer setup.
4. On-prem deployment in Asia-Pacific where Chinese-origin models are preferred or required by procurement.
5. Fine-tuning base for vertical SaaS. Qwen 3.5 4B–14B sizes are the most cost-effective fine-tuning bases in the ecosystem — small enough to fine-tune on a single GPU, capable enough to ship.
6. Edge deployment. The Qwen 3.5 0.8B and 2B sizes run on phones and IoT devices — useful for offline AI features in mobile apps.

---

Meta — Llama 4 family

The most-supported open lineup in the world. Every inference framework, fine-tuning library, and tool integration supports Llama first. Llama 4 introduced MoE (Scout + Maverick) and native multimodality. Llama 3.3 70B is still the dense workhorse; Llama 4 Behemoth (288B active / ~2T total) was announced as a teacher model but has not been released as open weights.

Sizes you actually use:

• Llama 3.3 70B (dense) — still the most-deployed open 70B in production.
• Llama 4 Scout (MoE, 17B active / 109B total, 16 experts) — fits on a single H100 with INT4 quantization, 10M-token context.
• Llama 4 Maverick (MoE, 17B active / 400B total, 128 experts) — fits on a single H100 DGX host (8× H100), 1M context, native multimodal.

License: Llama 4 Community License. Permissive for most users; requires special license above 700M MAU. Not available to EU-domiciled companies as of early 2026 — significant gotcha for European deployments.

Real use scenarios:

1. Internal company assistant trained on your wiki/docs. A LoRA fine-tune of Llama 3.3 70B on internal documentation, served via vLLM on a single H100, gives every employee a private ChatGPT-equivalent. Most common Llama deployment pattern.
2. Multimodal RAG on document libraries (PDFs with diagrams, scanned forms, charts). Llama 4 Scout’s native image understanding + 10M context handles this in one model.
3. Long-document workflows — full codebase analysis, book-length document processing, multi-session conversational memory. Scout’s 10M context is genuinely useful here.
4. Multi-tenant SaaS where you need to self-host outside the EU. Llama is the safest open choice because every dependency you’d need (vLLM, TGI, Ollama, llama.cpp, MLX) supports it day one.
5. Fine-tuning teams who need maximum library support. Llama is the most-documented, most-supported fine-tuning base in the ecosystem.

---

Mistral

Europe’s flagship lab. Pragmatic, well-licensed, code-focused. Less hype than DeepSeek or Kimi, more reliability. Particularly important now that Llama 4 isn’t available in the EU.

Sizes you actually use:

• Mistral Small 3 (~24B dense) — efficient, strong instruction-following.
• Mistral Medium / Large 3 — frontier-tier dense and MoE flagships.
• Codestral / Devstral — code-specialized; Devstral is tuned for agentic multi-file coding.
• Magistral (~24B reasoning) — open reasoning model.

License: Apache 2.0 for most releases.

Real use scenarios:

1. GDPR-compliant on-prem chatbot for a European mid-market company. With Llama 4 unavailable in the EU, Mistral has become the default open choice for European enterprises.
2. Agentic coding tool that edits multiple files. Devstral is purpose-built for this — it’s the model behind several open-source Cursor alternatives that don’t want Chinese-origin models.
3. Function-calling backend for product features. Mistral models are reliable at structured JSON output without exotic prompting. Common in “natural language → structured query” features.
4. Document processing in EU languages (French, Portuguese, Italian, Spanish) where Mistral has a measurable edge over Qwen and Chinese models.
5. Cheap local coding assistant on a single GPU. Devstral 24B on a 24GB GPU runs comfortably and handles real refactoring tasks.

---

Google — Gemma family

Google’s open answer to Llama and Qwen. Apache 2.0, sizes from ~1B to ~30B, with vision and tool calling in the latest generation.

Sizes you actually use:

• Gemma 4 9B — strong small model with vision + tool calling.
• Gemma 4 27B — mid-size dense; strong instruction-following.

License: Apache 2.0.

Real use scenarios:

1. Local agent with tool calling on modest hardware. Gemma 4 9B on a 16GB GPU handles function calling reliably — good for desktop assistants, browser extensions, lightweight automation.
2. Vision + text extraction pipeline without paying API prices — reading screenshots, extracting data from charts, processing scanned forms.
3. Edge or on-device deployment for mobile apps, kiosks, industrial devices. Gemma is the most-optimized open family for this.
4. Apps where Apache 2.0 is legally required. Some procurement processes and OSS distributions specifically require an OSI-approved license. Gemma and Mistral are the cleanest options.
5. Workloads on Google Cloud / Vertex AI where Gemma has first-class infrastructure support.

---

NVIDIA — Nemotron family

NVIDIA’s open releases, primarily showcasing what its training and inference stack can do. Worth considering if you’re already deeply invested in CUDA/TensorRT/NeMo.

Sizes you actually use:

• Nemotron Nano (~4B–9B) — efficient reasoning.
• Nemotron Cascade / Ultra — larger reasoning-tuned MoE variants.

License: Varies by release; mostly permissive open-weight.

Real use scenarios:

1. Production inference squeezing every token/sec out of H100/H200/B200. Nemotron is co-designed with TensorRT-LLM and gives measurably better throughput than equivalent Llama/Qwen on the same NVIDIA hardware.
2. Reasoning workload on NVIDIA NIM microservices — if your platform team standardized on NIM, Nemotron is the path of least resistance.
3. Fine-tuning teams already using NVIDIA NeMo. Staying within one toolchain is worth a lot operationally.

Continue with part two of this article.

The 2026 Open-Weights LLM Playbook – Part 2