Skip to main content

Gemma 4: Google’s Open Model Play Just Got Serious

Google DeepMind releases four open-weight models from 2B to 31B parameters with multimodal input, 256K context, native thinking, and benchmark scores that embarrass last year’s frontier models. I dig into what’s real, what’s hype, and why the MoE variant might be the most important model released this year.

finished certainty: likely importance: 8

There’s a particular feeling you get when you download a model, load it up on your own hardware, feed it a problem you know is hard, and watch it just… nail it. Not in the “oh that’s a nice summary” way but in the “wait, this thing just solved an AIME problem that I’d need twenty minutes and scratch paper for” way. I had that feeling three weeks ago when I first ran Gemma 4’s 31B instruction-tuned variant, and I’ve been trying to figure out what to make of it ever since.

Google DeepMind released the Gemma 4 family on March 11, 2026, and the numbers are, frankly, a little absurd. The flagship 31B dense model posts an 89.2% on AIME 2026 without tools — for context, Gemma 3’s 27B managed 20.8% on the same benchmark. That’s not an incremental improvement; that’s a different universe. The Codeforces ELO jumped from 110 to 2150, which translates roughly from “can’t solve the easiest problems” to “competitive with strong human contestants.”1 But the model that has me most excited isn’t the 31B. It’s the weird one — the 26B Mixture-of-Experts variant that activates only 3.8 billion parameters per token while scoring within spitting distance of the dense model on nearly every benchmark. If you care about actually running powerful models on consumer hardware, that’s the story here.

Let me back up and tell the whole story, because the context matters.

The Gemma Arc: From Afterthought to Contender

Google’s relationship with open-weight models2 has been… complicated. When Meta dropped LLaMA in early 2023, it felt like Google got caught flat-footed. Their response, Gemma 1, arrived in February 2024 and was fine — competent 2B and 7B models that nobody was particularly excited about. Gemma 2 in June 2024 was better, introducing the sliding window attention pattern3 that would become a signature architectural choice, but it still felt like Google was playing catch-up while Meta, Mistral, and the Chinese labs were setting the pace.

Gemma 3 in March 2025 was when things got interesting. The 27B model was legitimately good — multimodal, long context, competitive benchmarks. But it still had that “one model at one size” feel that made it hard to build an ecosystem around. You had a small model for edge deployment and a medium model for everything else, and neither felt like they were pushing boundaries in any single dimension. Meanwhile, DeepSeek was publishing papers that made the entire field rethink what was possible with efficient architectures, and Qwen was quietly shipping models that punched way above their weight class.

Gemma 4 feels different. It feels like Google finally decided to stop hedging and actually compete in the open-weight space. Four models spanning from 2.3B effective parameters to 31B dense, covering edge devices through workstation-class deployments, all under Apache 2.0, all with multimodal capabilities, all with native thinking support. The HuggingFace page shows 678K downloads and 1,144 likes in roughly a month, which puts it firmly in “the community is paying attention” territory.

But downloads and likes are vanity metrics. Let’s talk about what actually changed under the hood.

The Architecture: Four Models, Three Philosophies

The Gemma 4 family isn’t just “the same model at different sizes.” There are genuinely different architectural decisions at each scale point, which is unusual and, I think, underappreciated. Here’s the lineup:

The two small models — E2B (2.3B effective / 5.1B total parameters) and E4B (4.5B effective / 8B total) — use a technique called Per-Layer Embeddings, or PLE.4 This is one of those ideas that sounds obvious in retrospect but apparently nobody tried before: instead of sharing a single embedding table across all decoder layers, each layer gets its own embedding lookup. The “E” in E2B stands for “effective” — the model has 5.1B total parameters, but only 2.3B are “active” in the traditional sense because a huge chunk of the parameter budget is distributed across these per-layer embedding tables. The practical result is that you get a model that punches significantly above its effective parameter count on knowledge-intensive tasks, because each layer can specialize its representation of the input tokens. It reminds me of ELMo’s insight from 2018 that different layers capture different linguistic properties — except applied at the embedding level rather than as a post-hoc feature extraction trick.

The 31B dense model is more conventional in architecture but pushes the envelope on attention mechanism design. It uses a hybrid attention scheme5 that interleaves sliding window layers (with windows of 512-1024 tokens) with global attention layers, with the constraint that the final layer is always global. This isn’t new to Gemma — the sliding window approach traces back to Longformer and has been a Gemma signature since version 2 — but the implementation details matter. The global attention layers use Proportional RoPE (p-RoPE)6, a variant of Rotary Position Embedding that scales the rotation frequencies proportionally rather than using the standard exponential decay. They also employ Unified Keys and Values in global layers, which means the key and value projections share parameters across attention heads within a layer. This is a memory efficiency trick that reduces the KV-cache footprint7 without — if the benchmarks are to be believed — meaningfully hurting quality.

And then there’s the 26B A4B MoE, which is the model I keep coming back to.

Why the MoE Variant Might Be the Most Important Model Released This Year

Mixture-of-Experts is having a moment, and it’s been a long time coming. The core idea — route each token to a subset of specialized “expert” subnetworks rather than processing it through the entire model — dates back to Jacobs et al. in 1991. It languished in relative obscurity for decades, had a brief resurgence with Shazeer et al.’s work at Google in 2017, and then exploded into the mainstream when Mistral released Mixtral 8x7B in December 2023 and showed that you could get 70B-class performance with 12B-class inference costs.

Gemma 4’s MoE variant takes this further than anyone has in the open-weight space. The numbers: 25.2 billion total parameters, 128 expert subnetworks, 8 active per token, plus 1 shared expert that’s always on. The effective active parameter count is 3.8 billion. Let me say that again: 3.8 billion active parameters producing results that compete with a 31 billion parameter dense model. On AIME 2026, the MoE scores 88.3% versus the dense model’s 89.2%. On GPQA Diamond, it’s 82.3% versus 84.3%. On LiveCodeBench v6, 77.1% versus 80.0%. These gaps are tiny. The MoE model achieves roughly 95% of the dense model’s performance while activating roughly 12% of the parameters per forward pass.

The practical implication is that you can run near-31B-class intelligence on hardware that would buckle under the dense model. Yes, you still need enough RAM to hold the full 25.2B parameters (the experts aren’t active but they’re still in memory), but the actual compute per token is dramatically lower. With GGUF quantization from Unsloth, you can fit this model comfortably in 16GB of VRAM at Q4 precision, which means it runs on an RTX 4080 or an M2 Pro MacBook. A 31B dense model at the same quantization level needs closer to 20-24GB. That difference matters enormously for the hobbyist and researcher demographic that actually drives open-model innovation.

The shared expert is worth pausing on. Most MoE architectures route tokens purely through the top-k selected experts, but Gemma 4’s MoE always activates one additional “shared” expert regardless of the routing decision. This acts as a kind of common knowledge backbone — it processes every token and presumably captures the general-purpose language modeling capabilities that every expert benefits from, while the routed experts specialize. DeepSeek-R1 uses a similar shared-expert design, and I suspect this is becoming the consensus architecture for production MoE systems because it dramatically reduces the “expert imbalance” problem8 where some experts get undertrained due to routing patterns.

The thing nobody is talking about is what 128 experts means for fine-tuning. With that many experts, you potentially have a very fine-grained modular structure that could be exploited for targeted adaptation — activating or deactivating specific experts for specific domains, merging expert weights from different fine-tunes, or even doing expert-level pruning for deployment on more constrained hardware. The community has already started experimenting with this, and I expect the MoE variant to develop a richer fine-tuning ecosystem than the dense model precisely because there’s more structure to exploit.

The Benchmarks: Impressive, Self-Reported, Proceed With Caution

I’ll show you the numbers because they’re genuinely informative, but I want to preface this with my standard caveat: these are Google’s self-reported results on their chosen benchmarks. The community hasn’t had enough time to fully verify everything, and we’ve all seen enough benchmark gaming9 in this industry to be appropriately skeptical. That said, the improvements over Gemma 3 are large enough that even if the absolute numbers are somewhat optimistic, the directional story is clear.

Core Reasoning & Knowledge

Benchmark

Gemma 4 31B

Gemma 4 MoE

Gemma 4 E4B

Gemma 4 E2B

Gemma 3 27B

MMLU Pro{.link-annotated-not}

85.2%

82.6%

69.4%

60.0%

67.6%

AIME 2026 (no tools)

89.2%

88.3%

42.5%

37.5%

20.8%

GPQA Diamond

84.3%

82.3%

58.6%

43.4%

42.4%

Code

Benchmark

Gemma 4 31B

Gemma 4 MoE

Gemma 4 E4B

Gemma 4 E2B

Gemma 3 27B

LiveCodeBench v6

80.0%

77.1%

52.0%

44.0%

29.1%

Codeforces ELO

2150

1718

940

633

110

Vision

Benchmark

Gemma 4 31B

Gemma 4 MoE

MMMU Pro (vision)

76.9%

73.8%

MATH-Vision

85.6%

82.4%

Long Context

Benchmark

Gemma 4 31B

Gemma 4 MoE

256K context MRCR v2

66.4%

44.1%

A few things jump out. First, the AIME performance is extraordinary. Going from 20.8% to 89.2% in one generation — with thinking mode enabled, admittedly — suggests that whatever training recipe Google used for the reasoning capabilities represents a genuine qualitative shift, not just scaling. For reference, Claude 3.5 Sonnet was scoring in the 60-70% range on similar math competition benchmarks when it launched, and that was Anthropic’s frontier model. An open-weight 31B model matching or exceeding that is a big deal.10

Second, the Codeforces ELO numbers deserve unpacking. A rating of 2150 puts Gemma 4 31B firmly in the “Candidate Master” range on Codeforces, which means it can solve medium-difficulty competitive programming problems consistently and occasionally crack hard ones. An ELO of 110 (Gemma 3) means it basically can’t solve anything — that’s below the floor of meaningful competitive programming ability. The jump is so large it almost doesn’t make sense as a comparison; it’s like comparing a chess novice to a national-level player. The MoE at 1718308ya is “Expert” level, which is still remarkably strong. I should note that Codeforces ELO as a benchmark has known issues11 — the evaluation methodology varies across papers, the problem distribution matters enormously, and models can exploit patterns in competitive programming that don’t transfer to real-world software engineering. But even with generous error bars, this is impressive.

Third, and this is where I get skeptical: the long-context benchmark shows a real gap between the 31B and MoE models. At 256K tokens, the 31B scores 66.4% on MRCR v2 while the MoE drops to 44.1%. That’s a 22-point gap, much larger than the 2-3 point gaps we see on other benchmarks. I suspect this reflects a fundamental limitation of MoE architectures for long-context tasks — when you’re routing tokens to different experts, maintaining coherent attention over very long sequences requires the routing decisions to be consistent in ways that may be hard to learn. This is an area where the dense model’s advantage is real and meaningful, and if your use case involves processing entire codebases or long documents, the 31B is probably worth the extra compute.12

Running Gemma 4 Locally: A Practical Guide

Getting these models running locally is — refreshingly — not that painful anymore. The ecosystem has matured enough that the gap between “model released” and “running on my laptop” is measured in hours rather than weeks. Here’s what actually works as of early April 2026.

The fastest path for most people is Ollama, which had Gemma 4 support within days of launch:

# Pull the MoE model (recommended for most hardware)
ollama pull gemma4:26b-a4b

# Or the dense 31B if you have the VRAM
ollama pull gemma4:31b

# Run with thinking enabled
ollama run gemma4:26b-a4b

# In the chat, you can toggle thinking:
>>> /set parameter num_ctx 131072
>>> Solve this step by step: What is the integral of x^2 * e^(3x) dx?

For more control, llama.cpp with the Unsloth GGUF quantizations is the way to go:

# Grab the Q4_K_M quantization of the MoE variant
huggingface-cli download unsloth/gemma-4-26B-A4B-it-GGUF \
  gemma-4-26B-A4B-it-Q4_K_M.gguf --local-dir ./models

# Run with llama.cpp server
./llama-server \
  -m ./models/gemma-4-26B-A4B-it-Q4_K_M.gguf \
  -c 131072 \
  -ngl 99 \
  --port 8080

# Now you have an OpenAI-compatible API at localhost:8080

For Python-native workflows, vLLM supports Gemma 4 out of the box and handles the MoE routing efficiently on multi-GPU setups:

from vllm import LLM, SamplingParams

llm = LLM(
    model="google/gemma-4-27b-a4b-it",
    tensor_parallel_size=2,  # for multi-GPU
    max_model_len=131072,
)

sampling = SamplingParams(
    temperature=0.7,
    max_tokens=4096,
)

output = llm.generate(
    ["Explain the strategic implications of MoE architectures "
     "for open-weight model deployment."],
    sampling,
)
print(output[0].outputs[0].text)

A word of practical advice: the thinking mode makes a huge difference on reasoning tasks but significantly increases token usage and latency. For simple Q&A or text generation, you’re better off with thinking disabled. The model supports toggling this at inference time via the <|think|> special token, which is a nice design choice — you don’t need separate model weights for thinking vs. non-thinking mode.13

I’ve been running the MoE variant on an M3 Max MacBook Pro with 64GB unified memory, and it’s surprisingly usable. Token generation is around 15-20 tokens/second at Q4_K_M quantization, which is fast enough for interactive use. The 31B dense model on the same hardware is closer to 8-10 tokens/second. If I’m doing anything that requires back-and-forth iteration — debugging code, exploring ideas — the MoE’s speed advantage matters more than the dense model’s marginal quality edge.

What Google Is Actually Doing Here (The Strategy)

Let’s zoom out from the technical details and talk about why Google is doing this. Because open-weight models are expensive to develop and release, and Google is not a charity. There’s a strategic logic here that I think is worth examining.

The cynical reading is that open-weight models are a competitive moat strategy. By releasing models that are “good enough” for most applications under Apache 2.0, Google ensures that the default infrastructure for AI development runs on Google-originated architectures. Developers build on Gemma, which means they build familiarity with Google’s ecosystem — Google Cloud, Vertex AI, the AI Studio playground. When those developers need something more powerful, they’re already primed to reach for Gemini rather than Claude or GPT. It’s the Android playbook applied to AI: give away the OS, sell the services.

The less cynical reading is that Google genuinely believes open models accelerate the field in ways that benefit everyone, including Google. More researchers working with Gemma-derived architectures means more published improvements that Google can incorporate back into Gemini. The fine-tuning community discovers failure modes, develops new training techniques, and stress-tests capabilities in ways that Google’s internal red team never could. DeepSeek’s work on reinforcement learning for reasoning demonstrably pushed the entire field forward, and Google’s reasoning improvements in Gemma 4 almost certainly benefited from the public discourse around chain-of-thought training that DeepSeek catalyzed.

The truth is probably both, and I don’t think that’s a bad thing. The Apache 2.0 license is genuinely permissive — you can use these models commercially, modify them, redistribute them, without meaningful restrictions.14 Compare this to Meta’s Llama license, which has a monthly active user threshold that introduces commercial uncertainty for scaling startups, or to the various “open but not really” licenses we’ve seen from other labs. Google choosing Apache 2.0 is a real commitment, and it deserves credit regardless of the strategic motivations.

There’s also a geopolitical dimension that I think is underappreciated. The race for open-weight model supremacy is increasingly a proxy for influence over global AI development norms. When a developer in Lagos or Jakarta or São Paulo reaches for an open model to build their application, the model they choose shapes their understanding of what AI can do, how it should work, and whose documentation they read. Qwen from Alibaba, DeepSeek, and other Chinese lab releases are serious contenders in this space, and the US labs — Google and Meta primarily — have strategic reasons to ensure that Western-originated models remain competitive and accessible.15 This isn’t about model quality in a vacuum; it’s about the infrastructure of global AI development.

The Multimodal Story: Better Than You’d Expect, Worse Than You’d Hope

All four Gemma 4 models handle images natively, which is table stakes at this point. The small models (E2B and E4B) additionally handle audio input, which is a nice touch for edge deployment scenarios — think voice assistants running on-device. The larger models are image-only on the input side, text-only on the output side. No image generation, no video understanding, no audio for the 31B/MoE. This is a meaningful limitation compared to frontier models like GPT-4o or Gemini itself, which handle arbitrary modality combinations.

What’s clever about the vision implementation is the variable resolution system. Rather than resizing all images to a fixed token count (which wastes context on simple images and starves complex ones), Gemma 4 lets you configure a visual token budget per image: 70, 140, 280, 560, or 1,120 tokens.16 At the low end, you’re getting a thumbnail-level understanding; at the high end, you can actually parse fine details in charts, diagrams, and dense documents. The MMMU Pro score of 76.9% for the 31B model is strong — this benchmark specifically tests multimodal understanding on college-level problems across multiple domains — and the vision-math performance (85.6% on MATH-Vision) suggests that the model can genuinely “read” mathematical notation in images and reason about it.

I’ve tested the vision capabilities on a mix of tasks: reading handwritten notes (decent but not great), understanding architectural diagrams (surprisingly good), parsing screenshots of code with syntax highlighting (excellent), and analyzing plots from academic papers (good at description, mediocre at drawing non-obvious conclusions). The overall impression is that vision is a solid B+ — clearly useful, clearly integrated into the reasoning pipeline rather than bolted on, but not at the level where I’d trust it for critical visual analysis tasks without verification.

Per-Layer Embeddings: A Small Idea That Might Be Big

I want to spend a moment on PLE because I think it’s the most intellectually interesting architectural choice in Gemma 4, even though it only appears in the small models. The standard transformer architecture uses a single embedding matrix that maps tokens to vectors at the input, and then those vectors get transformed through successive layers. Every layer sees the same initial embedding of each token, modified only by the accumulated attention and feed-forward computations of previous layers.

PLE gives each decoder layer its own embedding table. This means layer 5 can have a completely different initial representation of the word “bank” than layer 12, before any contextual computation happens. The total parameter count goes up substantially (which is why the E2B has 5.1B total parameters despite only 2.3B “effective” parameters in the traditional sense), but the compute per forward pass doesn’t increase proportionally because embedding lookups are cheap relative to matrix multiplications in attention and feed-forward layers.

The intuition is compelling. We know from probing studies17 that different layers in transformers capture different types of information — early layers tend to handle syntactic structure, middle layers semantic relationships, and later layers task-specific reasoning. If different layers want different things from the token representation, why force them all to start from the same embedding? PLE essentially gives each layer a head start by letting it begin with a representation that’s already tuned to its role in the network.

This reminds me of the insight from ELMo back in 2018, where Peters et al. showed that using a weighted combination of all LSTM layers as the word representation (rather than just the top layer) dramatically improved downstream task performance. The key finding was that different layers captured different and complementary information. PLE is like the architectural dual of that insight: instead of combining layers’ outputs post-hoc, give each layer its own input representation. Whether this idea scales to larger models is an open question — the parameter overhead might become prohibitive at 31B+ scale — but for the small model regime, it’s a genuinely clever way to pack more capability into fewer effective FLOPs.

The Thinking Mode: Following the DeepSeek Playbook

Let’s be honest about what’s happening with “native thinking” in Gemma 4: Google is following the playbook that DeepSeek-R1 wrote. The <|think|> token that toggles chain-of-thought reasoning, the training pipeline that uses reinforcement learning to improve reasoning traces, the dramatic benchmark improvements when thinking is enabled — this is all downstream of the revolution that DeepSeek kicked off in January 2025 when they showed that RL-trained reasoning could close the gap between open and closed models on mathematical and coding tasks.

I don’t say this as criticism. Following good ideas is how science works, and Google’s implementation is clean. The thinking mode is toggleable at inference time without needing separate weights, which is operationally much nicer than maintaining separate “reasoning” and “non-reasoning” model variants.18 The training approach — which the documentation describes as using both supervised fine-tuning on reasoning traces and reinforcement learning from verifiable rewards — is now the standard recipe, but execution matters and Google clearly executed well given the benchmark results.

What I find more interesting is the meta-question: are we converging on a standard architecture for “reasoning models”? The pattern is now clear across multiple labs: take a strong base model, train it to produce chain-of-thought traces using a combination of SFT and RL, add a special token to toggle the behavior, and ship both modes. OpenAI’s o1, DeepSeek-R1, Qwen’s QwQ, and now Gemma 4 all follow this template. The differentiation is moving from “can your model reason?” to “how efficiently and reliably does it reason?” — which is probably healthy for the field, even if it makes for less exciting product announcements.

The benchmarks with thinking enabled versus disabled tell the story quantitatively. The AIME 2026 score of 89.2% for the 31B model is with thinking on; I’d estimate it drops to somewhere in the 40-60% range with thinking off, based on the typical gap seen in other reasoning models. Google doesn’t publish the non-thinking numbers for all benchmarks, which is itself a data point — the thinking mode is clearly doing most of the heavy lifting on the hardest reasoning tasks, and the base model without thinking is probably “merely good” rather than “extraordinary.” This isn’t a complaint, exactly, but it’s worth keeping in mind when you see the headline numbers.

The Vocabulary and Tokenizer: 262K and Why It Matters

A detail that’s easy to overlook: Gemma 4 uses a vocabulary size of 262,144 tokens — roughly 262K. This is massive by historical standards (GPT-2 used 50K, LLaMA used 32K, most modern models use 32K-128K) and reflects a deliberate choice to spend more parameters on the embedding and output layers in exchange for more efficient tokenization of diverse text.

A larger vocabulary means that common words and phrases get their own tokens rather than being split into subword pieces. This has several cascading benefits: fewer tokens per document (which means you can fit more content in the same context window), faster inference (fewer forward passes per page of text), and potentially better handling of non-English languages where smaller vocabularies force aggressive subword splitting that can harm model performance.19 The tradeoff is a larger embedding matrix, which increases model size and memory footprint. For the MoE variant, where the embedding matrix is a fixed cost regardless of expert count, this is a particularly interesting tradeoff — you’re paying a constant memory overhead for tokenizer quality that benefits every inference.

I haven’t done systematic tokenizer efficiency comparisons, but anecdotally, Gemma 4’s tokenizer handles code, markdown, and mixed-language text noticeably better than older tokenizers. JSON structures that would eat dozens of tokens with GPT-2’s tokenizer compress down to reasonable lengths. This matters more than you’d think for practical applications where you’re feeding structured data into the model.

What’s Missing (And What Bugs Me)

No model release is perfect, and Gemma 4 has some gaps that are worth noting.

First, the documentation is good but not great. The official docs cover the basics but leave architectural details frustratingly vague in places. I wanted to understand exactly how the PLE tables are initialized and trained, and the best I could find was a brief mention without sufficient detail to reproduce the approach. For a model released under Apache 2.0, the training code and detailed methodology remain closed, which limits the “open” in “open-weight.” You get the weights but not the recipe, which is better than nothing but worse than what OLMo from AI2 provides.20

Second, the audio capabilities are limited to the small models only. I understand the engineering reasons — audio adds significant complexity to the architecture and training pipeline — but it’s disappointing that the most capable models can’t process audio. In a world where voice interfaces are becoming increasingly important, having audio understanding only in the 2B and 4B variants feels like a missed opportunity.

Third, and this is a broader concern about the industry rather than Gemma specifically: the benchmark numbers are becoming increasingly hard to interpret meaningfully. When every lab reports results on their own selection of benchmarks, with their own evaluation harnesses, with or without thinking enabled, with or without tool use — comparison becomes nearly impossible without independent third-party evaluation. The Open LLM Leaderboard on HuggingFace helps, but it hasn’t caught up with the latest models yet, and its benchmark suite may not capture the reasoning improvements that define this generation of models. I’d love to see a well-funded, independent benchmarking organization that tests every major model release under standardized conditions within a week of launch. Until then, take all benchmark claims — including Google’s — with appropriate seasoning.21

Fourth, the system prompt support, while welcome (Gemma 3 didn’t have native system prompts), has some quirks in practice. The model sometimes “leaks” system prompt behavior into its responses in ways that suggest the boundary between system and user context isn’t as clean as in models that were architected around system prompts from the beginning. Anthropic’s Claude and OpenAI’s GPT-4 handle this more gracefully, though they’ve also had more iterations to refine it.

The Community Response and Ecosystem

One month in, the community ecosystem around Gemma 4 is developing rapidly. Unsloth had GGUF quantizations available within days, including optimized formats for Apple Silicon and NVIDIA GPUs. NVIDIA released NVFP4 quantizations for TensorRT deployment. There’s already a dealignai uncensored variant for people who want to remove the safety fine-tuning (which I have mixed feelings about but acknowledge is a legitimate use case for research). Multiple LoRA fine-tunes have appeared for domain-specific applications ranging from medical Q&A to legal analysis to creative writing.

The speed of ecosystem development reflects both the model’s quality and the maturity of the open-model toolchain. Tools like LangChain and LlamaIndex support Gemma 4 out of the box through their Ollama and vLLM integrations. llama.cpp added native support within the first week. If you’re building an application that needs a strong local model, the barrier to adoption is genuinely low.

What I’m most curious about is how the fine-tuning story develops for the MoE variant specifically. Mixture-of-Experts models have historically been tricky to fine-tune because you need to be careful about which experts get updated and how the routing is affected. Unsloth’s approach of applying LoRA adapters to the shared components while leaving expert-specific weights frozen seems promising, but I haven’t seen rigorous evaluations of fine-tuned MoE variants yet. If the community cracks efficient MoE fine-tuning, the 26B A4B could become the default “smart local model” in a way that no single model has managed before — combining the intelligence of a 25B+ model with the inference speed of a 4B model.

Looking Forward

Gemma 4 is, I think, a genuine inflection point in the open-weight model landscape. Not because any single capability is unprecedented — DeepSeek showed that reasoning could be trained via RL, Mixtral showed that MoE could work at scale, various models showed that multimodal understanding was feasible — but because Gemma 4 combines all of these advances into a coherent, well-engineered family of models under a permissive license with strong ecosystem support. It’s a “best practices compilation” release, and sometimes those matter more than any individual breakthrough.

What People Are Actually Saying

I’ve been tracking the community reaction across Hacker News (1,800 points, 472 comments — one of the highest-scoring AI model stories this year), Reddit’s r/LocalLLaMA, and various YouTube channels, and the conversation is more nuanced than you’d expect from a model release that looks this good on paper.

The consensus seems to be: Gemma 4 is excellent, but it’s not a clean sweep. Several people have noted that Qwen 3.5 outperforms it on pure coding tasks — one commenter gave both models the same Rust project and Qwen ran for over an hour making progress while Gemma gave up after twenty minutes with failed tool calls. But then another noted that “on my 16GB GPU, Gemma 4 is better and faster than Qwen 3.5, both at 4-bit — so it’s not so clear cut.” The picture depends heavily on your hardware, your quantization level, and what you’re actually asking the model to do.22

The on-device story is where Gemma 4 genuinely shines and nobody is arguing otherwise. People are running E2B on iPhones at 30 tokens per second, and someone managed to get it to solve hCaptcha screenshots — a task that requires genuine visual understanding, not just text pattern matching. Google’s official numbers claim 56 tokens/second on iPhone and 52 on Qualcomm’s NPU, though Pixel owners report lower numbers around 22 tok/s — leading one commenter to note dryly that Google “doesn’t even bother listing Tensor in their performance table — maybe because it would be too embarrassing.” Someone else built a real-time AI assistant with audio and video input running entirely on an M3 Pro MacBook using E2B. A year ago that would have been science fiction.

One HN commenter put it perfectly: “It is my firm belief that the only realistic use of AI in the future is either locally on-device for almost free, or in the cloud but way more expensive than it is today.” Gemma 4 E2B is the strongest evidence for the first half of that prediction.

The agentic coding use case has drawn mixed reviews. Several developers have switched from Claude Code’s API to running Gemma 4 locally via LM Studio or llama.cpp, and the reports are honest: it works, but not perfectly. As one developer put it: “I got fed up with Claude Code limits and have been using a combination of Qwen3-Coder, Gemma 4, and Qwen3-VL locally. Gets me 90% of the way there and CC is still around if I need it.” That “90% of the way there” is both the promise and the limitation — good enough for most work, not quite good enough to cancel your API subscription entirely.

There’s a fascinating meta-observation from another HN commenter that I think captures where the whole field is heading: “The real story here isn’t Gemma 4 specifically, it’s that the harness and the model are now fully decoupled. Claude Code, OpenCode, Pi, Codex all work with any backend. The coding agent is becoming a commodity layer and the competition is moving to model quality and cost.” This is exactly right. The agent frameworks are becoming model-agnostic, which means models have to compete purely on capability and efficiency rather than ecosystem lock-in. Gemma 4’s Apache 2.0 license positions it well for this world.

One last thing worth mentioning — and this always happens, like clockwork: within a week of release, someone had published an uncensored variant with 482 likes on Hugging Face. Multiple users had noticed inconsistencies in how the model handles sensitive topics, being more willing to engage critically with some subjects than others. The speed at which “jailbroken” fine-tunes appear for every major model tells you something about the gap between how labs think users want to interact with models and how they actually do.23

Where Does This Leave Us?

The MoE variant is the headline for me. If the efficiency claims hold up under community scrutiny — and early signs are promising — then we’re looking at a model that could genuinely run useful, intelligent AI on consumer hardware without meaningful quality compromises for most tasks. That’s been the promise of open-weight models since LLaMA first leaked, and we’re closer to delivering on it than ever before.

What I’m watching for in the coming months: independent benchmark verification (especially on reasoning tasks with thinking enabled), fine-tuning recipes for the MoE variant, long-context performance analysis beyond Google’s reported numbers, and — perhaps most importantly — how Gemma 4 performs on real-world tasks that aren’t captured by any benchmark. The models that win in practice aren’t always the ones that win on MMLU. They’re the ones that reliably follow instructions, produce useful code on the first try, handle edge cases gracefully, and don’t hallucinate when you need them to be accurate. On these soft metrics, a month of community usage isn’t enough to draw conclusions.

But I’ll say this: I’ve been running the MoE variant as my daily driver for local AI tasks for three weeks now, and I haven’t felt the urge to switch back to a larger model. For a 3.8B-active-parameter model running on a laptop, that’s remarkable. Google set out to make their open models competitive with the best in the world, and with Gemma 4, they’ve done it. The question now is whether they can maintain this pace when DeepSeek, Meta, and Qwen inevitably respond. If the last two years of the open-model race have taught us anything, it’s that nobody stays on top for long — and that’s exactly how it should be.

Model: Gemma 4 — Google DeepMind. Released April 2, 2026. Hugging Face | Launch Blog | Documentation | GitHub. Apache 2.0 license.