Qwen 3.8 27B vs Qwen 3.6 27B vs Qwen 3.5 27B Teardown

Qwen shipped three 27B models in about six months: Qwen 3.5 27B in February, Qwen 3.6 27B in April, and Qwen 3.8 27B in August. Each release came with bigger benchmark numbers. A simple question is left unanswered by all of them: did the architecture change, or did the training change?

A GGUF file is how Ollama stores a model on disk. In simple words, it starts with a header that lists every tensor in the model, with its name, its shape, and its data type.

In this blog, we will read those headers for all three releases straight off our own disk, and then run 397 measured generations on one RTX 5090 to see where the gains actually live.

Setup: One RTX 5090 and Ollama 0.32.14

Everything below ran on one machine in one serving session.

• GPU: RTX 5090, 32GB VRAM, power capped at 600W
• Runtime: Ollama 0.32.14 on Windows 11
• Quantization: Q4_K_M for all three models, straight from the official Ollama library
• Runs: 397 measured generations
• Controls: forward and reverse model order, a discarded warmup after every load, and a full VRAM unload between models

Speculative decoding was pinned by hand on every run. We never let it fall back to each model's own default, because those defaults are not the same, and that turns out to matter a lot.

GGUF Teardown: Qwen 3.8 27B vs Qwen 3.6 27B vs Qwen 3.5 27B

We wrote a small parser and read the GGUF headers of all three blobs. No downloads, and no trusting anyone's spec sheet.

Here, we can see that Qwen 3.5 27B and Qwen 3.6 27B are structurally identical. Same 1,307 tensors, same names, same shapes, same data types, and the parameter counts match to the last digit at 27,781,427,952. If we deleted the weights and kept only the blueprints, we could not tell the two apart.

Qwen 3.8 27B keeps the same backbone. Same 64 blocks, same 5,120 hidden size, same 24 query heads and 4 key value heads, same 3 to 1 mix of linear attention and full attention, same 248,320 vocabulary, same 262,144 context. Two things moved. The vision encoder now lives in a separate 461M file, and the multi token prediction head became a real, active 65th block.

The surprise is that the draft head was always there. Qwen 3.5 27B and Qwen 3.6 27B carry the same mtp.* tensors in their files, sitting unused. Qwen did not add a speed feature in 3.8. They fixed one they had been shipping dormant all along.

There is a runtime check on top of the file check. With speculative decoding switched off, all three models decode within 1.5% of each other, 78.6 to 80.4 tok/s at a 32k context, and their speed decays with context along the same curve. Identical compute graph, measured twice.

So when a Qwen 3.6 27B against Qwen 3.5 27B benchmark shows a gain, that gain is post-training by definition. There is nowhere else for it to live.

Qwen's Own Benchmark Numbers From 3.5 to 3.8

Before our own measurements, it is worth looking at the scores Qwen published themselves. They frame what each release was actually for.

These scores come from the official model cards, not from our benchmark. The two cards cross-check where they overlap. The 3.6 card and the 3.8 card list the same 3.6 numbers on their own: 53.5 on SWE-bench Pro, 83.9 on LiveCodeBench, and 87.8 on GPQA. So nobody quietly moved the goalposts between releases.

The shape of the trajectory is the story.

Knowledge and pure reasoning barely moved. GPQA Diamond went 85.5 to 87.8 to 89.2. That is 3.7 points in six months, from a start that was already high. Math says the same. The 3.6 card shows AIME26 at 92.6 against 94.1, and one HMMT split did not move at all, 84.3 to 84.3. These abilities were essentially finished in February.

Coding moved steadily. LiveCodeBench went 80.7 to 83.9 to 90.3. SWE-bench Pro went 51.2 to 53.5 to 61.7.

Agentic work exploded. Terminal-Bench went 41.6 to 59.3 to 73. The agentic benchmarks that appear only on the 3.8 card show the same pattern against 3.6.

Why did agency improve so much faster than knowledge? Our teardown gives the mechanical answer.

An agent benchmark is hundreds of small steps. Read something, decide, act, repeat. A model that burns 13,000 tokens re-checking every small decision is slow, expensive, and runs out of budget mid task. That is exactly the failure we caught live in our hard tier. A model that thinks in 1,850 tokens and commits, drafts its own output twice as fast, and never stalls is a far better worker. It is also the same machine, answering the same one shot science questions.

The release in between fits too. Qwen pitched 3.6 as an agentic coding release, and its card shows just that: Terminal-Bench up about 18 points, knowledge up 2.

So Qwen spent 2026 not making the 27B smarter, but making it a better worker. The teardown above proves they did it without touching the machine. The sections below measure the three habits they did change: how much it thinks, how it thinks, and how fast it produces tokens.

The Twelve Verified Problems

We tested two departments, math and reasoning, four problems each, plus a harder four problem tier once the first set turned out to be too easy for everybody.

Every problem has one exact answer, and we did not trust ourselves to know it. A separate script brute forces each one. It counts all 105 lcm pairs, walks all 1,024 coin sequences, tries every knight and knave assignment, and checks that each logic puzzle has exactly one solution. If a puzzle had two valid answers, the script would have caught it before any model saw it.

Models must end with a line like ANSWER: 807, and a string matcher grades it. No human judgement, and no model acting as a judge.

Accuracy and Thinking Tokens at Matched Settings

At identical settings for all three, with thinking on and speculative decoding off for everybody, they all solve almost everything.

The hard tier, with a 24,576 token budget, tells the same story more loudly.

That is a 7x cut in thinking tokens for the same accuracy, and it is why 3.8 finished the hard tier in 3.5 minutes while the other two each took over 23 minutes.

One detail matters more than the scores. Both of 3.6's misses were not wrong answers. They were runs where the model was still thinking when it hit the token budget and got cut off. Across all 397 runs, every single failure was that same event. No model ever reasoned its way to a wrong conclusion and stated it confidently.

Direction of Thinking: Where the Answer First Appears

We measured one simple thing in every thinking trace. At what point does the correct final answer first appear inside the model's own thinking?

Qwen 3.5 finds the right answer 12% of the way into its trace. It then spends the remaining 88% checking, re-checking, and second guessing an answer it already had. It says "wait" or "actually" about 14 times per problem. Qwen 3.6 is the same behaviour, mildly trimmed.

Qwen 3.8 finds the answer about halfway through a much shorter trace, verifies once, and stops. It averages 0.6 self-corrections per problem.

This is what a post-training improvement looks like in concrete terms. Same brain, different habits. The 3.5 to 3.6 step trimmed the old habit. The 3.6 to 3.8 step replaced it.

Reasoning Effort on Ollama vs the Official Chat Template

If you have read about Qwen 3.8 27B online, you may have seen the opposite complaint. Simon Willison found that it defaults to wildly overthinking, burning 22,276 thinking tokens on one SVG drawing, and a Hugging Face thread is titled around the model not being able to stop thinking. Our benchmark says it barely thinks at all. Who is wrong?

Nobody. We are measuring two different serving stacks, and we can point at exactly where the difference lives.

Qwen 3.8 27B has an official chat template with a reasoning_effort variable, and its default value is xhigh. LM Studio and llama-server --jinja apply that template, so their users get the trained deep reasoning regime.

Ollama never uses that variable. We read the renderer for this model family in Ollama 0.32.14 source, and what it actually does is prepend a plain English sentence to the system message:

• think:"low" injects a sentence asking the model to keep its thinking brief.
• think:"medium" injects nothing at all.
• think:"high" and think:"max" inject a sentence saying that effort is xhigh.
• think:true maps to "medium", which again means nothing is injected.

Then we measured what those sentences actually do, across 8 problems each: low gives 1,020 tokens, medium 1,115, high 600, and max 662. Every level answered all 8 correctly.

Read that again. The "max effort" setting produces the shortest thinking. The sentence works as a polite nudge, not as the trained deep thinking trigger. The model's real xhigh regime cannot be reached through Ollama at all.

Decode Speed by Content Type on Qwen 3.8 27B

Across our own past reports, Qwen 3.8 27B has measured 171 tok/s, then 125.5, then 96.5. When today's controlled runs said 79 with speculative decoding off, it looked like the benchmarks contradicted each other.

They do not. The explanation is speculative decoding, and specifically that its speedup is not a property of the model. It is a property of what the model is writing.

Here is the mechanism in plain words. Qwen ships a small draft head that guesses the next few tokens cheaply, and the full model then checks those guesses in one pass. When the guesses are right, we get several tokens for the price of one. When they are wrong, the check was wasted work. So the speedup depends entirely on how predictable the output is.

We tested that directly: same model, same settings, seven kinds of content.

One model, one machine: 205 tok/s counting to 200, 187 to 189 writing code, 176 writing JSON, 104 writing an essay, and 88 writing deliberately weird fiction. The draft head accepts 100% of its guesses on counting and 27% on surreal prose.

Now every historical number lands on one curve. The 171 was a code heavy suite at temperature 0.2 with drafting on. The 125.5 was a llama.cpp ladder at draft depth 2. The 96.5 to 104 is essay prose at roughly 0.36 acceptance. The 70 to 79 is drafting off. Four operating points of one mechanism.

If somebody quotes a tok/s figure without saying what the model was writing and whether speculative decoding was on, they have told us almost nothing.

Draft Acceptance: Qwen 3.8 27B vs Qwen 3.6 27B vs Qwen 3.5 27B

The same curve also holds the version story.

All three models carry the draft head, so we forced speculative decoding on for every one of them and measured the acceptance rate by content type.

Two things jump out of that table.

Qwen 3.5 27B and Qwen 3.6 27B have draft heads so weak on prose, 9% and 12% acceptance, that switching drafting on makes them about 25% slower than no drafting at all. They only profit on very predictable output.

Qwen 3.8 27B's head is better at every point on the curve, with 4x the acceptance on essays, and it never lost in this sweep. That head was retrained, and that retraining, not any architecture change, is why Qwen 3.8 27B is the fast release. It also explains why Ollama switches drafting on by default for that one model only.

Long Context to 80k Tokens

We filled each model's context with 16k, 48k, and then 80k tokens of relevant reference material, and then asked a question that the material genuinely helps with.

Nobody degraded by 80k on these tasks. Decoding slows from about 70 to about 59 tok/s as the window fills, and prompt reading slows from about 3,340 to 2,320 tok/s. The three lines sit on top of each other because it is the same machine underneath.

One behavioural note. Qwen 3.5 27B thinks about 15,000 characters no matter how much context it just read. Qwen 3.8 27B stays between 1.7k and 3.9k characters. The thinking policy does not react to context depth in either direction.

Our first attempt at this ladder sized the filler using a 4.0 characters per token estimate. The real ratios are 3.36 for the math filler and 4.75 for the reasoning filler, which pushed some cells past the context boundary where the server truncates or refuses. We archived that data, recalibrated against measured ratios, and reran the whole ladder. The token counts in the table above are the server's own numbers, not estimates.

Qwen's Recommended Settings vs Matched Settings

We ran one more pass with each model's officially recommended settings instead of our matched ones.

All three recommended settings failures were the same event on the same problem, computing 7^2025 mod 1000. Verifying modular arithmetic means writing similar looking lines over and over, and a presence penalty exists precisely to punish repetition. So the model gets pushed off its natural verification loop, wanders, and burns its whole token budget.

With the penalty at 0, the same model went 16/16. Qwen 3.5 27B's Ollama modelfile ships presence_penalty 1.5 as its default, so a stock ollama run qwen3.5 inherits this failure mode. Qwen 3.8 27B's modelfile sets it to 0.

Architecture vs Post-Training Attribution

Let me tabulate the attribution for your better understanding.

Benchmark Controls and Limits

A few controls are what make these numbers comparable at all.

• The GPU never lied about which model was loaded. Before every switch the previous model was fully unloaded, and the script polled until VRAM reported empty.
• Heat could not pick the winner. This card slows by about 7% once it is hot, enough to fake a result from run order alone. So every comparison ran twice, once in forward model order and once in reverse, with a throwaway warmup generation after each load. The measured order effect was under 1%.
• Speed toggles were pinned, not inherited. Ollama enables speculative decoding for 3.8 and not for 3.5 or 3.6, so we forced it off for all accuracy runs and tested it separately for all three.
• Answers were graded by string matching against brute forced ground truth, so no judgement call sits anywhere in the scoring path.
• Failures were investigated, not discarded. The context ladder was rerun after a calibration bug, and the discarded data is archived. Transient server errors were retried and logged, and no failed request was ever graded.

Two limits are worth stating. Accuracy cells are 2 samples each at temperature 1.0, which is plenty for a 7x token gap but not for a one problem difference, so Qwen 3.5 27B against Qwen 3.6 27B on accuracy is a tie within noise. Cross session drift on this rig is about 6%, so every comparison table here is paired inside one session.

Conclusion

We read the GGUF headers of Qwen 3.8 27B, Qwen 3.6 27B, and Qwen 3.5 27B, then ran 397 measured generations on one RTX 5090 through Ollama. The three releases are the same machine, and Qwen's own model cards point the same way: science reasoning gained 3.7 points in six months while agentic terminal work gained 31. Qwen 3.5 27B and Qwen 3.6 27B are identical tensor for tensor, and Qwen 3.8 27B keeps the same text backbone while activating a draft head that was already sitting in the older files.

Key takeaways:

• Qwen 3.8 27B is the default pick on Ollama. Same accuracy as its predecessors here, 6 to 9 times faster end to end, and 176 to 205 tok/s on structured output.
• Everything that improved is post-training. The thinking policy, the draft head quality, and the shipped defaults changed. The architecture did not.
• Leave speculative decoding on for Qwen 3.8 27B. For Qwen 3.5 27B and Qwen 3.6 27B it helps only on code and structured output, and it costs about 25% on prose.
• Use presence_penalty 0 in thinking mode for all three, including Qwen 3.5 27B, despite what its own model card recommends.
• A tok/s number without its content type and its drafting state is not a measurement. The same model legitimately reads 79 or 205 on the same GPU.

Next steps:

• Compare these models against other vendors in Qwen 3.8 27B vs Nemotron 3.5 vs Muse Glimmer.
• Check whether your model gives the same answer twice with our Qwen 3.8 vs Muse Glimmer vs Gemma 4 consistency test.
• Tune draft depth, KV cache type, and context size on your own card with the Qwen 3.8 27B speed settings on llama.cpp.

This is how a model teardown works. We read the files first and proved the architecture never changed, then measured thinking, speed, and long context on one machine, and found that six months of Qwen releases went into teaching the same 27B machine to think less, draft better, and waste nothing.