Qwen 3.8 27B vs Nemotron 3.5 vs Muse Glimmer on an RTX 5090

Easy questions cannot separate local models any more. Ask a modern 27B model to write a palindrome checker and all of them get it right, so the score tells us nothing.

In this blog, we will push three of them until they break. We built sixteen hard problems where every answer can be checked exactly, ran each model alone on one RTX 5090, and measured not just who was correct but how each one failed.

Three Different Bets on the Same Size Class

The three models we tested are close in size and very different inside.

Nemotron is the odd one out. It is a sparse mixture of experts, the biggest on paper at 32.9B, and it carries a 1M token context window. It also has the narrowest embedding of the three at 2,688. It pays for that ambition in memory: 25.4GB on the card while running, against roughly 17GB for the other two.

Muse Glimmer is a dense 27.9B model with the widest embedding at 6,656. It is the only one of the three that does not expose a separate thinking field, so whatever reasoning it does comes out inline in the answer.

Qwen 3.8 sits in the middle. It is a dense 27.3B model with a 256K context and a vision projector built in. It is the smallest of the three and it still takes image input, which Nemotron cannot do at all.

Our Setup

We kept the hardware and settings identical for every model so the numbers stay comparable.

• GPU: RTX 5090, 32GB VRAM
• Runner: Ollama 0.32.12 on Windows 11
• Context window: 65,536 tokens for every task
• Temperature: 0.2, one run per task
• Quantization: Q4_K_M for all three models
• Generation cap: 32,768 tokens

One rule mattered more than the rest. Before any model loaded, every other model was unloaded and the script polled Ollama until the GPU reported nothing resident. No model ever shared VRAM with another, so every speed number is a clean solo run.

The Sixteen Problems

Every problem has one exact answer that a script can check. We grouped them into three families.

Math has six problems: the digit sum of 1 + 11 + 111 and so on out to 300 ones, the last three digits of the power tower 7^(7^7), the expected number of rolls to get three sixes in a row, domino tilings of a 4 by 10 board, arrangements of MISSISSIPPI with no two S letters touching, and an expected value draw problem.

Code has six problems: an expression evaluator that handles **, //, and % with correct right associativity and no eval, counting inversions in O(n log n), an O(1) LRU cache, fixing a closure late binding bug, comparing version strings, and predicting the exact output of a tricky Python snippet.

Reasoning has four problems: a 5 by 5 logic grid, a formatting task where six sentences must spell PYTHON and each must be exactly seven words, a JSON conversion with a discount and GST to compute, and a 64K document with three chained facts and deliberate decoy notes.

Nothing here was graded by eye. Code answers ran in a subprocess against test cases, with size and time gates so a correct but slow O(n squared) answer still fails. We also checked the graders against known wrong answers before the run, including a check that they reject an inversion counter that is too slow and an LRU cache that never tracks recency.

We did not trust our own arithmetic either. Every expected answer was computed by a separate script. The 4 by 10 tiling count came out as 18,061, which matches OEIS sequence A005178. That number matters later.

The Scoreboard

Muse Glimmer takes the top score, and the other two tie behind it.

Here, we can see a one task gap between first and second. That gap is the least interesting thing in the whole run, because a tie hides the fact that these models failed in completely opposite ways.

The matrix uses three states instead of two. A green check means solved, a red cross means a wrong answer, and the third symbol means the model was still reasoning when it hit the token ceiling. That third state is the one most benchmarks throw away.

Qwen 3.8 Is the Efficiency Pick

Qwen finished all sixteen problems in 6.3 minutes. Nemotron took 8.3 minutes. Muse Glimmer took 16.2 minutes, more than two and a half times longer than Qwen, and it bought one extra correct answer.

Qwen also got there on fewer tokens, 59,524 against 75,197 for Glimmer and 102,736 for Nemotron. It thought the least as well, 23,812 words against 43,487 for Nemotron.

The last panel is the one that decides it for us. Qwen 3.8 was the only model that never ran out of generation budget. Nemotron hit the ceiling twice and Glimmer hit it once. A model that stalls gives you nothing at all, and you have already paid for the wait.

Where the Speed Shows Up

On the thirteen problems that all three models solved, Qwen finished first on eight of them.

The typical case is the number worth remembering. The median task took Qwen 5.9 seconds, against 10.6 for Nemotron and 14.5 for Glimmer. On a normal request you wait about half as long for Qwen as for Nemotron.

Let me tabulate the eight wins for your better understanding.

Look at what kind of work those are. Four of the eight wins are Code tasks and three are Reasoning tasks. Only one is Math. Now look at the five it lost: three of them are Math. Give Qwen something specified, such as write this function, fix this bug, follow this format, produce this JSON, and it is both fastest and correct. Give it an open ended derivation and it slows down and starts making mistakes.

Where Qwen loses, it loses badly. The expected value draw problem took 32.1 seconds against 10.6 for Nemotron. The 64K three hop needle took 28.9 seconds against 10.6. Long derivations and long context retrieval are where the Nemotron design earns its 25.4GB.

Now comes the honest part, and it is more interesting than a clean win. If we add up those thirteen shared tasks, Nemotron is actually ahead at 130.2 seconds against 140.1 for Qwen. Qwen's eight wins are worth 28.1 seconds put together. Its five losses cost 51.3 seconds. It wins many small races and loses a few big ones.

So where does the 6.3 versus 8.3 minute lead over the whole suite come from? It comes from the three tasks that are not in that table. Nemotron spent 310.9 seconds, more than five minutes, on its two capped tasks and produced nothing usable from either. That is the entire margin and more.

This is the real argument. Qwen 3.8 is not reliably faster per task than Nemotron. It is faster to finish, because it never spends five minutes producing an empty reply.

The Trap That Did Not Work

Our favourite problem in the suite asks the model to predict exactly what a Python snippet prints. The important line is this one.

g = gen()
print(list(zip(g, g)))

Both arguments to zip are the same generator. So zip pulls 1 and 2 for the first pair, then asks for a third value, gets 3, asks for a fourth, and the generator is empty. The half built pair is thrown away. The answer is [(1, 2)], not a two pair list.

You cannot pattern match your way through that one, you have to trace the execution order. All three models got it right, which surprised us. Qwen 3.8 got it in 4.4 seconds against 10.0 for Glimmer and 23.3 for Nemotron.

Where Qwen 3.8 Still Falls Down

We are not going to skip the two misses, because they are the most useful part of the run. Both of Qwen's failures were wrong answers, not timeouts. It finished comfortably inside its budget and was confidently incorrect.

On the repunit digit sum it answered 1229. The correct answer is 1230. It was off by one after 27,179 tokens of work.

The tiling task is the one that should worry us. Qwen answered 17305, then wrote that this matches the known sequence for 4 by n domino tilings, OEIS A005178. It does not. That sequence runs 2245, 6336, 18061. Qwen's own working table has a(4) = 35 where the real sequence has 36, so its recurrence drifted from the fourth term onward and everything after it was wrong. It then cited a real, checkable sequence as support for a number that sequence does not contain.

Running Out of Room Is a Real Failure Mode

Three of the five total failures across all three models were the generation cap, not bad reasoning.

The category chart also shows where the suite still separates these models. On Code and Reasoning they are almost identical. Math is the only family that pulled them apart.

That raises an obvious objection about the cap. Maybe our ceiling was simply too low. So after the scored run we went back and re-ran only the three capped tasks with the ceiling raised to 60,000 tokens, nearly double.

It rescued none of them. The Nemotron repunit task and the Glimmer tiling task both burned all 60,000 tokens without finishing. That is not a ceiling set slightly too low, that is runaway reasoning. So when Qwen answers in 4 seconds and Glimmer is still going after two minutes, the extra time is not always buying accuracy.

What Each Model Costs to Run

Four numbers tell us what we are paying for in day to day use.

Nemotron generates fastest at 216 tokens per second and still finishes behind Qwen, because it generates so much more. Glimmer has the opposite problem. On the shared tasks it produced the fewest tokens of the three, so its 16.2 minutes is not caused by writing too much, it is raw generation speed at 81 tokens per second.

Memory is the one place Qwen does not lead. Glimmer is lightest at 16.9GB, Qwen sits at 17.5GB, and Nemotron is a long way back at 25.4GB. On a 32GB card that gap is the difference between comfortable and tight.

Conclusion

We built sixteen hard problems with exact answers, ran Qwen 3.8 27B, Nemotron 3.5 Lightning, and Muse Glimmer alone on one RTX 5090, and checked every code answer by running it. Muse Glimmer scored highest at 15 of 16, while Qwen 3.8 and Nemotron tied at 14. Qwen finished the suite in 6.3 minutes and was the only model that never ran out of generation budget.

Key takeaways:

• Qwen 3.8 27B is the practical daily pick. It is fast on short specified work, light on VRAM, and it never stalls.
• Muse Glimmer is the accuracy pick. It scored highest and never gave a wrong answer, but it costs 16.2 minutes for the same suite.
• Nemotron 3.5 Lightning is the long context pick. It won the 64K retrieval task by a wide margin, and it needs 25.4GB to do it.
• A token cap turns into a silent failure. Recording why generation stopped is what separates a model that cannot solve a task from one that simply ran out of room.
• Speed is not free. Qwen produced two confident wrong answers, one of them supported by a citation that does not say what the model claimed.

Next steps:

• Read our earlier head to head, Nemotron 3.5 vs Muse Glimmer 30B, to see how these two behave on everyday tasks instead of hard ones.
• Set your own machine up with the Ollama setup guide before running any of these models locally.
• Use the local LLMs technical reference guide to match a quantization and context size to your own GPU.

This is how a small local benchmark tells us more than a score table: by recording not only what each model answered, but how it failed.