Ornith 1.5 9B and 35B-A3B Benchmark on RTX 5090

ornith-ai released two models on Hugging Face: Ornith-1.5-9B-GGUF and Ornith-1.5-35B-A3B-GGUF. The cards promise a huge 262,144 token context. But they never say how the models are built inside, and that is what decides whether the context really fits on our card. So we opened the files and measured everything ourselves.

In this blog, we will measure both models on one RTX 5090. We also run two rival models on the same card for comparison. Gemma 4-26B-A4B is a mixture of experts, like Ornith 35B-A3B. In simple words, a mixture of experts keeps many small expert networks inside one model and wakes up only a few of them per token. This design is also called sparse, because most of the model stays asleep on every token. Qwen 3.8 27B uses the same architecture as Ornith, without the mixture. Between them we can tell what comes from the mixture design and what comes from being Ornith.

We open the GGUF files first. Then we measure speed, memory and context. Then quality and tool use.

Setup: Four Models, One RTX 5090

Everything ran on one machine in one session.

• GPU: RTX 5090, 32 GB VRAM, full GPU offload
• Runtime: llama-server build 10448 on Windows 11
• Quantization: Q4_K_M for all four models
• KV cache: f16 with flash attention on
• Grading: pure Python functions, no judge model, meaning no AI grading another AI

Every model started the same way.

llama-server.exe -m Ornith-1.5-35B-Q4_K_M.gguf --host 127.0.0.1 --port 8099 -c 8192 -ngl 999 -fa on -np 1 --jinja --no-warmup --metrics

Decode tests used the raw /completion endpoint with ignore_eos, so the model cannot stop early. Every model produced exactly 512 tokens.

What Is Inside the Ornith GGUF Files?

A GGUF file is how a model is stored on disk. Its header lists every tensor inside the model, so we can count the blocks ourselves instead of trusting the card. We read all four files.

Ornith 9B reports the architecture qwen35. Ornith 35B-A3B reports qwen35moe. Both sit on the Qwen 3.5 stack, so any gain comes from training rather than structure.

The header also names 256 experts, 8 of them active per token.

Grouping the tensors by block index shows something the cards never mention. Most Ornith blocks carry no attention weights at all. In the 35B, attention sits at block 3, 7, 11 and every fourth position after that.

Here, we can see the schedule. Ornith places one attention block at every fourth position. The three blocks in between are Gated DeltaNet. Those carry ssm_alpha, ssm_beta and ssm_out instead of key and value projections.

A Gated DeltaNet block keeps a fixed size running state. That state does not grow as context grows. Only attention blocks hold a KV cache.

Gemma solves the same problem differently. All 30 of its blocks are attention blocks, but 25 use a 1,024 token sliding window.

Let me tabulate all four models for your better understanding.

Here, Residual is the width of the model's running hidden state, the workspace every block writes into. Ornith is the finer mixture, with 256 experts against Gemma's 128. Both activate 8 per token. Qwen 3.8 reads all 27B every time.

Which Model Writes Tokens Fastest?

Decode speed is how fast a model writes its answer, in tokens per second. The block layout above gives us a prediction. If speed follows active parameters, both mixtures should beat Qwen by a wide margin.

The prediction holds across both vendors. Ornith leads Gemma by 18%. Against the dense Qwen the gap is 3.5 times.

So speed here belongs to sparsity, not to Ornith.

Content made almost no difference. The spread across counting, code, JSON, essay and fiction stayed between 0.3% and 1.5% for every model.

How Fast Does Each Model Read a Long Prompt?

Prefill is how fast a model reads the prompt before answering. We measured it from 512 tokens to 32k.

All numbers in the table are tokens per second.

Both mixtures are slow on very short prompts. Picking which experts to use costs a fixed amount on every step. A 512 token prompt is too short to absorb that cost.

From 2k upward Gemma overtakes Ornith 35B and stays ahead. On long prompts and short answers, Gemma is the faster mixture.

How Much Memory Does the KV Cache Cost?

The KV cache holds one entry per token already in context. It decides how much context a card can hold.

Ornith and Gemma land on the same number by different routes. Ornith skips the cache on three blocks in four. Gemma keeps 25 of its 30 blocks inside a 1,024 token window.

The Gated DeltaNet schedule is a real win over Qwen 3.8, which shares Ornith's family and still costs 64 KiB. It is not a win over Gemma.

We can also store the cache in smaller numbers to save memory. That is KV quantization, and it behaved identically for all four models.

How Much Context Can Each Model Really Use?

All four models advertise 262,144 tokens. All four start at that size and report it back through the server. So the number a server reports proves nothing.

So we measured the point where decode speed drops instead. The ceiling below is the largest context where a model still runs within 90% of its own short context rate.

Qwen 3.8 holds 78 tok/s at 245,760 and drops to 7.3 at 262,144. A 7% larger window costs it 91% of its speed. Its 64 KiB per token puts it at the edge of the card.

The other three stay flat to the full window. Gemma does it with the most room to spare.

How Fast Can Each Model Answer From a 128k Document?

We hid a fact inside documents of 16k, 64k and 128k tokens. Then we asked for it back.

Ornith 35B and Gemma finish about half a second apart. Qwen takes three times longer.

Every model found the hidden fact every time. Retrieval accuracy separated nothing.

Does the MTP Draft Head Make These Models Faster?

Both Ornith files carry a nextn block. That is a multi-token prediction head, or MTP. In simple words, a small helper inside the file guesses the next few tokens, and the full model checks all the guesses in one go. Neither Ornith card mentions it. Qwen 3.8 has one too. Gemma does not. We tuned this same mechanism flag by flag in the Qwen 3.8 27B speed settings post.

llama-server ignores those tensors unless we ask for them.

llama-server.exe -m Ornith-1.5-35B-Q4_K_M.gguf -c 8192 -ngl 999 -fa on --jinja --spec-type draft-mtp --spec-draft-n-max 4

Here is the catch. MTP helps on structured text and hurts on prose.

Acceptance rate explains it. The draft head guesses right 43% to 86% of the time on structured output. On prose that falls to 26% to 39%, and drafting stops paying for itself.

This one goes to Qwen 3.8. Its head reaches 2.34x on JSON where Ornith 35B manages 1.10x. It also stays above 1.0 on prose, where both Ornith models turn slower.

Which Model Serves Many Users at Once?

We fired 1, 4 and 8 requests in parallel and measured total throughput. That is the tokens per second across all requests added together.

Ornith leads on a single stream. Gemma overtakes it at four slots and finishes 4% ahead at eight. Gemma's weights take less VRAM, which leaves more memory free for the extra caches that parallel requests need.

For serving many users at once, Gemma is the faster mixture.

Which Model Answers Hard Problems Fastest?

Speed means nothing if the fast models answer worse. We ran 12 problems with checkable answers: a base tier of eight problems run once, and a hard tier of four problems run at two seeds each. A function grades every run by reading the final ANSWER: line.

Accuracy separated almost nothing. Three models got a perfect score on both tiers. Qwen 3.8 missed one of its two runs on one hard math problem, and it answered the same problem correctly at the other seed. The table below shows the hard tier, eight scored runs per model, because it is the only place the models had to work.

So the real question is what an answer costs.

Gemma spends 9,759 tokens where Ornith spends 2,110. That is 4.6 times more thinking for the same score. It turns Gemma's prefill and concurrency wins into a 43.6 second answer.

This is the clearest Ornith result in the benchmark. It comes from training, not from architecture.

Ornith also thinks proportionally less, at 70% of output against 89% for both rivals. It commits to an answer sooner.

Qwen 3.8 spends the fewest tokens of anyone at 1,420. It still takes 19.2 seconds, because it decodes at 77 tok/s. Token count and wall clock point in opposite directions.

How Well Does Each Model Use Tools?

We built a tool-calling test with a virtual database and a small source tree. Ten tasks ran at two seeds each. In simple words, we ran every task twice, each time with a different random seed. The tools return fixed data, and nothing executes model-written code.

One of our own tasks was badly worded. It ended with "report the timezone of that region", which reads as asking for the timezone alone. Every model answered UTC+1 and our grader wanted the datacenter too. We excluded it and fixed the wording. That leaves 18 scored runs per model.

Both Ornith models solved everything. Gemma missed one. Nobody produced a malformed tool call in 72 runs.

The thinking pattern repeats. Gemma spends 5,573 characters per task against Ornith's 2,230.

What Does Ornith's Model Card Claim?

Let's check every claim on the card against what we measured.

Which Model Should We Run?

Ornith 35B-A3B and Gemma 4-26B-A4B are near twins on hardware. Same KV cost, same full window, about half a second apart on a 128k question. Gemma is lighter on VRAM, faster on prompts above 2k, and faster under eight-way load.

Training separates them, not silicon. Ornith reaches the same answers in a fifth of the time because it does not keep re-checking itself. For question answering and agent loops, take Ornith. For long prompt processing or many users, take Gemma and save 3.3 GiB.

Ornith 9B is the efficiency pick. It holds a full 262k window in 15.4 GiB and reads short prompts faster than either mixture.

Qwen 3.8 27B is the slowest here, and the only model that cannot hold its own advertised context. It keeps the best MTP head and the lowest token count per answer.

What We Did Not Test

• Vision. Both Ornith repositories ship an mmproj file. We tested text only.
• Quality cost of KV quantization. We measured the sizes, not the output.
• Quality cost of MTP. We proved the output changes, not that it gets worse.
• Agentic coding at SWE-bench scale, where Ornith's main claim lives.

Conclusion

We opened the GGUF headers of Ornith 1.5 9B and 35B-A3B, then ran them against Gemma 4-26B-A4B and Qwen 3.8 27B on one RTX 5090. The two mixtures came out nearly identical on hardware: same 20 KiB KV cost per token and the same full 262k context at full speed. What separates them is training. Ornith reaches the same answers as Gemma in a fifth of the time because it stops thinking sooner.

Key takeaways:

• Decode speed follows active parameters, not total size. Both mixtures beat the dense Qwen 3.8 by a wide margin, so the speed belongs to sparsity, not to Ornith.
• Ornith 35B-A3B and Gemma 4-26B-A4B land on the same KV cache cost by different routes: skipped cache blocks against sliding windows.
• On Windows, "the context loads" proves nothing. Measure where decode speed falls off a cliff instead.
• Ornith's real edge is training. It answers hard problems in 8 seconds where Gemma needs 43.6, at the same accuracy.
• MTP speeds up structured output and can slow down prose. Qwen 3.8 carries the best head of the three.

Next steps:

• See these same models face IBM's latest release in the Granite 4.2 vs Gemma 4 vs Qwen 3.8 benchmark.
• Tune draft depth, KV cache type, and context size on your own card with the Qwen 3.8 27B speed settings on llama.cpp.
• Match a quantization and a context size to your GPU with the local LLMs technical reference guide.

This is how the Ornith 1.5 release measures up. We read the files first and found the hybrid layout the cards never mention. We measured speed, memory, and context, and saw the two mixtures land as near twins on silicon. And we finished with the training difference that decides the winner.