Ornith 1.5 9B and 35B-A3B Benchmark on RTX 5090

ornith-ai published two models on Hugging Face: Ornith-1.5-9B-GGUF and Ornith-1.5-35B-A3B-GGUF. The 35B card claims it beats Gemma 4-31B and Muse Glimmer 30B by wide margins on agentic coding.

In this blog, we will run both Ornith models against those two rivals plus Qwen 3.8 27B on a single RTX 5090, across nine measurements. We will also find out why the context length printed on every one of those model cards is not the context length you actually get.

Setup: One Card, One Runtime, Five Models

Everything below ran on one machine in one session, so no number here is being compared against something measured on a different day.

• GPU: RTX 5090, 32 GB VRAM, full GPU offload
• Runtime: llama-server build 10448 on Windows 11
• Quantization: Q4_K_M for all five models, so quant class is never a confound
• KV cache: f16 with flash attention on, except where we sweep that on purpose
• Grading: pure Python functions, no judge model anywhere

Every model was started the same way.

llama-server.exe -m Ornith-1.5-35B-Q4_K_M.gguf --host 127.0.0.1 --port 8099 -c 8192 -ngl 999 -fa on -np 1 --jinja --no-warmup --metrics

For decode speed we used the raw /completion endpoint with ignore_eos, so every model produced exactly 512 tokens. That removes the most common way a benchmark lies to itself, which is a model stopping early and looking fast.

GGUF Teardown: Ornith Is Qwen 3.5 Underneath

Neither Ornith card publishes a layer count. Both talk about an end-to-end self-improvement loop instead. So we opened the files and read the headers.

from gguf import GGUFReader

r = GGUFReader("Ornith-1.5-35B-Q4_K_M.gguf")
for field in r.fields.values():
    if "architecture" in field.name or "expert" in field.name:
        print(field.name, "=", field.contents())

general.architecture = qwen35moe
qwen35moe.expert_count = 256
qwen35moe.expert_used_count = 8
qwen35moe.expert_feed_forward_length = 512
qwen35moe.expert_shared_feed_forward_length = 512

The 9B reports qwen35 and the 35B reports qwen35moe. Both are Qwen 3.5 with different weights. Let me tabulate the whole field for your better understanding.

Here, we can see that there is no new architecture in Ornith 1.5. Whatever these models are worth comes from post-training, not from a structural idea.

The 35B is an unusually fine-grained mixture. It hangs 256 experts of width 512 off a residual stream only 2,048 wide, and activates 8 of them plus a shared expert per token. That works out to roughly 3B parameters touched per forward pass.

Decode Speed: The Mixture Wins by a Wide Margin

Now the first real measurement. Five models, 512 tokens each, five content types, speculative decoding off.

Ornith 35B-A3B runs at 272 tokens a second. That is 3.5 times Qwen 3.8 27B and 42% faster than the Ornith 9B, while being four times the file size on disk.

In simple words, decode speed on a single card is set by how many parameters the model reads per token, not by how many it stores.

One more thing worth noting from these runs. The spread across counting, code, JSON, essay and fiction was between 0.3% and 1.4% for every model. With speculative decoding off, decode speed does not care what you are generating.

Prompt Reading Speed, and the Cost of Being a Mixture

Decode speed is only half of what you feel. The other half is how fast the model reads your prompt.

Here, we can see the one real weakness of the mixture model. At a 512 token prompt it reads at 1,223 tokens a second, against 7,493 at 8k. Routing 256 experts carries a fixed cost on every forward pass, and on a short prompt there are not enough tokens to spread that cost over.

So for short interactive turns the 9B is roughly five times faster to first token, and the throughput charts hide that completely.

KV Cache Is the Real Divider

The KV cache is the memory the model keeps for every token already in context. It is the number that decides how much context your card can hold, and it varies far more between these models than their file sizes do.

Every qwen35 model in this test caches KV on only a quarter of its layers. The Ornith 9B keeps 8 of 32, the 35B keeps 10 of 40, and Qwen 3.8 keeps 16 of 64. None of the three model cards mentions this.

Gemma and Muse both use sliding window attention, and llama-server allocates two caches for each of them.

llama_kv_cache: size = 5120.00 MiB ( 65536 cells, 10 layers, 1/1 seqs), K (f16): 2560.00 MiB, V (f16): 2560.00 MiB
llama_kv_cache: size = 1200.00 MiB (  1536 cells, 50 layers, 1/1 seqs), K (f16):  600.00 MiB, V (f16):  600.00 MiB

That is Gemma at 64k context. Its 50 windowed layers are capped at 1,536 cells and cost a flat 1,200 MiB no matter how long the context gets. But its 10 full attention layers have a head dimension of 512, and those alone cost 5,120 MiB. The windowing saves it from being far worse, and it still ends up the most expensive model per token in the group.

Quantizing the KV cache behaves identically for all five models, so it is a uniform lever rather than a per-model tuning knob.

Context Ceilings Are Cliffs, Not Slopes

This is the part that surprised us, and it is the reason this article exists.

All five models advertise 262,144 tokens. All five of them start at 262,144 and report it back cleanly.

curl -s http://127.0.0.1:8099/props | python -c "import json,sys; print(json.load(sys.stdin)['default_generation_settings']['n_ctx'])"

262144

That number is worthless, and here is why.

We caught it in Gemma's own memory numbers. As we raised the context from 155,648 to 262,144, its KV cache grew by 8.3 GiB while measured VRAM never moved off 31.5 GiB.

Here, we can see that the last 8 GiB of cache went somewhere other than the GPU, and nothing told us.

So "does it load" has no failure point on this platform, and asking that question gives an answer that is always yes. Instead we measured the point where decode speed falls off, using the model's own short-context rate as the reference.

def usable_ceiling(model, floor_fraction=0.90):
    """Largest context where decode stays within 90% of the small-context rate."""
    reference = decode_at(model, ctx=8192)
    floor = reference * floor_fraction
    ctx, best = 65536, 8192
    while ctx <= 262144:
        if decode_at(model, ctx) >= floor:
            best, ctx = ctx, ctx * 2
        else:
            break
    return best

The drop is a cliff, not a gentle decline.

Qwen 3.8 holds 76.9 tokens a second at 245,760 and collapses to 2.9 at 262,144. A 7% increase in context costs 96% of the throughput. Gemma loses 59% of its speed for 5% more context, and 44% of its advertised window is simply not available.

Ornith 35B-A3B is the only large model here that delivers its full advertised window at full speed, and it does it with almost 6 GiB to spare, because 20 KiB per token across 262k tokens is only 5 GiB of cache.

Long Context in Practice

Next we placed a fact inside documents of 16k, 64k and 128k tokens, at 10% and 90% depth, and asked for it back. All five models found every needle at every depth, 6 out of 6 each. Retrieval accuracy is not what separates them.

Gemma takes three and a half minutes to answer what Ornith answers in 26 seconds. Part of that is the spill we just described, because at the 139,264 context this phase used, Gemma was already past its cliff.

Muse Glimmer deserves a mention here. Its prefill speed decays only 20% from 16k to 128k, against 46% for Qwen 3.8, which is exactly what sliding window attention is supposed to buy.

The MTP Head Nobody Documented

While reading the headers we found something neither Ornith card mentions. Both GGUF files carry a nextn block, which is a multi-token prediction head used for speculative decoding.

blk.40.nextn.eh_proj.weight [4096, 2048] Q4_K
blk.40.nextn.enorm.weight [2048] F32
blk.40.nextn.hnorm.weight [2048] F32
blk.40.nextn.shared_head_norm.weight [2048] F32

llama-server ignores those tensors and prints them as unused unless we ask for them.

llama-server.exe -m Ornith-1.5-35B-Q4_K_M.gguf -c 8192 -ngl 999 -fa on --jinja --spec-type draft-mtp --spec-draft-n-max 4

So, here comes the catch. It helps on some content and actively hurts on the rest.

Acceptance rate explains all of it. On structured output the draft head guesses the next tokens correctly 63% to 86% of the time. On prose that falls to about 30%, and below roughly 40% the cost of drafting exceeds what it saves.

Qwen 3.8's head is clearly better trained than either Ornith head, reaching 2.36 times on JSON where the Ornith 35B manages 1.16.

Concurrency: Where the Mixture Pulls Away

Single stream speed is not the whole story if you are serving more than one request.

Ornith 35B-A3B reaches 669 tokens a second across 8 slots, more than double the best non-Ornith model.

Notice that the two Ornith models have the worst relative scaling, at about 2.7 times, while Muse Glimmer scales best at 3.7 times. That is only because the Ornith models start from such a high single stream baseline, and Muse still ends up at less than half the absolute throughput.

Gemma goes backwards between 4 slots and 8. That is the third separate symptom of the same problem: on a 32 GB card this model is memory bound, so every axis that adds memory pressure breaks it.

Accuracy and Thinking Economics

We ran 12 reasoning problems with objectively checkable answers, graded by a function that extracts the final ANSWER: line. No judge model.

Out of 40 runs across five models, 39 were correct. The only miss was Qwen 3.8 on one hard problem. Nothing hit its token cap.

Since accuracy is saturated, the useful question becomes what a correct answer costs.

Two things are worth pulling out of that table.

Token count and wall clock point in opposite directions. Qwen 3.8 is the most economical model in the group at 1,420 median tokens, and it still takes 2.4 times longer than Ornith 35B to produce an answer, purely because it decodes at 77 tokens a second instead of 272. Quoting either number on its own would give the wrong impression.

The Ornith models also think proportionally less. The 35B spends 70% of its output inside the think block, against 95% for Muse Glimmer and 89% for Qwen 3.8. It commits to an answer sooner.

The 9B is the interesting counter case. It reaches the same 8 out of 8 by thinking harder, 3,481 tokens against its larger sibling's 2,110, and pays 18.1 seconds instead of 7.8 for the identical score.

Agentic Tool Use: Everything Tied

Last, we built a tool-calling test with a virtual database and a small source tree, 10 tasks run at two seeds each. The tools return fixed data, and nothing executes model-written code.

The hard tier included an ambiguous name lookup that punishes guessing, a manager chain that needs five chained calls, and a proposed bug fix that looks right and is not.

Everything solved everything, with zero malformed tool calls in 100 runs.

Cost separates them again. Same 18 out of 18, and Ornith 35B finishes a task in 4.9 seconds while Muse Glimmer takes 23.4 and spends 6,192 characters thinking to Ornith's 2,256.

We also had to throw out one of our own tasks. It ended with "report the timezone of that region", which reads as asking for the timezone alone. All five models answered UTC+1, and our grader wanted the datacenter too, so it marked all five wrong. That was an ambiguous prompt on our side, not a model failure, so we excluded it and fixed the wording.

Which Model Should We Run

Ornith 1.5 35B-A3B is the model to run on a 32 GB card. Fastest decode, fastest correct answer, highest throughput under load, and the only large model that holds a real 262k context. The two prices are 19.7 GiB of VRAM and slow prefill on very short prompts.

Ornith 1.5 9B is the efficiency winner at 36.7 tokens a second per gibibyte, roughly seven times Gemma's rate, and it holds the full context window in 14.6 GiB.

Gemma 4-31B is a poor fit for this hardware. Not because of its answers, which matched everyone, but because 80 KiB of KV per token breaks it on every axis that adds memory pressure.

What We Did Not Test

Being clear about the gaps matters more than a tidy conclusion.

• Vision. Both Ornith repositories ship an mmproj file and their templates carry image tokens. We tested text only.
• Quality cost of KV quantization. We measured the sizes at q8_0 and q4_0, not what they do to output.
• Quality cost of MTP. We proved the output changes, not whether it gets worse.
• Agentic coding at SWE-bench scale, which is where the card's main claim lives.

This is how a local benchmark works: measure what the hardware actually does, name the number the model card leaves out, and say plainly which questions you could not answer.