Analysing the same bid with GLM-5.3 Flash and a tiered Claude stack

We ran the same analysis twice: one tender, one bid, the same pipeline, the same prompts. The first run used Claude — Opus 4.8 to reason, Sonnet 4.6 to verify, Haiku 4.5 to read the requirements one by one. 2 hours 11 minutes, $36.13. The second used GLM-5.3 Flash for all three jobs, through OpenRouter. 10 hours 18 minutes, $4.82.

Both produced an analysis we would send to a bid manager, and both reached the same recommendation. What separates them is not judgement. It is the bill and it is reliability.

1. What we ran

A public tender in several lots, with a complete bid against it: 210 files, a contract value in the seven figures per year. The bid is invented — we wrote it ourselves and made it as close to a real submission as we could. We are not naming the buyer or the sector.

Both runs used the identical pipeline — indexing, a main reasoning agent, a coverage stage that checks each requirement, criterion and lot in parallel, a verification pass that re-reads every finding against the source, and register screening of every company and person named in the bid. The only difference was which engine served which stage.

2. Both runs did the job

Same recommendation, and agreement on eleven of the sixteen award criteria. Both independently caught a self-declared percentage that does not recompute from the bid's own figures, and a foreign-language document submitted without a certified translation. Where they differ is how heavily they weighed what they found.

GLM-5.3 Flash escalated more. It rated three defects critical where the Claude run recorded comparable issues lower down, or left the criterion for a human to settle. Severity is a judgement call, and neither weighting is obviously right — a bid manager who wants every formality flagged loudly is served better by the first, one who wants a short list of real blockers by the second. Both are defensible readings of the same documents.

The cheaper model worked harder to get to the same place. 28 shell commands, 15 file reads, 12 image extractions and 7 greps against Opus's 11, 5, 3 and none — at one point opening a document's underlying XML to read a checkbox state directly. Being cheap per token is what let it afford that.

3. The bill

The per-requirement stage alone cost more on Claude ($17.39) than the entire GLM run ($4.82). That stage runs on Haiku, Anthropic's cheap tier. A cheap tier is only cheap next to its own family.

GLM's token appetite is still there: 2.98 million output tokens and 2.13 million thinking tokens against Claude's 0.85 and 0.10 million, about five times as much writing to reach an answer. In August that appetite ate a 3.6× price gap down to 17%. This time the gap was wide enough to absorb it.

4. Reliability

This is where the cheap run pays for itself in the wrong currency. 10h18m against 2h11m, and almost none of the difference is extra thinking — the run sat idle for stretches of 14 to 78 minutes waiting on provider capacity. That reflects how loaded a routed endpoint is on a given afternoon rather than anything about the weights, but a specialist who starts an analysis before lunch and expects to read it after will notice.

GLM also still breaks on JSON. One coverage worker died on an unterminated string mid-tool-call and another call needed a retry, costing one unchecked requirement out of the set. That is the failure mode that killed five of six GLM-5.2 runs when we first benchmarked it in May. It has gone from fatal to a dropped item, and it has not gone away.

The Claude run was not spotless either: one lot worker aborted after five consecutive tool errors and had to be written off the same way. One dropped item each, from different causes.

5. What we take from it

One run each is not a benchmark, and the honest summary is unglamorous. Two engines seven times apart on price did the same job to a comparable standard, disagreed about severity in ways a human reviewer would arbitrate anyway, and differed most on how long you wait and how often something drops.

For us the engine is a configuration flag — the same pipeline runs on Claude, GPT, Gemini or GLM, and every stage uses the engine you picked, with no silent substitution. If a stage can quietly drop to a cheaper model, you cannot know what your analysis actually saw. That matters more than which of these two you choose.

6. How we counted

Both runs are single production analyses of the same bid against the same tender, with the same prompts, tools and documents. The GLM figure is the cost OpenRouter reported per call, summed. The Claude figure is computed from Anthropic's published rates including the cache-read discount and cache-write premium, because first-party responses do not return a price. Findings counts separate model-authored findings from register screening throughout. Our May comparison, five runs per model, is the larger sample.