98.0% on GPQA-Diamond, and What It Costs
Tiered dispatch reaches frontier accuracy at a fraction of frontier price. Below it: what an agent costs at rest and in flight, how fast a sandbox starts, and what retrieval recovers — with the losses and the gaps.
Pareto Frontier on GPQA-Diamond
Reasoning accuracy versus token pricing across tiered model dispatch.
Detailed GPQA-Diamond evaluation table →| Model | Accuracy (%) | Correct / Total | Cost (per 1M) |
|---|---|---|---|
| Enso Ultra | 98.0 | 194 / 198 | $25 |
| Enso | 96.0 | 190 / 198 | $20 |
| Enso Flash | 92.9 | 184 / 198 | $4 |
| GPT-5.6 Sol | 92.9 | — | $25 |
Speculative verification on high-difficulty reasoning benchmarks. The Flash routing tier achieves 92.9% accuracy at 84% lower token expenditure ($4/M vs $25/M) through selective fallback.
Process Concurrency & Memory Residency
Heap residency, serialization throughput, and resumption latency under concurrent agent execution.
Execution Sandbox Cold-Start Latency
Initialization latency across V8 contexts, V8 isolates, and microVM instances without prewarmed standby pools.
Detailed sandbox isolation benchmark →Multi-Hop Retrieval & Fact Reconciliation
Recall accuracy and temporal conflict resolution over extended context windows (up to 262K tokens).
AST-Aware Cross-File Dependency Retrieval
Syntactic and dependency graph traversal for code completion on RepoBench-R.
Detailed RepoBench-R code retrieval table →Lexical BM25 retrieval degrades across multi-file repositories due to symbol shadowing. Semantic graph indexing walks import hierarchies and call trees, boosting R@1 from 18.8 to 33.2 (+14.4 points over baseline).
Prefill & Decode Throughput vs llama.cpp
Throughput evaluation of Hanzo Engine against llama.cpp on identical hardware, weights, and quantization levels.
Detailed cross-backend inference matrix →| Backend | Model | Context | Speedup |
|---|---|---|---|
| CUDA | Zen-1.7B | pp4096 | 1.088× |
| CUDA | Zen-4B | pp2048 | 1.051× |
FlashAttention-3 integration and custom fused GEMM kernels prevent memory bandwidth saturation during prefill. Paged KV-cache allocation eliminates memory fragmentation under concurrent batch requests.
Evaluation Protocol & Reproduction
All benchmark harnesses, raw run telemetry, and analysis scripts are published in the open repository. Every published figure is directly regenerable from committed test fixtures.
Clone the evaluation harness and execute the test runner to regenerate raw telemetry and comparison tables from scratch:
git clone https://github.com/hanzoai/cloud && cd cloud/bench/brain && make reproduceInvariant greedy decoding (temperature 0.0), fixed pseudo-random seeds, and uniform prompt templates across all evaluation arms.
95% bootstrap confidence intervals computed over questions and test splits to distinguish empirical signal from stochastic variance.
Bare-metal benchmarking with pinned CPU affinity, isolated memory regions, and dedicated GPU/accelerator contexts.