About

Two tools born from one annoyance

Benchmark numbers for LLM serving are usually a single average tokens-per-second figure, measured by whoever is selling something, with the denominator left undefined. These projects exist because measuring properly — per token, per regime, per request, under load — turned out to be worth the effort of building the instrument first.

What the two products are

VeloBenchmark is the flagship: a single-binary benchmarking and live-stats console for any OpenAI-compatible endpoint. It began as the question “what is this server actually doing per token?” and grew into chat instrumentation, repeatable test suites, a concurrent runner, session reports, comparisons, and an OTLP telemetry receiver that can sit in front of a serving engine's own OpenTelemetry stream.

veloGB10 is the engine side: a from-scratch Rust + CUDA inference engine for NVIDIA GB10 systems (DGX Spark and compatible OEM boxes), where ~255 GB/s of memory bandwidth is the scarce resource and every design decision is subordinate to spending it well. It emits the OpenTelemetry stream that VeloBenchmark's receiver was built to consume — the two projects measure and serve the same reality, and each one's docs reference the other.

Principles

Measured, not claimed
Numbers come from real token timing or engine gates on real hardware. Where a value is inferred, the UI labels it estimate.
One binary, zero deps
The whole console — UI, API, payloads — compiles into a single file. Deployment is copying it. The engine ships as a binary plus its PTX kernels.
Telemetry never hurts decode
The receiver is passive and the engine's emitter drops rows rather than block. A slow dashboard must never slow a generation.
Honest edges
Limitations are documented, not discovered: known limitations are part of the manual, and the engine's README states what it does not support and why.
Server-side truth
Measurements are computed in the server, not the browser. The UI and the stored reports are views of the same data.

Licensing

VeloBenchmark is AGPL-3.0: free to use, modify, and deploy, including commercially; network deployments of modified versions share their source. That is a deliberate choice for a measurement tool — the methodology stays inspectable by the people whose decisions it informs. veloGB10 is Apache-2.0. Paid work (integration, extensions, tuning, support, and commercial licences where the AGPL does not fit) is described on the services page; none of it gates the software itself.

How the work is produced

Both projects are developed with strong assistance from open-source LLMs, under experienced human technical direction — architecture decisions, ideas, testing and extensive debugging included — and this is stated openly because it shaped how they were built. Every performance figure is gated or measured on physical hardware; where a number is an estimate, the docs say so. The acknowledgement matters in both directions: none of this exists without the kernels, formats, and hard-won engineering knowledge of the wider open ecosystem.

Maintainer

Both projects are written and maintained by Stav Katsoulis (sf-stav) — kernels, scheduler, transport, gates, console, docs and releases. Bug reports and well-formed issues are always free and welcome; contracted work is quoted per request. The shortest path to a human is stav@scalarflow.co.uk.