VeloBenchmark

What it does

An instrument, not a scoreboard

Every number is computed server-side from real streaming token timing and snapped to the provider's authoritative usage counts when reported. The UI is a window onto the same data the reports store.

Live instruments

Rolling decode tok/s, TTFT, inter-token latency, min/median/max and stability — streamed per token while the answer generates, with PNG/PDF export of the panel.

Regime-aware analysis

Output is tagged as prose, code, math, JSON or reasoning as it streams. Charts split by regime, so a model slowing down on math and speeding up on code is visible inside a single answer.

Repeatable suites

A visual builder with five step types — sections, prompts, exact context fills, fixed-shape bench requests and vision steps — plus built-in reference suites and a JSON editor.

Concurrent runner

N workers walk the same test behind a step barrier. Per-worker snapshots live, and the report gains a workers + Σ decode timeline: your effective concurrency throughput.

Reports & comparisons

Per-session analytics — throughput, latency distributions, token composition, speculation diagnostics — and persistent side-by-side comparisons of any sessions, exportable to PNG/PDF.

Telemetry receiver

A built-in OTLP/HTTP-JSON receiver turns a serving engine's OpenTelemetry stream into live per-stream panels — and one click records a rolling window into a permanent session report.

Measurement vocabulary

The metrics the field actually uses

VeloBenchmark speaks the ecosystem's language: the same definitions as vLLM, AIPerf and Artificial Analysis, computed from the stream alone where possible and labelled estimate where they are inferred. Speculative-decoding behaviour surfaces as an acceptance-rate estimate and a speculation-depth distribution — from bimodal inter-token latency, without server counters.

TTFT / TTST / TTFO decode tok/s rolling + final TPOT decode-normalised ITL p50 / p90 / p99 ICL jitter prefill t/s = ISL / TTFT acceptance α estimate spec depth distribution goodput vs SLO

How each metric is computed →

Live stats panel: decode rate, responsiveness, progress, timeline and histograms

The live stats deck — identical on the chat page and on every telemetry panel.

Two products, one lineage

Measure it, then run it

Flagship

The benchmarking and live-stats console. Chat with live instruments, build test suites, scale them to concurrency, compare sessions, and front your serving engine's OpenTelemetry stream with the built-in OTLP receiver. One binary; the whole UI embedded; settings and sessions stored server-side.

Rust + Angular AGPL-3.0 Linux · macOS

Documentation Repository

Engine

veloGB10

A from-scratch Rust + CUDA inference engine for NVIDIA GB10 systems (DGX Spark & compatible): NVFP4 tensor-core serving, bitwise-lossless MTP speculation, and tensor parallelism across two or four boxes as a speed mode — up to ~125 tok/s sustained on Qwen3.8-27B.

Rust + CUDA Apache-2.0 sm_121

Overview Documentation

Inside the console

Every screen is a report in the making

Open source is the product. Services make it fit.

VeloBenchmark and veloGB10 are free forever under AGPL-3.0 and Apache-2.0. When you need it integrated into your pipeline, extended for your workload, or tuned on your hardware, that work is available under contract — scoped, quoted, and delivered with acceptance criteria.

Services & licensing