Celeris vs Groq vs Cerebras: measured tokens per second, end to end
In our benchmarks post we showed how Celeris-1 performs against the frontier labs and hyperscalers, who serve their models on conventional, off-the-shelf GPU infrastructure. Here we run the same end-to-end measurement against providers serving on custom inference hardware. Celeris-1 delivered tokens fastest, on commodity GPUs.
Measured from a client colocated with each provider, Celeris delivers tokens fastest - 8% ahead of Cerebras, 3.5× ahead of Groq. All figures are end-to-end wall clock: the one basis every provider can be compared on.
MethodWhat we measured, and from where
The workload mirrors Artificial Analysis's standard performance tier: ten prompts of ~1,000 input tokens each (counted with o200k_base), asking for a long-form analytical answer, sent one at a time - non-streaming - over a warm keep-alive connection, ten samples per prompt. Every provider gets byte-identical prompts.
All throughput in this report is end-to-end wall clock: from request write to last byte read, measured from an EC2 instance placed in the closest AWS region we could find for each provider, so the network term is as small and as honest as we can make it. We deliberately do not rank on server-reported times, because the providers don't report the same thing: Cerebras and Groq return a generation-only completion_time that excludes queue and prompt-processing time, while Celeris' Server-Timing header covers total processing. Those clocks can't be compared to each other - end-to-end can. Server times were still recorded on every request and used as a cross-check on the network overhead (end-to-end minus server total), reported per vantage point below.
Region selection was empirical, not guesswork: authenticated one-token probes from five candidate AWS regions, comparing end-to-end minus server-reported time. Groq helpfully leaks its serving datacenter in an x-groq-region header - from the west coast it serves from The Dalles, OR (dls) and Kamloops, BC (yka); Cerebras is Cloudflare-fronted and silent, but its overhead floor points east.
| Provider | Endpoint origin evidence | AWS region used | Net overhead p50 |
|---|---|---|---|
| Celeris | hosted in AWS us-east-1 | us-east-1 | 3.9 ms |
| Cerebras | probe floor lowest from Ohio | us-east-2 | 98 ms |
| Groq | x-groq-region: dls / yka | us-west-2 | 52 ms |
Throughput is reported as an aggregate - total tokens over total time - rather than a mean of per-request ratios, so every second of the run is weighted equally:
ResultsCeleris delivers fastest; Groq trails by 3.5×
At the matched 2,048-token cap, Celeris delivers 1,596 tok/s end-to-end - 8% ahead of Cerebras' 1,480. Groq serves the same gpt-oss-120b weights as Cerebras at 460 tok/s: remarkably consistent request to request (p90 within 2% of p50), but 3.5× off the pace.
Where the gap comes from matters. Celeris' API lives in AWS us-east-1, so a colocated client pays ~4 ms of network per request - effectively nothing. Cerebras is fronted by Cloudflare and pays ~98 ms per request even from its best AWS region, which at these response lengths costs it roughly 12% of its delivered throughput. That overhead is part of what every real client experiences, which is exactly why it belongs in the measurement.
HardwareOff-the-shelf GPUs against custom silicon
This comparison crosses a hardware class. Groq and Cerebras run purpose-built inference silicon - a custom LPU and a wafer-scale engine, respectively. celeris-1 is served entirely from off-the-shelf GPUs in AWS us-east-1: hardware anyone can rent, with no custom silicon anywhere in the stack.
BehaviorWho actually finishes the answer
Throughput isn't the whole story: the same prompt produces very different responses. Asked for "about 1,000 words," celeris-1 wrote 1,282–1,558 tokens and finished naturally on all 100 requests - it follows length instructions and lands the answer inside the cap. gpt-oss-120b writes ~2,640 tokens (≈1,800 words, plus ~180 hidden reasoning tokens) to the same instruction, so at the matched 2,048 cap every one of its responses was cut off mid-answer. Its throughput is unaffected - decode speed is constant - but if you cap it near your target length, you get fragments; budget ~2.7k tokens for it to finish this kind of task.
Mean wall clock per complete answer tells the practical story: 895 ms for Celeris, 1,384 ms for Cerebras (truncated), ~4.4 s for Groq. Most of that spread is verbosity, not silicon.
IntelligenceSpeed is only half the comparison
Throughput measures delivery, not thinking, so we also ran both models through the MMLU-Pro harness from our benchmarks post: the full test set, five-shot chain-of-thought, scored two ways - strict, where the model must state the answer is (X) and match the gold letter, and anywhere, a lenient pass that credits the correct letter appearing anywhere in the output.
celeris-1 scores 75.9%, identical under both scorings - it always lands the answer in the required format. gpt-oss-120b on Cerebras scores 65.7% strict and 73.6% anywhere: even with the lenient scoring doing it every favor, it sits 2.3 points behind celeris-1, and it gives up a further 7.9 points to format errors under the strict template - the same instruction-following weakness as the truncation behavior above.
| Model | Strict | Anywhere |
|---|---|---|
| celeris-1 | 75.9% | 75.9% |
| gpt-oss-120b (Cerebras) | 65.7% | 73.6% |
So the throughput race above is between models that are not intellectual equals: the one delivering tokens fastest is also the one getting more of the answers right.
GroqThe asterisk on 460
Groq's number comes from the original 50-request run at a 2,042-token cap - with its free-tier limits (8,000 tokens/minute) pacing the run to about two requests per minute. A planned 100-request rerun on the revised prompts died at the day wall: the key's 200,000 tokens-per-day quota ran dry mid-run, throttling it to one request every five minutes. The 460 tok/s figure itself is trustworthy - per-request throughput was the most consistent of the three providers, and rate limiting between requests doesn't contaminate per-request timing - but a paid-tier rerun on the matched configuration is the obvious follow-up.