Cerebras is now serving Qwen 3.8 27B at roughly 1500 tokens per second, according to the company’s inference documentation. The HN thread announcing it pulled 498 points, and the reason is simple: that is roughly an order of magnitude faster than most GPU-based providers deliver for a model of this size.
For comparison points from the same discussion: OpenRouter’s fastest provider for Qwen 3.8 clocks around 80 tokens per second, and a GPU provider running the model on Blackwell hardware manages about 29.5 tokens per second server-side. Cerebras gets its speed from wafer-scale silicon rather than racks of GPUs, which is why the gap is structural rather than incremental.
The numbers that matter
From Cerebras’s model documentation:
- Throughput: approximately 1500 tokens per second generation for the 27B model
- Pricing: $0.99 per million input tokens, $1.49 per million output tokens
- Context: 64k tokens on the free tier, 128k on paid tiers
- Max output: 32k tokens free, 40k paid
Cerebras’s GPT OSS model runs even faster, at around 3000 tokens per second, so 1500 for Qwen 3.8 is mid-range by the platform’s own standards. That is the striking part: what would be a headline speed anywhere else is the slow offering here.
The catch: rate limits
Raw speed is not the same as unbounded capacity. The free tier allows 5 requests per minute and 1 million tokens per day. The developer tier raises that to 300 requests per minute, with 150k uncached tokens per minute. HN commenters were quick to note that a public endpoint limit of 150k tokens per minute means very high-throughput workloads will hit the ceiling even as individual requests scream along.
So the mental model is: very fast per request, moderate aggregate capacity. For an interactive app, a coding assistant, or an agent loop where each step waits on the last, 1500 tokens per second turns multi-second waits into near-instant responses. For batch jobs pushing millions of tokens, the per-minute caps matter more than the per-token speed.
What sub-second inference actually changes
Speed at this level is not just a convenience. It changes interaction design.
When generation runs at 1500 tokens per second, a 500-token response arrives in about a third of a second. That is fast enough to stream completions the way a local autocomplete behaves, to run agentic loops with dozens of sequential model calls without the latency compounding into minutes, and to do speculative multi-path generation where you explore two or three candidate responses and keep the best, because the cost of trying is trivial in time.
Where it fits against the competition
Context on where this lands: Qwen 3.8 27B is an open-weight model from Alibaba’s Qwen team, and 27B is the size class that has become the sweet spot for self-serve inference, big enough for competent reasoning and coding, small enough to serve cheaply. Cerebras has made a habit of pairing open-weight models with headline throughput, and the platform already serves GPT OSS at around 3000 tokens per second, so 1500 for Qwen 3.8 is mid-range by the platform’s own standards. That is the striking part: what would be a headline speed anywhere else is the slow offering here.
It also matters for privacy-sensitive setups. A 27B open-weight model at $0.99/$1.49 per million tokens with no queue and near-instant output is a credible option for workloads where sending data to a frontier API is a non-starter but a local GPU setup would be too slow or too expensive to maintain.
Should you switch?
A quick decision guide:
- Latency-sensitive interactive features: strong yes. The speed difference is user-visible and the pricing is competitive.
- High-volume batch processing: check the token-per-minute caps against your workload first, and compare total cost against GPU providers at your actual volume.
- Model quality boundary: 27B is capable but not frontier. Tasks needing deep reasoning still belong on larger models, whatever their speed.
The broader trend is worth watching. Specialized inference silicon from Cerebras and Groq keeps widening the gap over general-purpose GPU serving, and open-weight models keep closing the quality gap at smaller sizes. Both trends point the same direction: fast, cheap, private inference on open models is becoming the default for a growing share of workloads, and the frontier API is reserved for what genuinely needs it.
There is also an ecosystem effect. When a 27B open model is this cheap and this fast, it becomes the default evaluation target for tooling: agent frameworks benchmark against it, quantization research targets it, and fine-tunes build on it. Speed changes which models get the ecosystem investment, and ecosystem investment is often what determines which models improve fastest. A model nobody runs fast never accumulates that flywheel, regardless of its paper benchmarks.
One practical note for anyone trying it this week: the model’s ID in the API is qwen-3.8-27b, and the free tier’s 1 million tokens per day is enough for a serious evaluation but not for production traffic. The sensible path is to run your latency-sensitive features against it first, keep your hardest reasoning tasks on a larger model, and watch how the token caps interact with your real traffic before committing.