avday The bug that leaked other customers’ data on Cloudflare’s Workers AI was described in a public SGLang issue on 6 July, with a proposed fix. A bot closed that issue on 5 September.
Three days later, a researcher reported the leak to Cloudflare. Cloudflare knew about its own part, too. It launched GLM-5.3 on 27 August knowing its memory pool held 35% less than the context window it advertised. Any prompt longer than 681,280 tokens could come back with someone else’s data in it.
Were you affected? Cloudflare says “a small number of customers” were, across 155 responses. It doesn’t say whether it has contacted them. If you run SGLang yourself, there’s a one-line check in the third section below.
How the Workers AI leak happened
A researcher, Gadi Evron of Knostic, reported the leak through Cloudflare’s HackerOne programme on 8 September. Cloudflare’s incident report, published on 26 October, names two causes that had to line up:
- A memory pool Cloudflare knew was too small. GLM-5.3 advertised a 1,048,576-token context window, but the KV cache pool on the prefill GPUs held 681,280 tokens. “We were aware of this mismatch before launch,” Cloudflare says. It expected the engine to return an error for anything that didn’t fit.
- An engine bug. SGLang, the open-source inference engine Cloudflare runs, rejected over-long prompts. But with prefill and decode on separate GPUs, a rejected prompt carried on as a one-token stub instead of failing.
The decode GPUs had reserved memory pages for the full prompt but received one page. They read the rest anyway. Those pages “still held what earlier requests left in them: other users’ prompts and partial generations.”
Cloudflare had also prototyped a check for exactly this. Over the summer its prototype tagged every KV page and confirmed ownership before decode read it, at under 1% throughput cost in its tests. It tried this on GLM-5.2 in mid-August, “but disabled it shortly afterwards due to reliability issues.” GLM-5.3 launched on 27 August without it.
Its launch tests missed the leak because, in Cloudflare’s words, “we only validated HTTP response codes and engine performance at long contexts, without validating model output and coherence.”
The SGLang bug was public in July
Cloudflare’s post says the bug “had been reported to SGLang by another user as issue #30233 on July 6, 2026”, and that it only noticed the issue while investigating. The upstream record:
- 6 July. Issue #30233 is filed against SGLang 0.5.11, running GLM-5.2. It sets out the root cause step by step and includes a proposed diff.
- 8 July. A community fix, pull request #30551, is opened. Its author called it mergeable on 10 July. It was still open on 27 October.
- 5 September. A bot closes the issue: “This issue has been automatically closed due to inactivity.” That’s 61 days after it was filed, and three days before Evron’s report.
- 6 September. Pull request #36988 is merged. It was written for a vision-model failure path and doesn’t mention #30233, but Cloudflare and the SGLang team both name it as the fix. It shipped in v0.5.20 on 18 September.
The issue describes “garbage” output. In a test where every request is yours, stale cache does look like nonsense. On a shared service, those stale pages hold someone else’s prompt.
Check your own SGLang deployment
SGLang isn’t only Cloudflare’s engine. Hugging Face’s page for DeepSeek-V4.1-Flash, the model in our piece on DeepSeek’s DeepInfer engine, lists SGLang as one way to serve it. This bug only bites when prefill and decode run on separate machines: the SGLang team says single-node set-ups returned an error instead. If yours is split:
- Compare two numbers. At startup SGLang logs
max_total_num_tokens=andcontext_len=on the same line (we read this inscheduler.pyat v0.5.22). The first is your KV pool. If it’s smaller than the second on any prefill or decode node, you’re in Cloudflare’s August set-up. - Upgrade to v0.5.22 or later, released on 19 October, as Cloudflare recommends. It carries #36988, plus #38935, which rejects over-long prompts before any memory is handed from prefill to decode.
- Read the replies to long prompts, not just their status codes. That’s the test Cloudflare says it skipped.
What Cloudflare and SGLang said
Cloudflare says it “found no evidence of deliberate exploitation”. No other Workers AI model was affected, it says, because every other disaggregated deployment’s pool is larger than its context length.
On 9 September it took GLM-5.3 offline for 5 hours 40 minutes. It brought the model back with a patched engine and a pool of about 2.04 million tokens, roughly twice the window.
The SGLang team published its own post the same day, with the engine-side detail and its advice for operators.
What happens next
The researcher’s own write-up is “available soon”, according to the SGLang team. Cloudflare says it is working with SGLang to build page-level ownership checks into the engine for every model. Separately, it plans to formally verify its KV cache transfer protocol. Neither has a date.
Our take
In a multi-tenant inference engine, “decode reading pages this request never wrote”, as Cloudflare puts it, is a security bug, whatever the output looks like. Issue #30233 had a reproduction and a proposed diff, and it sat for 61 days filed as a quality problem.
The PHP2Shell flaws patched on 21 October had a similar shape: a known bug whose fix stalled for months. Open-source maintainers can’t triage everything. A bug that reads another request’s memory should get a security label on day one, from whoever spots it first.
