fix(chat): keep the request alive while its answer streams, so a stop still reaches the provider (#965)

A Request's signal follows the signal it was created with through a weak
link inside undici: once the Request object is garbage collected, the link is
gone and request.signal never aborts (nodejs/undici#3644). The chat route
passes req.signal to streamText and holds nothing else from the request, so a
GC pause during a stream could leave the model call running after the client
stopped or disconnected, with no onAbort accounting.

The route now keeps each request in a WeakMap keyed by its response, which
Next holds while the body is piped.

This is also the cause of the unit test chat-route-abort hanging on CI (three
times this week, 60 ms locally): the test's Request is dropped as soon as the
route returns, and a GC in that window lost the abort. With a forced GC the
test hangs on Node 20 and 24 without this change and passes with it.
This commit is contained in:
Dayuan Jiang
2026-10-10 09:24:25 +09:00
committed by GitHub
parent e117095c7b
commit a7ae114edf
+6
View File
@@ -98,6 +98,11 @@ function createCachedStreamResponse(xml: string): Response {
// Responses streamed from the model, whose trace streamText's callbacks end
const modelStreamResponses = new WeakSet<Response>()
// A Request's signal follows the client's disconnect only while the Request
// object itself is alive: once it is garbage collected, the abort is lost
// (nodejs/undici#3644) and a stopped chat would run on at the provider.
// Each request is kept as long as its answer streams.
const requestOfResponse = new WeakMap<Response, Request>()
// Inner handler function
const DEBUG_LLM_PAYLOAD = process.env.DEBUG_LLM_PAYLOAD === "true"
@@ -787,6 +792,7 @@ Call this tool to get shape names and usage syntax for a specific library.`,
},
})
modelStreamResponses.add(response)
requestOfResponse.set(response, req)
return response
}