mirror of
https://github.com/DayuanJiang/next-ai-draw-io.git
synced 2026-10-10 19:49:52 +08:00
fix(chat): keep the request alive while its answer streams, so a stop still reaches the provider
A Request's signal follows the signal it was created with through a weak link inside undici: once the Request object is garbage collected, the link is gone and request.signal never aborts (nodejs/undici#3644). The chat route passes req.signal to streamText and holds nothing else from the request, so a GC pause during a stream could leave the model call running after the client stopped or disconnected, with no onAbort accounting. The route now keeps each request in a WeakMap keyed by its response, which Next holds while the body is piped. This is also the cause of the unit test chat-route-abort hanging on CI (three times this week, 60 ms locally): the test's Request is dropped as soon as the route returns, and a GC in that window lost the abort. With a forced GC the test hangs on Node 20 and 24 without this change and passes with it.
This commit is contained in:
@@ -98,6 +98,11 @@ function createCachedStreamResponse(xml: string): Response {
|
||||
|
||||
// Responses streamed from the model, whose trace streamText's callbacks end
|
||||
const modelStreamResponses = new WeakSet<Response>()
|
||||
// A Request's signal follows the client's disconnect only while the Request
|
||||
// object itself is alive: once it is garbage collected, the abort is lost
|
||||
// (nodejs/undici#3644) and a stopped chat would run on at the provider.
|
||||
// Each request is kept as long as its answer streams.
|
||||
const requestOfResponse = new WeakMap<Response, Request>()
|
||||
|
||||
// Inner handler function
|
||||
const DEBUG_LLM_PAYLOAD = process.env.DEBUG_LLM_PAYLOAD === "true"
|
||||
@@ -787,6 +792,7 @@ Call this tool to get shape names and usage syntax for a specific library.`,
|
||||
},
|
||||
})
|
||||
modelStreamResponses.add(response)
|
||||
requestOfResponse.set(response, req)
|
||||
return response
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user