i/next-ai-draw-io
1
0
Fork 0
mirror of https://github.com/DayuanJiang/next-ai-draw-io.git synced 2026-09-02 01:20:23 +08:00
Code Issues Packages Projects Releases Wiki Activity

fix: merge system messages for custom OpenAI-compatible endpoints (#774)

Browse Source 
* fix: merge system messages for custom OpenAI endpoints (fixes #734)

When using the OpenAI provider with a custom base URL (e.g., vLLM, LMStudio),
the app sends two system messages to the API. Open-source model chat templates
(Qwen, Llama, etc.) enforce that system messages must appear at the beginning
and reject multiple system message blocks, causing the error:
'System message must be at the beginning.'

Treat custom OpenAI endpoints (client-provided base URL or OPENAI_BASE_URL env
var) the same as other known single-system providers by merging both system
messages into one before sending.

* fix: also detect custom OpenAI endpoint from serverModelConfig.baseUrlEnv

---------

Co-authored-by: dayuan.jiang <jdy.toh@gmail.com>
This commit is contained in:
Octopus
2026-04-02 22:27:29 -05:00
committed by GitHub
parent f5ea5a0edd
commit 6c6cf98019
1 changed files with 12 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
app/api/chat/route.ts
View File

@@ -436,7 +436,18 @@ ${userInputText}
// - Breakpoint 2: Current XML context - changes per diagram, but constant within a conversation turn
// Some providers (e.g. MiniMax) don't support multiple system messages
// Merge them into a single system message for compatibility
const isSingleSystemProvider = SINGLE_SYSTEM_PROVIDERS.has(resolvedProvider)
// Also merge for OpenAI-compatible providers with custom base URLs (e.g. vLLM, LMStudio)
// because open-source model chat templates (Qwen, Llama, etc.) typically reject multiple system messages
const isCustomOpenAIEndpoint =
resolvedProvider === "openai" &&
!!(
baseUrl ||
process.env.OPENAI_BASE_URL ||
(serverModelConfig.baseUrlEnv &&
process.env[serverModelConfig.baseUrlEnv])
)
const isSingleSystemProvider =
SINGLE_SYSTEM_PROVIDERS.has(resolvedProvider) || isCustomOpenAIEndpoint
const xmlContext = `${
previousXml