feat: add API key load balancing for providers (#676)

Support multiple API keys per provider with random selection for load
balancing. When AI_MODELS_CONFIG has multiple apiKeyEnv values for
a provider, requests will randomly select one available key.

- Update schema to accept apiKeyEnv as string or string array
- Add random key selection in resolveApiKey()
- Update validation to check at least one key exists
- Add tests for array format support
This commit is contained in:
Dayuan Jiang
2026-02-02 15:54:35 +09:00
committed by GitHub
parent 4624ad40a1
commit cd33e131ef
5 changed files with 143 additions and 14 deletions

View File

@@ -186,7 +186,7 @@ async function handleChatRequest(req: Request): Promise<Response> {
// Check if this is a server model with custom env var names
let serverModelConfig: {
apiKeyEnv?: string
apiKeyEnv?: string | string[]
baseUrlEnv?: string
provider?: string
} = {}