fix(server): count quota by the key actually used, and more review fixes

Found by the PR review, each with a test that failed first:
- Quota: any key header skipped it, even one the provider never reads
  (x-aws-access-key-id with OpenAI), so a request ran on the server's
  key without being counted. The check now runs after the model is
  resolved and uses usesServerCredentials. On main already.
- usesServerCredentials read the raw base URL; "/" cleans up to none, so
  an Ollama request ran on the server's key past the server-model check.
- SGLang's default 127.0.0.1:8000 only fills the settings form. Chat and
  the model list used it as a real address, so the server called its own
  machine even with private URLs blocked. Now a base URL is required.
- With a user's OpenAI key and no base URL, the SDK read the server's
  OPENAI_BASE_URL. The official endpoint is now passed. On main already.
- The Test button refused nothing on the server's keys (Ollama Cloud),
  and a 15 s timeout reported "connected, no tool call".
- The model list for Ollama without a base URL came from ollama.com while
  chat went to the server's Ollama.
- Bedrock's "Too many tokens, please wait" counted as context too long.
- On the server's keys the provider's error text stays in the server log;
  it can name the server's AWS account, role or internal hosts.
- Desktop app: the preset keys are the user's own (NEXT_AI_DRAWIO_DESKTOP),
  so Max Output Tokens can be raised and keyless models in settings work
  again. A launch that found the remembered port taken no longer replaces
  it, which hid the user's chats and settings for good.
This commit is contained in:
dayuan.jiang
2026-10-04 23:04:21 +09:00
parent 504d2fa812
commit 0855b35ff2
14 changed files with 382 additions and 54 deletions
+31 -39
View File
@@ -133,36 +133,6 @@ async function handleChatRequest(req: Request): Promise<Response> {
userId: userId,
})
// === SERVER-SIDE QUOTA CHECK START ===
// Quota is opt-in: only enabled when DYNAMODB_QUOTA_TABLE env var is set
const hasOwnApiKey = !!(
req.headers.get("x-ai-provider") &&
(req.headers.get("x-ai-api-key") ||
req.headers.get("x-aws-access-key-id") ||
req.headers.get("x-vertex-api-key"))
)
// Skip quota check if: quota disabled, user has own API key, or is anonymous
if (isQuotaEnabled() && !hasOwnApiKey && userId !== "anonymous") {
const quotaCheck = await checkAndIncrementRequest(userId, {
requests: Number(process.env.DAILY_REQUEST_LIMIT) || 10,
tokens: Number(process.env.DAILY_TOKEN_LIMIT) || 200000,
tpm: Number(process.env.TPM_LIMIT) || 20000,
})
if (!quotaCheck.allowed) {
return Response.json(
{
error: quotaCheck.error,
type: quotaCheck.type,
used: quotaCheck.used,
limit: quotaCheck.limit,
},
{ status: 429 },
)
}
}
// === SERVER-SIDE QUOTA CHECK END ===
// === FILE VALIDATION START ===
const fileValidation = validateFileParts(messages)
if (!fileValidation.valid) {
@@ -292,14 +262,41 @@ async function handleChatRequest(req: Request): Promise<Response> {
)
}
// === SERVER-SIDE QUOTA CHECK START ===
// Quota is opt-in (DYNAMODB_QUOTA_TABLE) and counts what runs on the
// server's keys. Decided by the key actually used: a key header the
// provider never reads must not skip it.
const countsQuota =
isQuotaEnabled() && onServerCredentials && userId !== "anonymous"
if (countsQuota) {
const quotaCheck = await checkAndIncrementRequest(userId, {
requests: Number(process.env.DAILY_REQUEST_LIMIT) || 10,
tokens: Number(process.env.DAILY_TOKEN_LIMIT) || 200000,
tpm: Number(process.env.TPM_LIMIT) || 20000,
})
if (!quotaCheck.allowed) {
return Response.json(
{
error: quotaCheck.error,
type: quotaCheck.type,
used: quotaCheck.used,
limit: quotaCheck.limit,
},
{ status: 429 },
)
}
}
// === SERVER-SIDE QUOTA CHECK END ===
// Retry once if the provider rejects the requested budget, or (newer
// Claude models) the sampling or thinking settings
const model = withOutputTokenLimitFallback(
withDeprecatedParamsFallback(baseModel),
)
// The user setting can raise the budget only on their own key (desktop users
// can still raise it themselves); on the server's keys it can only lower it
// The user setting can raise the budget only on their own key (in the
// desktop app every key is the user's); on the server's keys it can only
// lower it
const maxOutputTokens = resolveMaxOutputTokens(
req.headers.get("x-max-output-tokens"),
onServerCredentials,
@@ -587,12 +584,7 @@ IMPORTANT: The "Current diagram XML" is the SINGLE SOURCE OF TRUTH for what's on
// Record token usage for server-side quota tracking (if enabled)
// Use totalUsage (cumulative across all steps) instead of usage (final step only)
// inputTokens already includes cache reads and writes in AI SDK 6
if (
isQuotaEnabled() &&
!hasOwnApiKey &&
userId !== "anonymous" &&
totalUsage
) {
if (countsQuota && totalUsage) {
const totalTokens =
(totalUsage.inputTokens || 0) +
(totalUsage.outputTokens || 0)
@@ -724,7 +716,7 @@ Call this tool to get shape names and usage syntax for a specific library.`,
const response = result.toUIMessageStreamResponse({
sendReasoning: true,
onError: streamErrorText,
onError: (error) => streamErrorText(error, onServerCredentials),
messageMetadata: ({ part }) => {
if (part.type === "finish") {
const usage = (part as any).totalUsage