fawney
2cb4d554aa
feat(routing): consolidate scheduling strategy configuration
2026-09-03 11:05:59 +08:00
elky
7323d41fbe
feat(routing): move sticky-key retries into routing policy with lazy attempts
...
Replace the provider/endpoint max_retries fields as the source of same-key
retries with a routing policy setting, sticky_key_attempts (default 2). Only
the first-ranked candidate is retried on the same key; every failover
candidate gets a single attempt so failover keeps advancing instead of
retrying each fallback key.
Materialize exactly one attempt per candidate and derive same-key retries in
the attempt loop after a candidate-scoped failure, so the retry budget no
longer inflates up-front materialization and needs no upper bound. The budget
travels in the report context; retries reuse the plan with a fresh candidate
id and incremented retry index. Pool groups only retry their first key within
the retry-index stride.
Expose the setting in the routing profile editor and the set_scheduling rule
action, and drop the max_retries input from the provider form.
2026-09-02 20:48:40 +08:00
ZheFox
633363e190
fix(gateway): handle pool saturation and malformed Gemini calls
2026-09-01 19:25:00 +08:00
AAEE86
71b54070e8
feat(gateway): Codex/OpenAI Responses WebSocket 代理模式
...
在 /v1/responses 上支持 WebSocket 升级，把客户端帧中继到上游 Codex /
OpenAI Responses WebSocket 端点，同时保持既有的路由、鉴权、配额与用量
语义：
- 路由与准入：control/route/ai.rs 识别 WebSocket 升级请求；
websocket/ingress.rs 复用 API Key 鉴权、IP 规则与并发许可，并引入
独立的 WebSocket 连接许可
- 中继：websocket/responses/* 按 connection / session / turn 分层，
帧解析归一化、socket 写入有界、continuation 保持调度亲和性
- 配额：orchestration/codex_quota_breaker.rs 在账号配额耗尽时熔断并
自动恢复，不再直接断开客户端连接
- 用量：每个 turn 的终态用量落库，request_metadata 记录
websocket_mode / websocket_transport，管理端与 usage 视图暴露
is_websocket
- 管理端：provider 可配置 Responses WebSocket 开关
2026-08-17 14:50:33 +08:00
elky
5b0c763086
fix(codex): fence concurrent quota updates
2026-08-14 09:28:07 +08:00
elky
050eb77508
fix(ai): harden responses replay and failure diagnostics
2026-07-30 17:19:54 +08:00
elky
1ab4f079c9
fix(gateway): restore failover and usage diagnostics
2026-07-30 09:12:11 +08:00
elky
a04673a90d
feat(gateway): harden failover and payload handling
...
Retry pre-response transport failures across candidates with an explicit stop policy, and propagate end-to-end timing into usage records and UI diagnostics.
Remove legacy body, import, cookie, PII, and tunnel replay caps while preserving optional operator-configured gateway limits.
2026-07-30 01:03:27 +08:00
zhefox
f8000012f7
fix(ci): stabilize cross-platform workflow checks
2026-07-29 21:55:43 +08:00
elky
531cf11025
feat(gateway): harden provider request execution
...
Preserve exact request payloads and model client surface and API operation explicitly.
Add Anthropic compatibility profiles, bounded stream commitment, and scoped OAuth retry behavior across provider transports.
2026-07-27 09:36:31 +08:00
elky
10d369f59c
feat(providers): add provider transfer limits
2026-07-26 15:06:56 +08:00
elky
713010fa0a
fix(gateway): restore auth role refresh and Rust checks
...
Refresh the resolved user role without bypassing owner group and key policies. Resolve Rust 1.95 Clippy failures and make the pending persistence bound test scheduler-independent.
2026-07-22 11:25:24 +08:00
elky
fc92c4f431
perf(gateway): scale request hot paths for 20k streams
...
Shard and singleflight hot-path caches, batch and prioritize candidate and usage lifecycle persistence, and extend database and pressure-test instrumentation for 20k concurrent streams.
2026-07-22 02:11:08 +08:00
elky
e58621a735
Merge PR #669 : align GPT-5.6 and Codex request protocols
2026-07-12 21:50:20 +08:00
elky
7f61bb43c7
feat(security): harden gateway request and runtime controls
2026-07-12 14:10:54 +08:00
MMEXA
0b30cc6b0f
feat(openai): unify tier authorization and settlement
2026-07-11 12:27:05 +08:00
elky
2f1cdb6a0b
Record exhausted usage failures synchronously
2026-07-02 16:08:04 +08:00
elky
6f00e9fc67
Improve gateway transport and usage runtime
2026-06-25 22:36:27 +08:00
elky
d336d1a7fa
Improve gateway scheduling and runtime admission
2026-06-24 01:53:45 +08:00
elky
e9b4efc2d4
fix(ai-serving): preserve explicit request encoding
2026-06-10 18:16:55 +08:00
elky
ef6a11c146
fix(gateway): preserve heartbeat no-path fallback
2026-06-03 00:25:01 +08:00
Entropy.Xu
bb5a4454a5
feat(gateway): 添加标准文本非流式心跳
2026-05-29 14:35:16 +08:00
fawney19
7e76c9763d
Clarify stream first byte timeout message
2026-05-26 15:02:50 +08:00
Novick Yuan
5dfccdec3e
Fix stream candidate watchdog timeout semantics
2026-05-25 21:43:13 +08:00
fawney19
d3249485fa
Fix stream timeout semantics
2026-05-25 20:09:37 +08:00
fawney19
d18b13a91a
fix: harden frontdoor and usage ingestion
2026-05-22 23:57:38 +08:00
stabey and Claude Opus 4.7
40434005c0
fix(gateway): use total_ms for non-stream upstream watchdog
...
When the endpoint forces upstream_stream_policy=force_non_stream while
the client streams, the local stream candidate watchdog still preferred
timeouts.first_byte_ms — a non-stream upstream produces no early first
byte, so the watchdog fired before the HTTP request_timeout and aborted
otherwise-healthy attempts at ~300s.
Read upstream_is_stream from report_context and invert the priority:
non-stream upstreams use total_ms first, falling back to first_byte_ms
and then the default; streaming upstreams keep the previous order.
Co-Authored-By: Claude Opus 4.7 <[email protected]
2026-05-21 15:26:42 +08:00
fawney19
1b570daf72
Revert "Merge PR #504 "
...
This reverts commit d216a9e21921e82abd54
2026-05-19 17:23:57 +08:00
fawney19
d8a9d7eb5e
chore: format merged PR changes
2026-05-19 08:13:56 +08:00
MMEXA
4317ff78b1
Expose local scheduling failures in usage UI
2026-05-18 16:28:37 +00:00
Kayphoon
2958041dc7
feat(gateway): add reversible chat pii redaction
2026-05-13 18:25:13 +08:00
fawney19
bacb14e5f0
refactor: lazy pool key scheduling
2026-05-11 00:12:05 +08:00
Entropy.Xu
5268f62a64
feat: add sync image heartbeat toggle
2026-05-10 12:40:52 +08:00
Entropy.Xu
3b4f27f767
Add Codex image progress heartbeat
2026-05-09 01:21:26 +08:00
fawney19
fd44906bb7
fix provider pool quota status handling
2026-05-07 11:28:44 +08:00
fawney19
f959f02d40
Implement transport profile routing
2026-05-05 22:21:23 +08:00
fawney19
a24e4a793d
Refactor pool candidate scheduling
2026-05-03 20:14:29 +08:00
fawney19
47ee8b9c13
refactor: fold ai surfaces into formats
2026-05-02 18:19:39 +08:00
fawney19
c130d0e2c9
refactor ai serving modules and crates
2026-05-02 13:23:54 +08:00
fawney19
4fc7cecf30
Fix OAuth token import and table filters
2026-05-01 02:14:49 +08:00
fawney19
07a319259b
Normalize canonical API formats
2026-04-29 10:20:41 +08:00
fawney19
3b542434a2
Unify candidate ranking pipeline
2026-04-27 17:24:31 +08:00
fawney19
4ec591fbf2
centralize openai responses alias handling
2026-04-26 23:59:53 +08:00
fawney19
5b914aa78c
migrate ai format conversion to responses adapters
2026-04-26 23:59:53 +08:00
fawney19
baa0ddd787
Improve runtime miss usage diagnostics
2026-04-26 01:44:54 +08:00
fawney19
429fdb47e6
fix(usage): 简化本地执行错误提示
2026-04-25 20:46:22 +08:00
fawney19
f3c9835759
feat(pool): 引入 pro_first 调度预设、Pool 候选持久化跳过与诊断信息优化
...
- 新增 pro_first 调度预设（Pro 优先），更新 plus_first 仅针对 Plus 计划，移除 free_team_first
- Pool 内部候选（pool_key_index 不为空）跳过 DB 持久化（available/skipped/unused 均适用）
- LRU 排序新增 catalog_lru_score 回退：runtime 无记录时使用 last_used_at_unix_secs
- 执行路径 miss 诊断消息细化为中文，按 reason 分类输出可读说明
- build_local_request_candidate_status_record 补充 extra_data 和 created_at_unix_ms 字段
- OpenAI CLI 计划构建流程补充候选评估进度跟踪与 terminal reason 设置
- 前端 PoolSchedulingDialog 增加 pro_first 预设展示，修复 LRU 默认预设检测逻辑
2026-04-24 13:29:05 +08:00
Entropy.Xu
0f94f92c37
fix(provider): 将rust分支的gemini cli端点行为对齐到python分支 ( #321 )
...
* fix(provider): 对齐 Vertex/Gemini 上游发包与 Python master
- provider-transport: 为 custom+aiplatform 推断 Vertex API key 上下文并统一 URL 构建顺序，复用共享 request_url 构建最终上游地址
- ai-pipeline/gateway: Vertex Gemini 路径改为仅使用 URL query key，不再向上游附带 x-goog-api-key header；同步对齐 standard/admin/test-connection/runtime miss 摘要中的最终 URL
- gemini conversion: 按 Python master 输出 Gemini 请求体，补齐 system_instruction / generation_config / tool_config / function_declarations 形态，并移植 Gemini schema 清洗逻辑
- scheduler/executor: 将最终 upstream_url、mapped_model、key_name 写入候选 extra_data，运行时 miss 诊断优先展示真实展开后的上游 URL 便于服务器排障
* fix(provider): 修复 Vertex provider 测试与本地调度链路
* fix(provider): 对齐 Vertex 本地执行与 Rust CI
2026-04-23 23:01:06 +08:00
fawney19 and Entropy.Xu
fa328e18a1
feat: provider api_formats 可空继承、OpenAI 图片 edit/variation 与用量配额多项补强
...
- 鉴权: provider_api_keys.api_formats 改为可空，OAuth 托管 key 自动继承 provider endpoints 激活格式，相关 handler/测试同步更新
- 图片 planner: OpenAI 图片路由新增 edit/variation 操作并完善参数校验、响应合并与流式处理
- 用量: user me usage 返回区分 client_requested_stream/upstream_is_stream，前端 usage 列表筛选与展示增强
- 统计: stats_daily_model 新增 cache_creation_ephemeral_5m/1h tokens 字段与回填链路
- 配额/observability: quota repository 新增内存与 SQL 扩展，admin observability usage 字段扩充
- 其它: OAuth 导入/轮询收敛、provider 汇总与 pool admin 读写链路小修、新增 system_config 缓存与 provider template handler
Closes #318
Co-authored-by: Entropy.Xu <[email protected]
2026-04-23 14:42:51 +08:00
Entropy.Xu
f55f22d2e8
feat(codex-image): 封装 GPT Image 2 图片接口并收紧错误处理
...
- 新增 openai:image 路由、planner 与 finalize，内部通过 Codex responses image_generation tool 执行生图
- 补充 Codex OAuth/header 兼容、图片 success report 本地处理与相关前后端/集成测试
- 禁止 chat/completions 使用 gpt-image-2，图片接口限制 n=1，并移除 Provider 模型页的图片能力开关
2026-04-22 22:46:28 +08:00
fawney