Compare commits

..
1470 Commits
Author SHA1 Message Date
elky 361952ada9 fix: resolve workspace lint and regression test failures 2026-09-09 11:34:45 +08:00
elky 6630856061 fix: harden routing failover, model testing, and wallet queries 2026-09-09 10:38:25 +08:00
elky a893bd0557 refactor(data): reuse payment order query 2026-09-09 09:21:09 +08:00
elky f2839ae6a7 feat(routing): add strategy failover controls 2026-09-09 09:12:09 +08:00
elky e58570d79d feat(routing): make client disconnect behavior strategy-scoped 2026-09-08 23:11:37 +08:00
elky 99f6499b2b fix(conversion): improve stream failures and diagnostic exports 2026-09-08 21:04:22 +08:00
elky 17d01d7fe0 fix(dns): unify provider resolution and bound SMTP and tunnel egress
Share provider DNS policy across WebSocket and connection probes, handle bracketed IPv6 literals, and preserve bounded address sets for outbound clients.

Bound SMTP DNS and TCP setup with multi-address fallback. Add opt-in trusted proxy DNS for tunnel upstreams while retaining default IP ACLs and origin isolation.

Document DNS policy boundaries and verify 809 gateway, tunnel, and HTTP regression tests.
2026-09-08 17:44:59 +08:00
elky 8b766930b0 fix(ci): resolve formatting, clippy and migration alias checks 2026-09-08 12:41:19 +08:00
elky c7e403b410 fix: restore container logging compatibility and normalize legacy policies 2026-09-08 11:43:51 +08:00
elky cf8ea19856 fix: harden OAuth identity and cookies and correct quota and JSON display 2026-09-08 10:51:25 +08:00
elky 7113d04f8a fix(usage): preserve original captured HTTP headers 2026-09-08 08:49:35 +08:00
elky 099b810a2f feat: optimize usage body viewing and provider card layout 2026-09-08 02:49:06 +08:00
elky 7aa0c89244 fix(gateway): restore HTTP and WS upstream support 2026-09-07 22:15:05 +08:00
elky 7847ae98c6 fix(gateway): reset stream first-byte timeout per candidate 2026-09-07 21:56:16 +08:00
elky a90d564931 fix: restore security hardening compatibility and validation
Restore authorized rule reveal, explicit full HTTP capture and retention, video task business fields, and valid payment URLs. Add opt-in credential preservation for trusted recovery, fix frontend type contracts and async races, and eliminate PostgreSQL test fixture resource leaks. Document audit coverage and successful fmt and CI-scoped Clippy checks.
2026-09-07 21:14:27 +08:00
github-actions[bot] a5c3699ae9 chore(tunnel): update download links for tunnel-v0.3.17 2026-09-07 08:06:59 +00:00
elky 7b8048c6ae chore(tunnel): release v0.3.17 2026-09-07 15:57:39 +08:00
elky ec95f2ca1f fix(tunnel): prevent stream stalls and harden session cleanup
Reliably deliver flow-control credits and terminal states, isolate slow streams and heartbeats, negotiate stream windows, and clean up cancelled streams and session tasks.

Add regression coverage for queue pressure, early cancellation, small-window streaming, drain, and reconnect. Validate 185 agent tests, 88 gateway tunnel tests, and 21 protocol tests.
2026-09-07 15:39:40 +08:00
elky aa7dbe67d3 feat(providers): add persistent card view and shared drag ordering 2026-09-07 14:13:59 +08:00
elky a26680f460 fix(modules): restore legacy SMTP password migration 2026-09-07 12:07:53 +08:00
elky 522b979052 refactor(transport): remove provider DNS filtering and allowlist settings 2026-09-07 12:07:50 +08:00
elky 808946312a fix(providers): retry initial empty quota before showing feedback 2026-09-07 11:18:31 +08:00
elky 741107bf71 fix(transport): make provider DNS address filtering opt-in 2026-09-07 11:18:31 +08:00
elky 6962731220 fix(antigravity): restore default OAuth client compatibility 2026-09-07 10:51:24 +08:00
elky 062e111c03 fix(observability): preserve admin upstream error diagnostics 2026-09-07 10:39:48 +08:00
fawney19 470c59e197 Merge pull request #804 from AAEE86/fix-frontend-eslint
Fix frontend ESLint issues
2026-09-07 10:06:40 +08:00
elky 2f929e74c7 fix(frontend): preserve session cleanup and Unicode navigation 2026-09-07 09:57:53 +08:00
AAEE86 fc0417ceb9 fix(frontend): resolve ESLint issues 2026-09-07 09:03:51 +08:00
elky 44174a31e0 chore: update architecture documentation ignore rules 2026-09-07 08:57:44 +08:00
elky b599fb7354 fix(frontend): complete i18n coverage and responsive layouts 2026-09-07 08:54:19 +08:00
elky 14f96c9fa0 fix(providers): preserve health in redacted key summaries 2026-09-07 08:53:41 +08:00
elky 6948852992 fix(release): support atomic installation on musl Linux 2026-09-07 01:07:34 +08:00
elky 1b01b08c31 fix(testkit): disable Unix sockets for temporary PostgreSQL
Use loopback TCP to avoid unwritable default socket directories on Ubuntu CI.
2026-09-07 00:37:56 +08:00
elky 2281f2b754 refactor(data): remove MySQL and SQLite support
Use PostgreSQL as the only database backend across runtime, schema tooling, installation, Compose, and CI. Update regression tests and reject removed drivers explicitly.
2026-09-07 00:09:42 +08:00
elky b5ed802277 chore(codex): bump default client version to 0.153.4 2026-09-06 19:58:35 +08:00
elky d1b5eb08ee fix(providers): correct endpoint health aggregation and display 2026-09-06 18:24:27 +08:00
elky dba5e6e9e9 feat: expose remote control as admin module 2026-09-06 00:13:57 +08:00
elky c125e78c5f fix(frontend): correct provider key total 2026-09-06 00:10:27 +08:00
fawney19 d1cb0ebecf Merge pull request #775 from fawney19/codex/provider-policy-hardening
feat(codex): add provider outbound policy boundary
2026-09-05 16:56:23 +08:00
elky 9d7a0665c0 Merge remote-tracking branch 'origin/main' into codex/provider-policy-hardening 2026-09-05 16:21:10 +08:00
fawney19 882bb43125 Merge pull request #732 from fawney19/subscription-usage-policies
feat(billing): add composable subscription usage policies
2026-09-05 15:35:57 +08:00
elky db6c522d60 fix(admin): use typed DNS config normalization error 2026-09-05 15:04:00 +08:00
elky e29442a06a merge(main): resolve subscription usage policy conflicts 2026-09-05 14:21:16 +08:00
elky e15ea0d5d3 fix(security): configure trusted Fake-IP DNS hosts 2026-09-05 14:05:07 +08:00
ZheFox 2f374d6af2 Merge pull request #802 from zhefox/fix/codex-catalog-ci
fix(ci): route Codex fingerprint through ai_serving facade
2026-09-05 13:43:28 +08:00
ZheFox 4a356f4ea5 fix(ci): access Codex version through the gateway serving facade 2026-09-05 13:33:20 +08:00
ZheFox c7676d567d Merge pull request #801 from zhefox/fix/codex-admin-model-catalog
fix(codex): refresh fingerprints and unify management model catalogs
2026-09-05 13:18:47 +08:00
ZheFox 5ca4f87951 fix(codex): unify management model catalogs and refresh fingerprints 2026-09-05 13:14:24 +08:00
ZheFox 1fee8954cc Merge pull request #788 from stabey/fix/antigravity-cross-format-envelope
fix(antigravity): wrap cross-format requests in the v1internal envelope
2026-09-05 11:49:03 +08:00
elky f69b770f5e fix(frontend): hide pricing source from model list 2026-09-05 09:56:34 +08:00
elky 856accdced test(data): expect revoked API-key ciphertext on import 2026-09-05 05:07:55 +08:00
elky 92749b4d6e fix(models): classify external catalog fetch failures 2026-09-05 04:56:29 +08:00
elky f08c2e6729 test(data): expect anonymized deleted API key names 2026-09-05 04:56:14 +08:00
elky e420bc6324 fix(ci): make postgres bootstrap tests apply pending migrations 2026-09-05 04:45:07 +08:00
elky d723fb92d3 fix(image): validate signatures and preserve output mime 2026-09-05 04:08:20 +08:00
elky 5b1de5f921 fix(tunnel): allow exact private relay hosts 2026-09-05 03:57:02 +08:00
elky 7ed48e7b58 fix(data): make postgres snapshot migrations idempotent 2026-09-05 03:44:10 +08:00
elky af712ebdbf test(gateway): keep capture limit fixtures test-only 2026-09-05 03:35:56 +08:00
elky 33d5cd5993 fix(ci): align lint-safe security paths 2026-09-05 03:19:53 +08:00
elky f5e1420ee6 fix(runtime): tolerate Linux socket device identity 2026-09-05 02:43:42 +08:00
elky b37b252b14 fix(ci): satisfy protocol utility clippy checks 2026-09-05 02:37:41 +08:00
elky 0097ea89ad fix(ci): document fixed tunnel auth transcripts 2026-09-05 02:31:57 +08:00
elky 10e63507f0 test(oauth): make antigravity exchange fixture deterministic 2026-09-05 01:48:12 +08:00
elky 9ff4d73d5c fix(merge): align security contracts with latest main 2026-09-05 01:31:21 +08:00
elky 0e3bd7eff4 merge(main): sync latest main into security branch 2026-09-05 00:30:16 +08:00
elky 1c89b5f9ab fix(models): allow manual entry during catalog load 2026-09-04 23:57:50 +08:00
elky cdbbda40a6 fix(tunnel): guard private owner relay targets 2026-09-04 23:44:46 +08:00
elky 29a9d608d9 fix(models): keep manual creation available on catalog failure 2026-09-04 23:11:23 +08:00
stabeyandClaude Opus 5 a6dc43d5f6 fix(antigravity): wrap cross-format requests in the v1internal envelope
The gemini:generate_content URL hook rewrites any Antigravity endpoint to
/v1internal:generateContent, but only the same-format passthrough and the two
OpenAI decision paths ever built the matching envelope. A Claude Messages or
Gemini client therefore reached the standard family planner, picked up the
rewritten URL, and posted a bare Gemini body that upstream rejects with
"Invalid JSON payload received. Unknown name \"contents\"" -- four retries
across every account, then a 503 that names none of this.

Route the standard family through the shared v1internal builder the same way
gemini_cli already is, so the URL and the body come from one decision. The
OpenAI-image-to-Gemini path cannot carry an envelope at all, so it now skips
Antigravity candidates instead of sending a request upstream can only reject.

Also stop treating a configured proxy as locally unsupported. The execution
plan carries the proxy itself, and the generic and Vertex gates moved to
transport_proxy_is_locally_supported long ago; Antigravity kept rejecting on
proxy.is_some(), which no longer matches how the local runtime executes. A
proxy that resolves to no route still disqualifies the request, and transport
profiles stay unsupported because the v1internal payload never carries one.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-04 22:53:09 +08:00
elky c6718754d3 fix(metrics): redact upstream admission target labels 2026-09-04 22:25:36 +08:00
elky afdd033745 fix(admin): validate cyber failover setting 2026-09-04 21:47:24 +08:00
elky d5f54ffe8b fix(image): scope synthetic DNS to ChatGPT storage 2026-09-04 21:41:14 +08:00
elky f5ec76c5c8 fix(models): isolate legacy catalog rows during fetch 2026-09-04 21:31:57 +08:00
elky 784a1e0611 test(oauth): make antigravity fixtures deterministic 2026-09-04 21:12:29 +08:00
elky 507cb33089 fix(workers): isolate malformed catalog proxy rows 2026-09-04 21:12:01 +08:00
elky b08fa3bdb6 fix(network): preserve remote DNS semantics for SOCKS proxies 2026-09-04 21:10:55 +08:00
elky 018af84d7d fix(usage): enforce canonical metadata projection 2026-09-04 20:42:24 +08:00
ZheFox 27b0381a9a Merge pull request #800 from zhefox/fix/pr745-sync-finalize
fix(gateway): complete cross-format sync finalization
2026-09-04 20:24:16 +08:00
ZheFoxandstabey 57cdef4b8d fix(gateway): harden cross-format sync finalization
Co-authored-by: stabey <[email protected]>
2026-09-04 19:53:26 +08:00
elky 36e9d21e3f fix(logging): sanitize transport error display 2026-09-04 19:49:01 +08:00
elky b72b6ab137 fix(workers): isolate legacy catalog credentials 2026-09-04 19:42:10 +08:00
ZheFox 30b2c8548a Merge pull request #799 from zhefox/fix/h2c-truncated-sse-ci
test(integration): stabilize truncated H2C stream ordering
2026-09-04 19:24:30 +08:00
elky 7c5cce4b3c fix(logging): redact oauth retry transport errors 2026-09-04 19:10:03 +08:00
elky 9362c34fcd fix(network): cover regional Kiro service origins 2026-09-04 19:09:46 +08:00
ZheFox 344b3031e9 test(integration): stabilize truncated h2c stream ordering 2026-09-04 18:59:52 +08:00
elky e89c3aa674 fix(models): resolve external catalog through configured proxy 2026-09-04 18:56:34 +08:00
ZheFox ddbbf835af Merge pull request #796 from AAEE86/main
ci: derive nightly image owner from repository
2026-09-04 18:24:11 +08:00
ZheFox cb58a63ee3 Merge pull request #797 from AAEE86/fix-routing-provider-model-filter
fix(routing): filter providers by selected model
2026-09-04 18:23:45 +08:00
ZheFox 6b1074cfcd Merge pull request #798 from zhefox/codex/antigravity-streaming-finish-reason
fix: harden stream lifecycle, Antigravity reasoning, and Gemini replay
2026-09-04 18:19:38 +08:00
elky d6894b5532 fix(oauth): tolerate synthetic DNS for trusted identity origins 2026-09-04 18:08:32 +08:00
elky 635c6765d9 fix(network): scope synthetic DNS exceptions for oauth and payments 2026-09-04 18:08:22 +08:00
ZheFox 86f7cc0d58 test(gateway): satisfy Rust 1.95 integer lint 2026-09-04 18:01:19 +08:00
ZheFox 206995645b fix(gateway): share stream first-byte deadline across retries 2026-09-04 18:01:19 +08:00
stabeyandClaude Opus 5 9282cce1d6 fix(gateway): settle stream attempts dropped before first byte
A local stream attempt writes its `usage` row and its `request_candidates`
slot as `pending` in `execute_execution_runtime_stream_inner`, then awaits
the provider's response headers. Everything after that point runs inside
the downstream request future, so a client disconnect drops it: the
dispatch `.await` never resumes and nothing settles either row. The stream
finalizer that already covers this only exists once upstream headers have
arrived, so the pre-first-byte window has no owner at all. Both rows stay
`pending` until the maintenance sweeper rewrites them as a 504 timeout ten
minutes later, losing the real outcome, the real latency, and the 499.

`AttemptCancellationGuard` takes that window. It is created disarmed, so
an attempt dropped before it owns any row does not grow a settlement row
it never had; it is armed as soon as the attempt owns its non-terminal
rows, and the stream wrappers disarm it the moment the attempt returns,
from where settlement belongs to the transport. On a cancelling drop it
settles the candidate slot through the same snapshot writer the `pending`
write above it uses, and the usage row through a terminal `Cancelled`
event.

The guard outlives the request future, so what it captures is retained for
the whole attempt. It therefore holds no request body: the plan carries the
provider request body and the report context carries the client request
body, and keeping both would double the request-body residency of every
in-flight stream attempt to serve a path that almost never runs. Simply
omitting them is not safe either, because a terminal write is
body-capture-authoritative: with both absent the seed carries the typed
`none` marker, which clears the stored capture rather than leaving it
alone. `build_usage_event_data_seed_describing_request_bodies` is the third
option -- it derives every capture state, body reference, request type and
derived request fact from the real plan and report context, and leaves out
only the two body values -- so the guard's snapshot is small and its
terminal write preserves the capture the `pending` write recorded.

The stream candidate first-byte watchdog also drops the attempt future, but
it settles the attempt itself through `build_transport_error_stop_response`.
It now marks the attempt abandoned before returning so the guard stands down
instead of racing a 499 against the watchdog's 504.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-04 17:40:32 +08:00
elky c005700a7e fix(network): tolerate synthetic DNS for trusted origins 2026-09-04 17:25:53 +08:00
ZheFox 14744abd57 fix responses replay across Gemini and Codex 2026-09-04 16:54:42 +08:00
ZheFox 66d6c17d2d fix antigravity reasoning streaming and terminal errors 2026-09-04 16:36:40 +08:00
elky c142d39951 feat(http): expose narrow benchmarking fake-ip predicate 2026-09-04 16:13:19 +08:00
AAEE86 1eb2d10dec fix(routing): filter providers by selected model 2026-09-04 15:25:32 +08:00
AAEE86 dabaeb8dfa ci: derive nightly image owner from repository 2026-09-04 14:50:56 +08:00
elky 2d17d4b73f refactor(models): clarify online pricing source label 2026-09-04 14:16:22 +08:00
ZheFox 18d78dd6c9 Merge pull request #795 from zhefox/codex/antigravity-import-email
test(gateway): align CI fixtures with routing contracts
2026-09-04 14:13:21 +08:00
elky 499942e3e7 fix(models): surface external catalog failures promptly 2026-09-04 14:00:59 +08:00
ZheFox ba11a72214 test(gateway): align CI fixtures with routing contracts 2026-09-04 13:54:50 +08:00
elky 12571764bc fix(models): support fake-ip DNS for official catalog 2026-09-04 13:52:34 +08:00
elky 1e13fa032c fix(frontend): fail fast on deterministic refresh errors 2026-09-04 13:24:17 +08:00
elky 47b21a25d3 chore(frontend): refresh browser compatibility data 2026-09-04 13:21:48 +08:00
elky 45a3ba8829 fix(frontend): avoid blocking public routes on invalid session 2026-09-04 13:14:16 +08:00
ZheFox 03f2914044 Merge pull request #794 from zhefox/codex/antigravity-import-email
Codex/antigravity import email
2026-09-04 13:11:08 +08:00
ZheFox c8d1ae3e7e test(codex): preserve reset credit fixture metadata 2026-09-04 13:02:22 +08:00
ZheFox c5ae9c2c77 fix(antigravity): sync discovered models into catalog 2026-09-04 12:06:39 +08:00
elky 4e47c00154 fix(dev): bootstrap embedded web dependencies 2026-09-04 12:03:57 +08:00
elky 313a637982 refactor(data): drop sqlite no-op migration 2026-09-04 11:47:05 +08:00
ZheFox fe8ff268df fix oauth identity and codex reset credits 2026-09-04 11:34:41 +08:00
elky bac6d6866a refactor(data): remove unpublished legacy cleanup migrations 2026-09-04 11:26:59 +08:00
elky 579f2c7cc1 feat(security): harden gateway boundaries and usage policies
Consolidate subscription usage policy enforcement, privacy-safe persistence, and gateway security hardening into one reviewable change.

Includes bounded HTTP and execution envelopes, header and protocol guards, DNS and relay validation, authentication and secret projection hardening, secure backup/install paths, and regression coverage.
2026-09-04 03:45:52 +08:00
elky ddcbeb3ae9 Merge remote-tracking branch 'origin/main' into worktree-linear-enchanting-bunny 2026-09-03 22:32:25 +08:00
ZheFox e25fc984af Merge pull request #792 from zhefox/codex/antigravity-import-email
fix(antigravity): preserve imported account email
2026-09-03 22:25:57 +08:00
ZheFox 4cf47b1dee fix(antigravity): preserve imported account email 2026-09-03 22:22:33 +08:00
elky 95cbd43097 test(gateway): align local execution fixtures with routing 2026-09-03 22:10:11 +08:00
ZheFox 09005939bf Merge pull request #791 from zhefox/codex/model-pricing-source-selection
feat(frontend): persist model pricing sources
2026-09-03 21:48:48 +08:00
ZheFox 7b612b8b5a feat(frontend): persist model pricing sources 2026-09-03 20:49:42 +08:00
elky 670d5e8d33 Merge remote-tracking branch 'origin/main' into worktree-linear-enchanting-bunny 2026-09-03 20:49:18 +08:00
elky 1de2e70d41 test(gateway): seed default routing strategy in request fixtures 2026-09-03 20:39:48 +08:00
ZheFox 89b57464d2 Merge pull request #790 from zhefox/codex/fix-antigravity-quota
fix(frontend): deduplicate Antigravity quota groups
2026-09-03 19:59:46 +08:00
ZheFox 09ef3adf70 fix(frontend): deduplicate Antigravity quota groups 2026-09-03 19:22:42 +08:00
ZheFox 3dfc15963c Merge pull request #789 from zhefox/codex/fix-antigravity-quota
Codex/fix antigravity quota
2026-09-03 18:48:58 +08:00
ZheFox f6884eb8c4 fix(gemini): accept signature-only reasoning exhaustion 2026-09-03 17:57:40 +08:00
ZheFox f8b4382a54 test(gateway): align Claude response message ids 2026-09-03 17:04:22 +08:00
ZheFox d78b5a81fb fix(gemini): preserve thought-only max-token responses 2026-09-03 17:04:12 +08:00
ZheFox 89fe9e9f0a fix(runtime): avoid JSON precision stream regression 2026-09-03 16:10:30 +08:00
elky 4291a91dc0 test(gateway): pass routing policy to affinity effects 2026-09-03 15:50:53 +08:00
ZheFox 4c6bafe255 Merge remote-tracking branch 'upstream/main' into codex/fix-antigravity-quota 2026-09-03 15:50:22 +08:00
ZheFox 979dbc4b33 fix(data): preserve jsonb numbers for metadata CAS 2026-09-03 14:37:10 +08:00
elky 9309ad844f test(gateway): align routing fixtures with strategy policies 2026-09-03 14:11:15 +08:00
ZheFox d672ba2068 test(gateway): seed routing strategy for antigravity flows 2026-09-03 13:45:57 +08:00
ZheFox 587486ab0c Merge remote-tracking branch 'upstream/main' into codex/fix-antigravity-quota 2026-09-03 13:24:39 +08:00
ZheFox 40a5e1470d fix(gateway): persist quota refresh from strong catalog reads 2026-09-03 13:16:40 +08:00
elky 058660ec2e fix(data): include routing order in postgres snapshot 2026-09-03 12:51:37 +08:00
elky 668bf5e40f test(data): include latest migration snapshot 2026-09-03 12:27:12 +08:00
elky 77f93c638d Merge codex/routing-strategy-consolidation into main 2026-09-03 12:00:51 +08:00
elky d0c0996b9f Merge origin/main into provider billing form fix 2026-09-03 11:36:49 +08:00
fawney 2cb4d554aa feat(routing): consolidate scheduling strategy configuration 2026-09-03 11:05:59 +08:00
ZheFox 76fb8905c9 test(gateway): align antigravity response message id 2026-09-03 11:04:18 +08:00
ZheFox f822df6cce Merge remote-tracking branch 'origin/main' into codex/fix-antigravity-quota 2026-09-03 10:46:49 +08:00
ZheFox 45c840b8d3 fix(providers): refresh Antigravity grouped quotas 2026-09-03 10:46:44 +08:00
elky 214f3d6406 fix(providers): hide billing fields in provider form 2026-09-02 23:17:01 +08:00
fawney e8d9877b79 Merge remote-tracking branch 'origin/main' 2026-09-02 22:08:36 +08:00
fawney19 cae9aa4134 Merge pull request #784 from fawney19/worktree-linear-enchanting-bunny
feat(routing): move sticky-key retries into routing policy with lazy attempts
2026-09-02 20:56:03 +08:00
elky 7323d41fbe feat(routing): move sticky-key retries into routing policy with lazy attempts
Replace the provider/endpoint max_retries fields as the source of same-key
retries with a routing policy setting, sticky_key_attempts (default 2). Only
the first-ranked candidate is retried on the same key; every failover
candidate gets a single attempt so failover keeps advancing instead of
retrying each fallback key.

Materialize exactly one attempt per candidate and derive same-key retries in
the attempt loop after a candidate-scoped failure, so the retry budget no
longer inflates up-front materialization and needs no upper bound. The budget
travels in the report context; retries reuse the plan with a fresh candidate
id and incremented retry index. Pool groups only retry their first key within
the retry-index stride.

Expose the setting in the routing profile editor and the set_scheduling rule
action, and drop the max_retries input from the provider form.
2026-09-02 20:48:40 +08:00
fawney19 e3644c6142 Merge pull request #779 from fawney19/worktree-linear-enchanting-bunny
feat(routing): make routing profiles the sole scheduler policy source
2026-09-02 17:16:57 +08:00
elky 415b2da81b feat(routing): make routing profiles the sole scheduler policy source
Bootstrap an enabled system-default routing group from the legacy
scheduler config keys on startup, resolve the default ordering config
from that group before falling back to the legacy keys, and stop merging
keep_priority_on_conversion with the legacy flag when a policy is
resolved. Thread the policy-derived ordering config into candidate
preselection so it no longer reads system config independently.

Add per-API-format key priority overrides so a key serving several
formats keeps independent ordering, matching the legacy
global_priority_by_format semantics. Expose keep_priority_on_conversion
in the routing profile editor and read the effective policy in the
model routing preview, monitoring metrics and provider page badge.
2026-09-02 17:04:04 +08:00
zhefox cc6f5e89b6 Merge remote-tracking branch 'zhefox/main' into zhefox-main 2026-09-02 15:57:43 +08:00
zhefox 2ed2cc66ef fix(formats): retain response field normalization import 2026-09-02 15:44:15 +08:00
zhefox b1bf7837cf style: format quota and response changes 2026-09-02 15:40:40 +08:00
zhefox 77229943d1 fix(pool): preserve generic quota windows for model scheduling 2026-09-02 15:29:56 +08:00
zhefox a0369cf49a Merge remote-tracking branch 'zhefox/main' into zhefox-main
# Conflicts:
#	crates/aether-admin/src/provider/quota.rs
#	crates/aether-ai/formats/src/formats/openai/chat/stream.rs
#	crates/aether-ai/formats/src/formats/openai/responses/mod.rs
#	crates/aether-provider/pool/src/provider.rs
#	crates/aether-provider/pool/src/quota.rs
2026-09-02 15:25:27 +08:00
zhefox dbbe7b22ab fix(pool): isolate dynamic model quota buckets and 429 scheduling 2026-09-02 15:23:23 +08:00
elky 166236c2ee Merge origin/main into CI dependency fix 2026-09-02 12:30:31 +08:00
ZheFox 0371a961d6 Merge pull request #777 from zhefox/main
Fix detection of DeepSeek custom relay models in gateway
2026-09-02 11:24:55 +08:00
ZheFox 144a28f544 feat(admin-users): add plan entitlement revocation flow 2026-09-02 11:23:24 +08:00
elky 611c29f1f5 ci: install VSCodex web dependencies in nightly build 2026-09-02 11:17:06 +08:00
ZheFox 6540c1e46a Merge branch 'fawney19:main' into main 2026-09-02 10:26:07 +08:00
ZheFox 24bf92a8bf Merge pull request #776 from zhefox/fix/deepseek-reasoning-text-replay
fix(gateway): detect DeepSeek custom relay models
2026-09-01 23:04:51 +08:00
zhefox 7ae984df4b fix(gateway): detect DeepSeek custom relay models 2026-09-01 22:49:35 +08:00
zhefox e2154629ca fix(gateway): detect DeepSeek custom relay models 2026-09-01 22:49:25 +08:00
ZheFox 0bfd48b9db Merge pull request #774 from Kayphoon/codex/fix-openai-responses-ping
fix(formats): ignore Responses ping stream events
2026-09-01 22:17:18 +08:00
elky d5f34b2ee2 feat(codex): add provider outbound policy boundary 2026-09-01 21:21:42 +08:00
Kayphoon 88d2b002be fix(formats): ignore Responses ping stream events 2026-09-01 12:58:21 +00:00
fawney 30a75832f8 feat(vscodex): add remote Codex collaboration module 2026-09-01 20:25:35 +08:00
fawney 5059093d29 feat(vscodex): add remote Codex collaboration module 2026-09-01 20:08:36 +08:00
ZheFox 5a69cfe40d Merge pull request #772 from zhefox/main
fix(gateway): handle pool saturation and malformed Gemini calls
2026-09-01 19:31:44 +08:00
ZheFox 3d87bbf230 style(rust): apply workspace formatting 2026-09-01 19:31:13 +08:00
ZheFox 633363e190 fix(gateway): handle pool saturation and malformed Gemini calls 2026-09-01 19:25:00 +08:00
fawney19 715f2773c3 Merge pull request #773 from fawney19/codex/codex-fingerprint-convergence
refactor(codex): generalize fingerprint convergence
2026-09-01 17:27:04 +08:00
elky d07dc86376 refactor(codex): generalize fingerprint convergence 2026-09-01 17:05:54 +08:00
elky ef7caa40e7 ci: publish nightly builds from main 2026-09-01 16:43:29 +08:00
fawney19 7fb8d5fc0a Merge pull request #771 from fawney19/codex/codex-context-stability
Merge Phase 1 Codex context and fingerprint convergence changes.
2026-09-01 16:07:41 +08:00
elky 3e540ce589 fix(gateway): route Codex context through transport facade 2026-09-01 15:53:47 +08:00
ZheFox 6c71f87589 fix(frontend): align Antigravity quota summaries 2026-09-01 15:38:00 +08:00
elky a39048ecce feat(codex): stabilize identity across retries 2026-09-01 15:33:40 +08:00
ZheFox b538aa2d66 Merge pull request #770 from zhefox/main
fix(pool): show Antigravity quota reset times
2026-09-01 12:09:41 +08:00
ZheFox 57abb20778 fix(pool): show Antigravity quota reset times 2026-09-01 11:29:48 +08:00
ZheFox d117a0cd13 Merge pull request #769 from zhefox/main
fix(pool): restore Antigravity quota progress bars
2026-09-01 10:43:05 +08:00
ZheFox ee55f46962 fix(pool): restore Antigravity quota progress bars 2026-09-01 10:40:37 +08:00
ZheFox 9210502e77 Merge pull request #768 from zhefox/main
fix(pool): isolate model quotas and compact account display
2026-09-01 10:20:09 +08:00
ZheFox 2fe2600021 fix(pool): isolate model quotas and compact account display 2026-09-01 10:15:57 +08:00
ZheFox 9b819169d5 Merge pull request #767 from zhefox/fix/provider-key-concurrency-cache-affinity
fix(gateway): improve provider pool concurrency, quotas, and affinity
2026-09-01 08:11:46 +08:00
ZheFox 9631b229b3 fix(gateway): add provider key concurrency and cache affinity modes 2026-09-01 08:06:58 +08:00
ZheFox 9372d6cfa5 Merge pull request #765 from Brisbanehuang/codex/usage-api-template
feat(provider-ops): 新增通用 API Key 用量查询模板
2026-08-29 23:23:06 +08:00
Brisbanehuang 4dbf98163e feat(provider-ops): add generic usage API template 2026-08-29 09:04:05 -04:00
ZheFox 6ec0771297 Merge pull request #764 from zhefox/main
fix(gateway): route Responses compaction only to Responses providers
2026-08-29 12:56:32 +08:00
zhefox 56395945c0 fix(gateway): route Responses compaction only to Responses providers 2026-08-29 12:28:55 +08:00
ZheFox 8032497045 Merge pull request #763 from zhefox/main
Fix response reasoning summaries for Chat
2026-08-29 11:54:15 +08:00
zhefox b35364d7fd fix(antigravity): normalize private search tool name 2026-08-29 11:09:15 +08:00
ZheFox f085fdf918 Merge pull request #762 from zhefox/main
fix(gemini): normalize mixed tools for same-format providers
2026-08-29 08:48:56 +08:00
ZheFox 36daba7a34 fix(antigravity): align tool schema wire fields 2026-08-29 08:45:19 +08:00
ZheFox 9837ce1197 fix(antigravity): use Gemini schema field for tools 2026-08-29 02:50:07 +08:00
ZheFox 1bc2287baa fix(gemini): normalize mixed tools for same-format providers 2026-08-29 01:08:01 +08:00
ZheFox a519bcf705 Merge pull request #761 from zhefox/main
Fix response reasoning summaries for Chat
2026-08-29 00:03:16 +08:00
ZheFox 9461b1004f Merge branch 'main' of https://github.com/zhefox/Aether 2026-08-29 00:00:13 +08:00
ZheFox 3c15f523be Merge pull request #760 from zhefox/fix/gemini-tool-wire-model-gating
fix(formats): gate mixed Gemini tools by model
2026-08-28 23:36:08 +08:00
ZheFox 5bcdcca784 fix(formats): normalize Responses additional tools for Chat 2026-08-28 23:03:56 +08:00
ZheFox 83098f98b6 fix(formats): gate mixed Gemini tools by model 2026-08-28 20:42:49 +08:00
ZheFox 5ab35ae6ba Merge pull request #759 from zhefox/fix/gemini-tool-schema-compat
fix(formats): enable mixed Gemini tool calls
2026-08-28 17:58:03 +08:00
ZheFox f0b0064f3d fix(formats): enable mixed Gemini tool calls 2026-08-28 16:55:38 +08:00
ZheFox 4879295f23 Merge pull request #758 from zhefox/fix/gemini-tool-schema-compat
fix(formats): sanitize Gemini tool schemas
2026-08-28 16:15:24 +08:00
ZheFox 64e5725331 fix(formats): sanitize Gemini tool schemas 2026-08-28 16:11:16 +08:00
ZheFox 1995198b18 Merge pull request #757 from zhefox/fix/responses-chat-reasoning-summary
fix(formats): degrade Responses reasoning summaries for Chat
2026-08-28 14:09:57 +08:00
ZheFox 5b6fce1a77 fix(formats): degrade Responses reasoning summaries for Chat 2026-08-28 14:05:48 +08:00
ZheFox fa8e443f7b fix(formats): degrade Responses reasoning summaries for Chat 2026-08-28 14:03:43 +08:00
ZheFox 08e7530adb Merge pull request #756 from zhefox/main
fix: preserve Gemini signatures and enforce provider limits
2026-08-28 14:01:43 +08:00
ZheFox 5687dad177 fix(formats): scope signature helper to tests 2026-08-28 13:21:35 +08:00
ZheFox dd2958a458 fix(admin): persist provider settings and enforce quotas 2026-08-28 12:47:54 +08:00
ZheFox c4b4dfa996 fix(formats): preserve Gemini tool thought signatures 2026-08-28 12:47:34 +08:00
ZheFox d88c454a2c Merge pull request #754 from zhefox/main
fix(ai): align Gemini and Responses compatibility
2026-08-28 08:15:35 +08:00
ZheFox 8cdfa338e5 fix(gemini): pair idless tool history 2026-08-28 02:15:00 +08:00
ZheFox 4da8c57fe3 fix(ai): align Gemini and Responses compatibility 2026-08-27 22:13:20 +08:00
ZheFox 7892aa9485 Merge pull request #753 from zhefox/fix/codex-namespace-tool-conversion
fix(formats): preserve Responses namespace tools through Chat
2026-08-26 01:13:53 +08:00
ZheFox 9d9892be6a fix(gateway): finalize cross-format sync JSON responses 2026-08-26 00:43:58 +08:00
ZheFox e2b003af24 fix(formats): preserve Responses namespace tools through Chat 2026-08-26 00:43:50 +08:00
ZheFox ffca7e0402 Merge pull request #752 from zhefox/main
fix(codex): isolate Spark quotas and repair contaminated account state
2026-08-25 22:13:15 +08:00
ZheFox ec6ddb43a7 fix(data): retain applied legacy backfill for upgrades 2026-08-25 21:22:57 +08:00
ZheFox 2f2d444f97 fix(codex): self-heal Spark-contaminated account quotas
Keep model-scoped Spark windows out of account state, preserve authoritative WHAM exhaustion flags, and repair historical cross-family quota generations without weakening stale-response guards.
2026-08-25 19:13:41 +08:00
stabeyandClaude Opus 5 42deab67b3 fix(admin): keep a model-scoped 429 reset out of the account quota slot
`parse_codex_websocket_usage_limit_error` backfills `primary_reset_at` and
`primary_reset_after_seconds` from the error body whenever the embedded headers
did not supply them. Now that a named per-model limit no longer claims the
unprefixed window headers, that absence is exactly what a Spark 429 produces —
and `resets_at` on such an error is the Spark window's reset, so the backfill
put model-scoped timing back on the account's own quota.

Gate the backfill on the same ownership rule the window parsing uses.

Reported by Cursor Bugbot on the fork PR.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-23 15:22:29 +08:00
stabeyandClaude Opus 5 1b1be918a9 fix(admin): keep Codex model-scoped rate-limit headers out of the account quota
`parse_codex_usage_headers` reads the unprefixed `x-codex-primary/secondary-*`
headers as the account's own quota. They are not: they carry whichever limit
governed the request, and `x-codex-active-limit` names it — `premium` for the
plan's own limit, or a metered feature such as `codex_bengalfox` for a named
per-model limit. On a request billed against a named limit the unprefixed
headers repeat that limit's windows verbatim.

So a single request to a model with its own limit writes that model's windows
into the account slots. The paid-window swap then makes it worse: a named
limit's secondary window is active, unlike the plan's disabled one, so the swap
promotes the model's weekly window into the account's weekly slot — the slot
the UI labels and the scheduler reads through `quota_usage_ratio`.

It also sticks. Both weekly windows share `window_minutes`, so
`codex_quota_same_window_identity` treats them as one window, and
`codex_quota_merge_same_window` drops an observation whose deadline is earlier
than the stored one. The two weeks start at different instants, so every later
account observation looks like a stale sample of a window that already rolled
over and is discarded until the model window's own deadline passes.

Observed on a `pro` key running both model families: one `gpt-5.3-codex-spark`
request replaced the account weekly window with the Spark weekly one, and the
~3000 plan-limit responses over the next 100 minutes were all discarded. The
account's real weekly usage never landed, and its reset time was reported nine
hours late.

The header set describes itself — every named limit announces
`x-codex-<feature>-limit-name` and carries its windows under the same prefix —
so parse the named families directly and only claim the unprefixed windows for
the account when no announced limit owns them. Responses without
`x-codex-active-limit` keep the previous behaviour.

This also stops the Spark windows from going stale: they were only ever written
by the `wham/usage` admin probe even though every response carries them.

Refs #746

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-23 15:22:29 +08:00
ZheFox 3f2b67f191 Merge pull request #749 from zhefox/main
test(gateway): fix Codex Realtime route fixture
2026-08-23 12:48:10 +08:00
ZheFox 6a9eea34a0 test(gateway): fix Codex Realtime route fixture 2026-08-23 04:00:16 +08:00
ZheFox 453a0b3ee7 Merge pull request #747 from zhefox/main
fix(gateway): support current Codex Realtime live routes
2026-08-23 03:18:46 +08:00
ZheFox 2cd20da1ec fix(gateway): support current Codex Realtime live routes 2026-08-23 02:51:49 +08:00
ZheFox ea4453321d Merge pull request #743 from zhefox/main
fix(usage): unify OpenAI Live and WebSocket records
2026-08-21 12:47:37 +08:00
ZheFox acde38b8e7 fix(usage): unify OpenAI Live and WebSocket records 2026-08-21 11:53:53 +08:00
ZheFox 9996e75a34 fix(ci): align database snapshot migration cutoff 2026-08-21 11:11:45 +08:00
ZheFox 16f96d73ec Merge pull request #742 from zhefox/main
feat(gateway): add Codex Live and OpenAI Realtime
2026-08-21 08:36:19 +08:00
ZheFox 2c89202001 feat(gateway): add Codex Live and OpenAI Realtime
Implement preflighted Live/Realtime WebSocket transports, protocol-aware authentication, usage auditing, UI filtering, and legacy Codex permission migration.
2026-08-21 04:27:34 +08:00
ZheFox fe38dcd294 Merge pull request #741 from zhefox/main
feat(gateway): add Codex Live transport
2026-08-20 22:22:37 +08:00
ZheFox 4185ad1b1e feat(gateway): add Codex Live transport 2026-08-20 21:59:05 +08:00
ZheFox 6916e9da76 Merge pull request #739 from zhefox/main
fix(ws): secure Responses continuation and opaque reasoning replay
2026-08-20 10:01:46 +08:00
ZheFox 654f798d25 fix(ws): harden Responses continuation state 2026-08-20 08:51:16 +08:00
ZheFox bef282cfee fix(responses): replay DeepSeek opaque reasoning state 2026-08-20 00:40:48 +08:00
ZheFox d21d8ce9f5 fix(codex): avoid replaying static config on websocket continuations 2026-08-20 00:40:40 +08:00
ZheFox 342f8b6a5f Merge pull request #737 from zhefox/main
fix(ws): preserve Codex continuations across turn metadata
2026-08-18 18:20:49 +08:00
ZheFox c50a1c6c46 fix(ws): preserve Codex continuation bindings 2026-08-18 17:17:45 +08:00
elky 535ee098c3 fix(auth): reject unsigned admin identity headers 2026-08-18 11:12:17 +08:00
ZheFox b45df89ce4 Merge pull request #730 from zhefox/fix/responses-websocket-current
feat(gateway): add OpenAI Responses WebSocket mode
2026-08-17 21:14:50 +08:00
ZheFox c8118edf36 fix(ws): harden Responses connection lifecycle
Revalidate control policy per turn, isolate downstream credentials, and make planner/turn ownership cancellation-safe.

Preserve opaque protocol events, align configurable timeout semantics, and extend end-to-end security and settlement coverage.
2026-08-17 18:50:29 +08:00
AAEE86 4a0775c4ea style: apply cargo fmt across gateway and aether-ai crates 2026-08-17 14:53:53 +08:00
AAEE86 6fc02dad3e fix(ws): restore redacted PII in provider frames before client delivery
Responses WebSocket 只实现了脱敏的一半:请求侧 mask 之后,provider 事件帧在推给
客户端之前没有还原,于是 session 映射内的占位符以 <AETHER:EMAIL:...> 的形式直接
透给客户端。这里补齐响应侧,语义与 HTTP 路径对齐。

- 还原点是 relay loop 的最后一跳(send_client_message 之前、capture_client_frame
  之前),对应 HTTP 的 restore_sync_response_body / StreamingResponseRestorer 所在
  位置。审计与终态观测继续消费脱敏态事件,只有发往客户端的那一份拷贝被还原。
- 复用 privacy::restore_json_strings(改为 pub(crate))与
  RedactionSession::restore_text,不复制任何还原逻辑:只还原本 session mask 过的
  映射,未映射的占位符原样保留;type / model / id 等协议字段不可能命中 sentinel,
  因此不受影响。批量 {"chunks":[...]} 帧一并递归还原。
- session 生命周期:mask 仍然是 per-turn(slot 依旧每轮新建),但 session 改由连接
  持有,按有界 FIFO 留最近 8 轮。理由是 WS 的会话历史留在上游,continuation 只发
  增量输入,per-turn 释放会漏还原后续响应里回显的更早轮次占位符;HTTP 不会漏,是
  因为它每次重发整段历史、重新 mask 会派生出同一个 sentinel。被挤出窗口的轮次退回
  「占位符原样透传」,不会错误还原成别的值。
- 未命中还原时不改写字节;连接上没有任何 mask session 时(未启用脱敏)连事件 clone
  都不做。

测试:redaction.rs 新增 8 条单测(还原命中/批量帧/未映射占位符原样/未命中不改写/
无 session 不介入/空 session 不留存/审计侧入参不被改写/跨轮还原/窗口有界);
responses_websocket_e2e 新增一条用例,mock 上游回显收到的 input,断言上游只看到
占位符而客户端拿到真实邮箱。
2026-08-17 14:53:46 +08:00
AAEE86 dbf2809bd6 fix(ws): settle the previous attempt before transparent retry replanning
评审第 2 条。配额透明重试原来的顺序是「detach 旧 attempt → 规划并绑定新
attempt → 把旧 attempt 的结算排进队列」。规划因此读到的是旧 attempt 还没投射的
health / adaptive / pool 状态,而且旧 attempt 仍占着自己的 pool key lease——替代
key 的挑选看到的是一把仍被占用的 key,最坏情况下判成「无可用供应商」而放弃一次
本可以成功的重试。

普通的新 turn 早就挡住了这件事:client.rs 在处理 response.create 前调用
await_pending_turn_finalization,注释写的正是「不要让新 turn 基于陈旧的 health /
adaptive / pool 状态规划」。透明重试是同一个问题的另一条入口,漏了这一步。

现在顺序是:detach → 释放准入 → 结算旧 attempt 并等它落地 → 规划/绑定新 attempt。

新增 lifecycle::settle_turn_finalization:与 queue_turn_finalization 的区别只在于
「等」。后者把 handle 挂在连接上让 relay loop 继续跑,用在结算之后不再读取共享
状态的出口;前者用在必须先看到结算结果才能继续的路径上。

顺序用类型固定,而不是靠注释:settle_turn_finalization 返回
PreviousAttemptSettled,retry_active_turn_after_quota_exhaustion 要求这个参数。
凭证只能由 lifecycle 颁发(结算完成,或明确「没有 attempt 要结算」),所以把顺序
写反连编译都过不了。

重试失败路径随之变化:旧 attempt 已经结算,不再 resume 回去。logical turn 仍停在
Replanning,后续分支的 end() / finalize_active_turn 只清 logical turn、不交出
attempt,因此不存在重复结算。结算 outcome 取值不变(两条路径用的都是
terminal_outcome.unwrap_or_else(upstream_closed),而这条分支里 terminal_outcome
必为 Some——usage_limit_error 成立意味着有一个已解析的 error 终态帧)。

代价(都落在「重试失败」这一侧,且只影响已终态 attempt 的报告注解,不影响计费):
- 那条最终转发给客户端的 429 事件不再进旧 attempt 的 client capture;
  provider 侧 capture 早在 observe_upstream_frame 里就记下了。
- 如果转发 429 给客户端也失败,record_client_delivery_aborted 落在一个已经结算的
  attempt 上,成为 no-op。

测试:
- lifecycle:await_turn_finalization_handle 必须「等到落地」而不是「排进队列」
  (C6 依赖的性质);结算完成后规划才读状态的顺序型断言(计数器替身);结算任务
  panic 也必须放行调用方,不能卡死 relay loop。
- turn_state:Replanning 状态下 end() 不再交出第二个 attempt(无重复结算)。
- e2e 新增 provider_quota_exhaustion_transparently_retries_onto_another_key:
  mock 上游首轮只回 Codex 的 429 usage_limit_reached,网关换到第二把 key 重放同一个
  response.create;断言客户端看不到 429、上游被连两次、两次用的不是同一把 key、两个
  attempt 各留一条终态行(429 的那条 + 计费的那条)。已验证它在改动前后都通过——
  它覆盖的是整条路径可用,顺序由上面的单测确定性覆盖。
  夹具随之参数化出 ProviderFixture::CodexKeyPair:透明重试只有 Codex adapter 会
  开启,而 codex 候选要求 auth_type = oauth,所以这个夹具用未过期的 oauth 凭证。
2026-08-17 14:53:40 +08:00
AAEE86 1d3051cb89 refactor(ws): structured terminal observation without SSE text round-trips
评审第 5 条:Responses WebSocket 收到的本来就是结构化协议事件,但为了复用面向
SSE 的 push_line,观测路径要先把每个事件序列化成 data: {json}\n\n,解析器再
decode 回 Value——一次纯粹的往返。这个「伪 SSE」形状是随手拼的,一旦拼装函数
以后被加上换行或分块逻辑,观测结果就会和真实事件悄悄分叉。

aether-ai-formats:
- OpenAIResponsesProviderState::push_line 机械拆成 decode + push_event,
  push_line 现在只做解码。协议状态机一行未动,diff 里除函数签名外只有
  &value → value(value 从拥有改成借用,持有结构化事件的传输不必为了调用它
  先克隆一份)。
- StreamingStandardTerminalObserver::push_event 走 TerminalStreamParser::Standard,
  service tier 的记录方式与 push_line 完全相同。openai:image 的终态状态机按 SSE
  行做增量解析、没有结构化入口,返回 AiSurfaceFinalizeError 让调用方
  disable_with_error 标记 parser_error,而不是静默丢事件、把摘要留成「未观察到
  终态」。ProviderStreamParser 的其余三个格式同样返回 Err:机械拆分随时可做,
  但不建无调用方的接口。

WS 侧:
- 新增 responses/observation.rs 的 ResponsesStructuredTerminalObserver,直接消费
  frame.protocol_events() 借出的事件。包一层的意义是让「不再拼 SSE」成为类型层面
  的事实——这个类型没有任何接受字节的方法,改回 push_line 不可能悄悄发生。
  finish() 里的 Ok(None) / Err → disable_with_error 兜底也一并收进来。
- body capture 不动,仍然是 SSE 形状(data: 开头、\n\n 结尾):
  aether_usage_runtime::report 用 line.strip_prefix("data:") 解析被捕获的 body
  判定 StreamCapturedTerminalState,而它是 stream_report_represents_failure 的一个
  OR 项,换成结构化 JSON 会让终态判定恒为 Missing。capture_sse_event /
  capture_client_frame / websocket_event_as_sse_line 全部保留,原因写在模块文档
  注释里。这一层只换观测,不换捕获。

差分测试(8 个,aether-ai-formats):同一组事件序列分别走 push_line 与
push_event,断言 ExecutionStreamTerminalSummary 完全相等——批量 delta 序列、
completed 带 usage、合法 incomplete、error、response.failed、未知事件、
service tier、缺终态;外加 openai:image 拒绝结构化入口。两条入口不可能有
过滤差异:任何 Value 序列化出来都不会命中 decode_json_data_line 的 empty /
":" / "event:" / [DONE] 四个过滤条件。

turn.rs 里三个既有的 WS 观测测试改走结构化入口;SSE 形状的断言留在 capture 一侧。
验收:crates/aether-usage 零 diff。
2026-08-17 14:53:33 +08:00
AAEE86 59e27524da refactor(gateway): extract transport-neutral execution attempt lifecycle
评审第 4 条:responses/turn.rs 实际复制了一整套 HTTP execution lifecycle——
usage 写入、candidate 状态流转、health/adaptive 效果投射、pool key lease 释放、
body capture、账单失败判定,与 HTTP 的顺序和超时语义只能靠人工对齐。

新增 execution_runtime/attempt_lifecycle.rs,把一次 provider attempt 的记账收成
transport 中立的三段:

  ExecutionAttemptLifecycle::begin        pending usage 行 + Pending candidate
  ExecutionAttemptLifecycle::mark_started usage stream_started + Streaming candidate(幂等)
  ExecutionAttemptLifecycle::settle       终态四段,顺序不可重排:
                                            1 usage terminal(detachable,不可丢)
                                            2 candidate terminal
                                            3 provider 效果 + 超时兜底释放 lease
                                            4 execution report(作废账单不提交)

顺序、5s 分段超时常量、detachable 语义、「每个效果分支都释放 lease」「作废账单
一律不提交 report」这些不变量全部保持原样。

一并上移的辅助设施:
- AttemptStageGuard 取代 await_websocket_lifecycle_stage /
  await_detachable_lifecycle_stage,把「等多久」参数化:WS 用 Bounded(5s),
  HTTP 接线时用 Unbounded 即保持它现在的语义。
- AttemptBodyCapture 取代 append_capture / encode_stream_capture,把
  「缓冲 + 截断标志」两个字段收成一个类型(WS 侧四个字段变两个)。捕获内容
  仍然是 SSE 形状:usage runtime 按 data: 行解析被捕获的 body 来判定
  StreamCapturedTerminalState,换成结构化 JSON 会让终态判定恒为 Missing。
- C2/C3 的结算表本来就不含任何 WS 类型,随之上移。效果表分支与注释逐字未改,
  仅按新位置改名为 AttemptProviderEffect / classify_attempt_provider_effect。
  responses/settlement.rs 只保留 WS 专属的一件事:把 relay loop 的结算信号
  ResponsesWebSocketTurnOutcome 翻译成两个正交事实。

ResponsesProviderAttempt 现在只持有 WS 专有状态:lifecycle 句柄、deadline、
终态观测器、两侧 capture、准入、provider/delivery 事实。plan / trace_id /
report_kind / report_context / candidate 起始时间戳都归 lifecycle。

HTTP 侧不接线:execution_runtime/stream/execution.rs 的
DirectPassthroughFinalizerCore(38 字段)与 failover / oauth 重试 / prefetch 深度
纠缠,无法在「行为等价 + 单 commit 可验证」的前提下改动。逐调用点映射表写在
模块文档注释里作为后续 PR 的接线依据。验收:git diff 对
execution_runtime/stream/ 与 crates/aether-usage 均为零 diff。

新增 6 个测试:效果段超时后仍走兜底 lease 释放、Unbounded 会一直等、detachable
写入在调用方停止等待后仍跑完、settle 四段顺序(计数器替身)、body capture 的
SSE 形状与编码状态(并显式记录默认上限是 usize::MAX,截断分支不可达)、
candidate error_type 映射。
2026-08-17 14:53:25 +08:00
AAEE86 247e7105a2 fix(ws): bill a provider-reached terminal even when client delivery fails
评审第 5 条后半:provider 终态已经到达、只是 gateway 写客户端 socket 失败时,
relay loop 用 client_disconnected() 覆盖了结算信号,于是一条供应商已经完成推理
并消耗了 token 的响应被记成 void billing、candidate 记 Cancelled、不投射供应商
效果、也不提交 execution report。上游成本凭空消失。

结算表只改一行:作废账单的条件从
    provider.cancelled_by_provider() || delivery.is_aborted()
收紧为
    provider.cancelled_by_provider() || (delivery.is_aborted() && !provider.is_terminal())

于是 Terminal{cancelled=false} + delivery Aborted 与 delivery Complete 落在同一侧:
Billed、candidate Success 或 Failed、投射供应商效果、提交 execution report。
状态码随之变成纯 provider 事实(不再把 200 改写成 499);作废分支的 provider
状态码本身就是 499,取值不变。

依据:供应商已经完成推理并消耗 token,客户端还能用 previous_response_id 续取
这条响应。供应商没给出终态时(客户端先走了)仍然作废,这一侧未改。

配套改动:
- connection.rs 写客户端失败处改为 record_client_delivery_aborted(reason) +
  settle_signal_for_client_delivery_failure(terminal_outcome):provider 终态已到达
  就用那条终态作结算信号,不再无条件覆盖。投递失败原因也不再谎称
  「客户端在终态前断开」。
- 投递结果记在 attempt 上而非 logical turn 上:结算按 attempt 进行,且配额透明
  重试时各 attempt 的投递结果彼此独立。
- report_context 新增 websocket_client_delivery="aborted" 与
  websocket_client_delivery_reason,只增字段不改既有字段,便于事后区分
  「客户端拿到了」和「客户端没拿到但已计费」。
- candidate error_type 新增 client_delivery_failed(原先这个场景写的是
  websocket_cancelled)。它排在供应商侧分类之前:这条记录之所以特别正是因为
  内容没送到客户端,供应商侧判定仍由 candidate_status 与 error_message 保留。
- finish_summary 改用作废判定而非「投递失败」判定:provider 终态已到达时摘要
  必须保留真实的 finish_reason 与 usage,否则计费记录会被写坏。

e2e 期望值变化:client_disconnect_mid_turn_still_settles_the_usage_row 改名为
client_disconnect_before_any_provider_output_settles_a_void_row,并补上
「不计费 + status=cancelled + status_code=499」的断言。原用例的 mock 行为是
StallAfterCreated(只发 response.created 就静默),provider 从未给出终态,所以
它走的是未改动的作废一侧;原来的文档注释说「must still be billed」与实际语义
不符,一并纠正。真正被修正的那一行无法在 e2e 里确定性触发——它取决于 relay
loop 的 select! 先观察到上游终态帧还是先观察到已关闭的客户端 socket,是构造性
竞态——因此由 relay 级单测确定性覆盖,e2e 里以注释指向这两个单测。

新增 7 个测试:结算表修正行(并与「投递成功」逐字段对照,只有 candidate 错误
分类不同)、无终态时仍作废、供应商声明取消即使送达也不计费、结算信号选择、
已记录的投递失败不被结算信号覆盖、relay 级「终态到达 + 客户端已关闭 ⇒ Billed /
Success / ProviderSuccess / 已提交 report 且 usage 完整保留」及其镜像、
report_context 只增不改。
2026-08-17 14:53:12 +08:00
AAEE86 dc3743aecf refactor(ws): 拆分 LogicalTurn 与 ProviderAttempt,结算改表驱动
评审第 5 条:一个 ResponsesWebSocketTurn 同时代表 logical turn 和 provider
attempt,finalize() 又用 outcome.cancelled() 一个布尔驱动 billing、candidate
状态和供应商效果,于是 provider 终态已经到达、只是最后一跳写客户端失败时,
供应商事实会被 Cancelled 覆盖掉。

- ResponsesWebSocketTurn → ResponsesProviderAttempt,
  ActiveResponsesWebSocketTurn → ActiveProviderAttempt:类型名字明确它只代表
  一次上游执行,logical turn 由 C1 落地的 LogicalTurn 承担。
- 新增 settlement.rs:AttemptProviderOutcome × AttemptClientDelivery 两个正交
  事实,classify_attempt_settlement 一张表推出 status_code / billing /
  candidate 状态 / candidate 错误分类 / 供应商效果 / 是否提交 execution report。
- attempt 观察到 provider 终态即记录 provider_outcome。结算信号
  ResponsesWebSocketTurnOutcome 只回答「为什么现在结算」:ProviderTerminal 与
  Failure 对 provider 是权威的,Cancelled 只描述客户端/连接层面的停止,不再
  覆盖已观察到的 provider 事实。
- candidate 状态与 candidate 错误分类分开输出:现状存在
  「missing_terminal=true 而记账层判 Success」的组合(report kind 不要求观察到
  终态事件时),会写出 status=Success + error_type=stream_missing_terminal_event,
  这个组合必须原样保留。

classify_responses_websocket_turn_effect 的判定表原样搬入 settlement.rs,分支
和顺序均未改动,两个既有不变量测试随之迁移。

行为等价。结算表当前口径与拆分前完全一致:客户端投递失败仍与「供应商声明取消」
落在同一侧(作废账单、candidate 记 Cancelled、只释放 lease、不提交 execution
report),即使 provider 终态已经到达——这一行由
settlement_table_row_client_delivery_failure_currently_voids_a_reached_terminal
锁住现状,修正它是下一步独立的行为修正。

新增 15 个测试:outcome → 双事实映射表逐行(含 stream_timeout 只在 504 失败一族
成立、provider 终态即使 504 也不投射流式超时)、结算表逐行、投递失败时
forced_error 必须为 None、已观察终态不被 Cancelled 覆盖、以及跨整张表的
「每个分支都释放 pool key lease」「作废账单一律不提交 report」不变量。
2026-08-17 14:53:05 +08:00
AAEE86 1c5ee5228c refactor(ws): 用 ResponsesTurnState 收敛连接 turn 状态
评审第 2 条:BoundResponsesConnection 用 response_in_flight、active_turn、
active_response_create 三个可独立变化的字段编码同一件事,8 种组合里只有 3 种
合法,非法组合只能靠调用点的 if 和「记得同时改另外两个字段」来避免。

三字段合并为一个 ResponsesTurnState:

  Idle                                  没有进行中的 logical turn
  Responding { logical, attempt }        logical 与 attempt 必须同时存在
  Replanning { logical }                 attempt 已取走去结算/重绑,logical 仍在

Replanning 不是新概念:配额透明重试期间现状就处于这个状态,只是靠
Option::take 意外得到。转换只能走 begin / detach_attempt / resume / end,
response_in_flight 与「是否接受新 response.create」都由变体推导。

由此消除的运行时不变量(原来全靠调用点自觉):
- 有 attempt 必有 logical turn
- response_in_flight 与 attempt 同生共死(原来 client 写失败后
  active_turn=None 而 response_in_flight 仍为 true)
- logical turn 结束时必须清 attempt:原来 `active_response_create = None`
  在 connection.rs 里手写 13 处,漏一处就残留;现在只有 end() 一个出口
- 上游绑定返回的连接不再自带 response_in_flight=true 的半成品状态

同时删除 update_response_in_flight:Started 帧把已经是 true 的字段再设一次,
Close 帧因为没有解析出的 frame 而根本不触发,是纯冗余写;它在 Idle 态收到
Started 帧时还会把 response_in_flight 置真,从而永久阻塞后续 response.create。

行为等价。ActiveResponsesWebSocketRequest 改名 LogicalTurn 并随状态机移入
新的 turn_state.rs;状态机对 attempt 类型泛型化,测试用轻量替身驱动同一套
转换逻辑,无需 AppState 或真实 socket。
2026-08-17 14:52:57 +08:00
AAEE86 9d80281b53 fix(ws): route WS planning and continuation through PII redaction 2026-08-17 14:52:49 +08:00
AAEE86 3b036299d4 fix(ws): enforce absolute upstream handshake and initial-message deadlines 2026-08-17 14:52:39 +08:00
AAEE86 f70ae68273 fix(ws): treat max_output_tokens incomplete as legitimate terminal 2026-08-17 14:52:33 +08:00
AAEE86 1353d76e07 feat(frontend): Responses WebSocket 配置与用量展示
provider 表单支持开启 Responses WebSocket;用量列表、状态与详情
区分 WebSocket 请求。
2026-08-17 14:52:25 +08:00
AAEE86 621a528083 test(ws): Responses WebSocket 端到端套件接入 CI
补齐 aether-integration-tests 的 responses_websocket_e2e 集成测试,
并把 CI 的 scenario 任务从 --bins 改为 --bins --tests,否则该套件
不会被执行。
2026-08-17 14:51:24 +08:00
AAEE86 a498875591 feat(gateway): Responses WebSocket 连通性探针
新增 aether-codex-ws-probe 与 aether-openai-responses-ws-probe 两个
二进制,用于在不暴露凭据的前提下验证上游 WebSocket 端点可用性:凭据
只从环境变量读取,不写入日志。公共流程放在
bin/support/responses_ws_probe.rs,各 profile 只负责自己的鉴权与
请求头要求。
2026-08-17 14:51:18 +08:00
AAEE86 71b54070e8 feat(gateway): Codex/OpenAI Responses WebSocket 代理模式
在 /v1/responses 上支持 WebSocket 升级,把客户端帧中继到上游 Codex /
OpenAI Responses WebSocket 端点,同时保持既有的路由、鉴权、配额与用量
语义:

- 路由与准入:control/route/ai.rs 识别 WebSocket 升级请求;
  websocket/ingress.rs 复用 API Key 鉴权、IP 规则与并发许可,并引入
  独立的 WebSocket 连接许可
- 中继:websocket/responses/* 按 connection / session / turn 分层,
  帧解析归一化、socket 写入有界、continuation 保持调度亲和性
- 配额:orchestration/codex_quota_breaker.rs 在账号配额耗尽时熔断并
  自动恢复,不再直接断开客户端连接
- 用量:每个 turn 的终态用量落库,request_metadata 记录
  websocket_mode / websocket_transport,管理端与 usage 视图暴露
  is_websocket
- 管理端:provider 可配置 Responses WebSocket 开关
2026-08-17 14:50:33 +08:00
ZheFox 9a0d346ff3 Merge pull request #728 from zhefox/main
fix(gateway): stop candidate persistence retry storms
2026-08-17 14:28:11 +08:00
ZheFox 32944538e9 fix(gateway): stop candidate persistence retry storms 2026-08-17 13:49:12 +08:00
ZheFox 0b17026eab Merge pull request #726 from zhefox/main
fix(codex): restore upstream model discovery
2026-08-15 20:16:12 +08:00
ZheFox b13d9b9b40 fix(codex): restore upstream model discovery 2026-08-15 19:36:28 +08:00
ZheFox b7fca851b8 Merge pull request #724 from zhefox/main
fix(codex): serve versioned dynamic model catalogs
2026-08-14 19:08:53 +08:00
zhefox 810c3dfe2b fix(codex): serve versioned dynamic model catalogs 2026-08-14 18:41:44 +08:00
elky a1d64e5239 fix(routing): preserve allowlist edits and save state 2026-08-14 11:43:24 +08:00
elky fb33ea57b0 Merge PR #715: decouple routing model overrides 2026-08-14 11:10:21 +08:00
elky 5b0c763086 fix(codex): fence concurrent quota updates 2026-08-14 09:28:07 +08:00
elky f3a12c1008 fix(ai): preserve Codex image edit validation 2026-08-13 11:31:17 +08:00
elky ca35e09eaa Merge pull request #718 from zjm54321/fix/custom-image-edit-json 2026-08-13 11:00:15 +08:00
elky 8cf381b0c3 feat(codex): add OAuth fingerprint convergence 2026-08-13 09:57:17 +08:00
elky 654c4f6978 fix(auth): preserve turnstile while typing email 2026-08-12 16:56:18 +08:00
elky edb8362adc fix(provider): omit default model test temperature 2026-08-12 16:56:18 +08:00
elky 29fa4aed19 perf(gateway): raise default server pool floor 2026-08-12 16:56:18 +08:00
zjm54321 41e93858e1 refactor(ai): simplify image edit serialization 2026-08-11 00:36:33 +08:00
zjm54321 8d918d0459 fix(ai): serialize image edits with images array 2026-08-11 00:30:00 +08:00
ZheFox 3a759fae89 Merge pull request #712 from zhefox/fix/claude-code-conversion-api-key-toggle
fix: restore Claude Code conversion and user API key toggles
2026-08-05 14:53:03 +08:00
zhefox 985ff3c36a test(gateway): align claude_code endpoint reconciliation 2026-08-05 14:34:20 +08:00
zhefox 908d4f2603 style(provider): apply rustfmt to claude_code tests 2026-08-05 13:58:46 +08:00
zhefox 4d67569873 fix(gateway): support claude_code cross-format Claude messages 2026-08-05 13:57:20 +08:00
ZheFox 1aab31a148 Merge pull request #710 from zhefox/main
fix: align Responses compatibility, routing, and model permissions
2026-08-03 19:31:04 +08:00
zhefox aedff9a704 fix(provider): validate mapped model reasoning effort 2026-08-03 19:22:51 +08:00
zhefox 669f4bddc5 fix: align Responses routing and model permissions 2026-08-03 18:48:01 +08:00
zbs 1a4eede34d fix(routing): decouple model overrides from allowed scope 2026-08-03 07:52:49 +08:00
elky 0318808db9 fix(providers): allow transfer limits on creation 2026-07-31 13:35:08 +08:00
elky 06f5d3c8c0 fix(gateway): complete worker registration cleanup 2026-07-31 11:32:07 +08:00
elky 082407fa51 Merge PR #697: prevent duplicate worker registrations 2026-07-31 11:11:14 +08:00
fawney19 6688ee26db Merge pull request #702 from MMEXA/fix/reconcile-auth-channel-mismatch-formats
fix(gateway): 修复批量更新 API 格式时的认证通道状态冲突
2026-07-31 10:28:37 +08:00
elky beb003b7ad feat(models): add external catalog proxy selection 2026-07-31 09:32:25 +08:00
MMEXA 6ecfe0f0a1 fix(gateway): reconcile auth mismatch formats on key update 2026-07-30 22:14:08 +08:00
ZheFox 12057db476 Merge pull request #701 from zhefox/main
Persist OpenAI Responses continuation history across instances
2026-07-30 21:08:34 +08:00
ZheFox ff47d8d48a fix(gateway): route response history through ai seam 2026-07-30 20:34:41 +08:00
ZheFox ef5f36cc2b fix(ai): satisfy response history clippy checks 2026-07-30 20:13:36 +08:00
ZheFox 84022c4d48 Merge upstream/main into main 2026-07-30 19:40:39 +08:00
ZheFox 118f441029 feat(gateway): persist OpenAI Responses continuation history 2026-07-30 19:26:52 +08:00
elky 20399b004d Merge PR #700: fix admin pool batch update body buffering
Preserve main's failover and usage metadata fixes, restore default tunnel regression coverage, and satisfy current Clippy.
2026-07-30 17:56:37 +08:00
elky 050eb77508 fix(ai): harden responses replay and failure diagnostics 2026-07-30 17:19:54 +08:00
elky 1ab4f079c9 fix(gateway): restore failover and usage diagnostics 2026-07-30 09:12:11 +08:00
MMEXA 6c733f7590 fix(usage): preserve request diagnostics in event seeds 2026-07-30 06:44:59 +08:00
MMEXA d7d8db45ba test(gateway): align tunnel error fixture with failover policy 2026-07-30 06:44:59 +08:00
MMEXA 8cf9af79da fix(ci): remove redundant usage policy update 2026-07-30 05:45:38 +08:00
MMEXA e55793c765 fix(ci): satisfy gateway clippy on upstream baseline 2026-07-30 05:14:34 +08:00
MMEXA d8902ea612 fix(gateway): buffer admin pool batch update bodies 2026-07-30 05:14:34 +08:00
elky a04673a90d feat(gateway): harden failover and payload handling
Retry pre-response transport failures across candidates with an explicit stop policy, and propagate end-to-end timing into usage records and UI diagnostics.

Remove legacy body, import, cookie, PII, and tunnel replay caps while preserving optional operator-configured gateway limits.
2026-07-30 01:03:27 +08:00
ZheFox a97acc07fc Merge pull request #698 from zhefox/main
fix(ci): stabilize cross-platform workflow checks
2026-07-29 22:16:17 +08:00
zhefox f8000012f7 fix(ci): stabilize cross-platform workflow checks 2026-07-29 21:55:43 +08:00
worker-2 6080f8cc88 fix(gateway): stabilize worker task records
Key worker boot records by task so process restarts update the existing
row instead of registering another row for each gateway instance.

Closes #693
Confidence: high
Scope-risk: narrow
2026-07-29 17:27:51 +08:00
ZheFox 37df5b93b1 Merge pull request #696 from zhefox/main
Fix client metadata handling across formats
2026-07-28 18:12:59 +08:00
ZheFox e53abdaec2 Merge branch 'fawney19:main' into main 2026-07-28 18:12:28 +08:00
ZheFox 2db32ea97e Merge branch 'main' of https://github.com/zhefox/Aether 2026-07-28 17:40:46 +08:00
ZheFox 581897ee74 fix(formats): ignore responses client metadata across targets 2026-07-28 17:40:41 +08:00
ZheFox 9a88f966d8 Merge pull request #695 from zhefox/main
Enhance provider capabilities and clean up OAuth keys
2026-07-28 13:58:33 +08:00
ZheFox 9d9316e434 Merge branch 'fawney19:main' into main 2026-07-28 13:56:34 +08:00
ZheFox 1b697b1111 feat(providers): support FedRAMP Codex agent identity registration 2026-07-28 13:29:30 +08:00
ZheFox 3043982486 fix(providers): derive Codex primary quota label from window 2026-07-28 12:57:45 +08:00
ZheFox 0bf92ffffc feat(providers): advertise responses API agent capability 2026-07-28 12:02:03 +08:00
ZheFox f0f87b56a3 feat(providers): add credential-fenced OAuth key cleanup 2026-07-28 11:32:11 +08:00
elky 4148ab1931 fix(routing): harden routed pool scheduling 2026-07-27 22:06:28 +08:00
elky 550cc36760 feat(providers): expand OAuth account management
Add Claude Code manual and cookie authorization, including redacted batch tasks. Harden OAuth imports, duplicate replacement, provider dialogs, and related account-management tests.
2026-07-27 15:53:28 +08:00
elky 531cf11025 feat(gateway): harden provider request execution
Preserve exact request payloads and model client surface and API operation explicitly.

Add Anthropic compatibility profiles, bounded stream commitment, and scoped OAuth retry behavior across provider transports.
2026-07-27 09:36:31 +08:00
elky 79b70f7b5c fix(frontend): align sidebar collapse button 2026-07-26 15:07:51 +08:00
elky 10d369f59c feat(providers): add provider transfer limits 2026-07-26 15:06:56 +08:00
elky 2ef7ac79bc feat(frontend): add collapsible navigation sidebar
Persist the desktop sidebar state, provide accessible compact navigation tooltips, and cover the collapsed navigation markup with a focused component test.
2026-07-25 21:28:51 +08:00
elky 778cfb1a5c feat(data): complete portable SQL backend parity
Align MySQL and SQLite schemas, migrations, usage, stats, export, and backfill behavior with the shared data contracts. Extend gateway startup and maintenance support across all SQL drivers.
2026-07-25 21:28:21 +08:00
elky 764e9fd131 feat(frontend): improve provider detail drawer and pool actions 2026-07-25 11:16:59 +08:00
elky 387134ca87 fix(models): correct fast pricing and online sync 2026-07-24 01:45:38 +08:00
elky a0767d957c fix(frontend): synchronize pool account state 2026-07-23 16:20:18 +08:00
ZheFox b94ef91d07 Merge pull request #692 from zhefox/main
Sync global model prices and track online pricing sources
2026-07-23 16:02:17 +08:00
ZheFox e7910751d9 Merge branch 'fawney19:main' into main 2026-07-23 15:19:48 +08:00
ZheFox 1d2655432d feat(models): track online pricing sources and unsupported fields 2026-07-23 15:18:08 +08:00
ZheFox 323273ff30 feat(models): sync global model prices from online catalog 2026-07-23 13:29:28 +08:00
ZheFox fb2009c65b Merge pull request #691 from zhefox/main
fix(formats): ignore Responses client transport metadata
2026-07-23 12:18:40 +08:00
ZheFox e186cc6848 Merge branch 'main' of https://github.com/zhefox/Aether 2026-07-23 12:17:46 +08:00
ZheFox 615ac99ad7 fix(formats): ignore Responses client transport metadata 2026-07-23 12:16:44 +08:00
ZheFox ec36cfbf75 Merge pull request #690 from zhefox/main
fix(provider): classify deleted Codex agent runtime as invalid
2026-07-23 11:22:30 +08:00
ZheFox 7bf228a33c fix(provider): classify deleted Codex agent runtime as invalid 2026-07-23 11:21:55 +08:00
elky 3606290ac8 fix(provider): harden Agent Identity OAuth lifecycle 2026-07-23 09:33:00 +08:00
elky e49024d33b fix(frontend): shorten Agent Identity tab label 2026-07-22 20:26:49 +08:00
elky fdbc2607ec feat(provider): add dedicated Codex Agent Identity flow 2026-07-22 20:19:29 +08:00
elky c7cc8fd7db test(provider): simplify agent identity assertions 2026-07-22 14:19:58 +08:00
elky 07efcb5146 fix(data): repair legacy active flag synchronization 2026-07-22 14:19:34 +08:00
elky 856605defa fix(model-directives): harden suffix configuration 2026-07-22 14:19:09 +08:00
elky 713010fa0a fix(gateway): restore auth role refresh and Rust checks
Refresh the resolved user role without bypassing owner group and key policies. Resolve Rust 1.95 Clippy failures and make the pending persistence bound test scheduler-independent.
2026-07-22 11:25:24 +08:00
ZheFox cd2fbeeead Merge pull request #689 from AAEE86/feat/agent-identity-support
feat(codex): enroll agent identity from session token
2026-07-22 10:32:06 +08:00
AAEE86 a4350a482a feat(codex): enroll agent identity from session token 2026-07-22 10:21:11 +08:00
ZheFox c825375367 Merge pull request #688 from AAEE86/feat/agent-identity-support
feat(codex): support agent identity accounts
2026-07-22 09:20:11 +08:00
elky fc92c4f431 perf(gateway): scale request hot paths for 20k streams
Shard and singleflight hot-path caches, batch and prioritize candidate and usage lifecycle persistence, and extend database and pressure-test instrumentation for 20k concurrent streams.
2026-07-22 02:11:08 +08:00
AAEE86 b61c590bdb feat(codex): support agent identity accounts 2026-07-21 20:58:49 +08:00
ZheFox 7756c0913f Merge pull request #685 from zhefox/main
fix(gateway): apply group policy to admin-owned keys
2026-07-20 15:52:06 +08:00
ZheFox c34ec7c1ee fix(gateway): apply group policy to admin-owned keys 2026-07-20 15:51:01 +08:00
elky f8778c4a23 feat(gateway): configure cyber policy failover 2026-07-19 23:27:19 +08:00
elky e0dbb233f7 fix(frontend): avoid misleading cache TTL fallback label 2026-07-19 22:21:46 +08:00
elky 9725f9abae fix(frontend): clarify processing tier pricing 2026-07-19 22:00:12 +08:00
elky 5d575f1590 test(stats): treat bulk API key snapshots as authoritative 2026-07-19 19:07:04 +08:00
elky d562c594c3 fix(frontend): preserve compact scope and detail badge 2026-07-19 16:42:32 +08:00
MMEXA ce226a3010 Merge 0c3f51bcec into 644ae9c1bf 2026-07-19 16:12:37 +08:00
elky 644ae9c1bf feat(pool): add table-driven account batch actions 2026-07-19 16:09:20 +08:00
elky 95053f9502 Merge PR #672: 支持账号批量配置与可用模型管理 2026-07-18 22:06:32 +08:00
elky 8fbda84acb fix(data): preserve API key history end to end 2026-07-18 21:58:21 +08:00
elky 03b7d573e0 Merge PR #683: decouple API key historical identity 2026-07-18 21:20:35 +08:00
MMEXA 0c3f51bcec merge(main): 解决 usage 模型展示契约冲突 2026-07-18 19:26:14 +08:00
elky e3d97b573b fix(usage): align fast-tier pricing and model metadata 2026-07-18 16:57:04 +08:00
MMEXA ac3796af84 fix(gateway): 恢复响应边界并统一格式入口 2026-07-18 06:45:37 +08:00
MMEXA f9c343eb07 fix(gateway): 适配 Rust 1.95 整除检查 2026-07-18 05:55:06 +08:00
MMEXA e31df5989a merge(main): 解决 usage 展示与生命周期同步冲突 2026-07-18 05:38:38 +08:00
MMEXA 98fbf029fc fix(data): 解耦 API Key 历史统计身份 2026-07-18 04:42:45 +08:00
MMEXA 4d9a648202 test(gateway): 统一流错误测试的格式层入口 2026-07-18 03:37:31 +08:00
MMEXA 405ca3e66a fix(ci): 恢复非流式错误体边界并适配新版 Clippy 2026-07-18 03:26:53 +08:00
MMEXA 0355c28683 fix(data): 解耦候选记录的 API Key 历史身份 2026-07-18 02:40:34 +08:00
fawney19 6c33b8d8fb Merge pull request #682 from MMEXA/codex/codex-prompt-cache-identity-20260717
fix(codex): 统一通用缓存键与原生会话身份
2026-07-18 00:33:20 +08:00
elky a6c6f14b09 style(frontend): align pool cycle stats values 2026-07-18 00:13:35 +08:00
elky e558f55cd9 style(frontend): refine badges and cycle stats 2026-07-18 00:05:22 +08:00
elky 88a057b8d9 fix(usage): force fast badge background transparent 2026-07-17 22:56:54 +08:00
elky ed27d404ac style(usage): make fast badge background transparent 2026-07-17 22:52:47 +08:00
elky 5dda34c66e style(usage): give fast tier an amber accent 2026-07-17 22:36:55 +08:00
elky 373ebf26d6 fix(pricing): default zero tier ratios to one 2026-07-17 21:33:21 +08:00
elky f65ed2795c fix(codex): support dynamic quota windows 2026-07-17 20:18:04 +08:00
elky 664c063a06 feat(usage): enrich audit metadata and detail views 2026-07-17 19:20:16 +08:00
MMEXA 75795c6fbc test(codex): 对齐 Compact 确定性缓存身份 2026-07-17 08:49:35 +08:00
MMEXA 5b332da7d7 fix(codex): 补齐缓存身份终态请求头 2026-07-17 08:11:16 +08:00
MMEXA d9796d502b fix(codex): 统一通用缓存键与原生会话身份 2026-07-17 06:13:09 +08:00
MMEXA 3b0d87b0fd Merge remote-tracking branch 'origin/main' into codex/pool-key-bulk-management-20260714 2026-07-17 00:17:02 +08:00
MMEXA 3c348dff3a Merge remote-tracking branch 'origin/main' into codex/usage-pending-reasoning-reset-expiry-20260712
# Conflicts:
#	frontend/src/features/usage/components/__tests__/UsageRecordsTable.spec.ts
2026-07-17 00:16:58 +08:00
MMEXA ec1783a35c Merge remote-tracking branch 'origin/main' into codex/pool-key-bulk-management-20260714
# Conflicts:
#	apps/aether-gateway/src/handlers/admin/request/provider/tasks.rs
#	frontend/src/api/endpoints/pool.ts
2026-07-16 23:43:04 +08:00
MMEXA 427030c5de Merge remote-tracking branch 'origin/main' into codex/usage-pending-reasoning-reset-expiry-20260712
# Conflicts:
#	crates/aether-ai-formats/src/formats/openai/responses/mod.rs
#	crates/aether-usage/runtime/src/runtime.rs
#	frontend/src/features/usage/components/UsageRecordsTable.vue
#	frontend/src/features/usage/components/__tests__/UsageRecordsTable.spec.ts
2026-07-16 23:41:58 +08:00
elky 0be380243b feat(pricing): support processing tier multipliers 2026-07-16 23:30:42 +08:00
fawney19 312583f055 Merge pull request #680 from Kayphoon/codex/s3-backup-user-agent
feat(admin): configure S3 backup User-Agent
2026-07-16 23:30:30 +08:00
fawney19 33f49ea9b0 Merge pull request #678 from AAEE86/fix
fix: map Developer role to "system" in OpenAI Chat Completions output
2026-07-16 23:29:55 +08:00
fawney19 470cef17cf Merge pull request #676 from MMEXA/codex/sync-capture-envelope-finalize-20260716
修复同步 finalize 的 Responses 流聚合与转换
2026-07-16 23:29:38 +08:00
ZheFox 3f5f65eb9a Merge pull request #681 from zhefox/main
Codex 重置功能和显示缓存修复以及批量key的导入和管理功能
2026-07-16 19:48:33 +08:00
ZheFox 6664c2dbb8 feat(pool): 支持批量导入 Key 和选择性更新设置 2026-07-16 19:31:07 +08:00
ZheFox 0099167a6d fix(codex): 避免重置机会缺失触发配额刷新 2026-07-16 19:13:09 +08:00
ZheFox f009fb73c3 缓存问题修复 2026-07-16 18:55:28 +08:00
ZheFox 715a5ed626 修复重置次数缓存问题 2026-07-16 18:09:10 +08:00
ZheFox 5cf38d1b35 Codex 重置功能和显示修复 2026-07-16 17:23:41 +08:00
Kayphoon 6b707f29a2 feat(admin): configure S3 backup User-Agent 2026-07-16 08:56:56 +00:00
elky 9ea84f9748 fix(frontend): show service tier transitions 2026-07-16 16:38:30 +08:00
elky c32d043afb fix(frontend): preserve fetched model preset pricing 2026-07-16 16:38:30 +08:00
elky 8fe4d24408 fix(usage): canonicalize cached token totals 2026-07-16 16:38:30 +08:00
elky e369e4aab1 fix(formats): preserve chat-backed Responses metadata 2026-07-16 16:38:30 +08:00
ZheFox 7dc919e8e3 Merge pull request #679 from zhefox/main
fix(frontend): 优化移动端弹窗并完善提供商配额刷新
2026-07-16 15:48:49 +08:00
ZheFox 1333efdad5 fix(frontend): 优化移动端弹窗并完善提供商配额刷新 2026-07-16 15:21:11 +08:00
AAEE86 cd8de1aa13 fix: map Developer role to "system" in OpenAI Chat Completions output 2026-07-16 14:29:08 +08:00
elky d6215d9dec ci(tunnel): reduce artifact retention 2026-07-16 13:12:30 +08:00
elky 9a47267545 fix(usage): bound terminal event persistence
Add end-to-end terminal admission, bounded database fallback, and observable overload handling. Preserve first-byte lifecycle state across asynchronous runtime and frontend updates.
2026-07-16 13:12:30 +08:00
MMEXA 7851503fbc fix(finalize): 严格聚合并投影同步 Responses 流 2026-07-16 12:50:00 +08:00
ZheFox c6d373e6aa Merge pull request #677 from zhefox/main
fix(codex): 移除 Responses Lite 请求中的 context_management
2026-07-16 12:28:34 +08:00
ZheFox 71fcb9c168 fix(codex): 服务端压缩使用标准 Responses 合约 2026-07-16 12:02:33 +08:00
ZheFox 3976652942 fix(codex): 移除 Responses Lite 请求中的 context_management 2026-07-16 11:25:11 +08:00
MMEXA 7b56546e21 fix(finalize): 聚合同步流捕获包装 2026-07-16 10:24:32 +08:00
fawney19 85854e4476 Merge pull request #675 from fawney19/fix/pr-669-tail
feat(codex): complete PR #669 protocol follow-up
2026-07-16 08:54:29 +08:00
elky b50242ab9f fix(test): handle absent empty testkit bin directory 2026-07-16 01:29:51 +08:00
MMEXA 20b27a13b2 feat(codex): 按操作语义路由 Responses V2 压缩
(cherry picked from commit 2fc604e047)
2026-07-16 00:34:42 +08:00
MMEXA 598b2fb374 fix(auth): 授权 Responses Compact 伴随端点
(cherry picked from commit e8afa03e45)
2026-07-16 00:32:14 +08:00
MMEXA ff7988430d fix(openai): encode tool errors in Responses output
(cherry picked from commit f127b67e73)
2026-07-16 00:31:23 +08:00
MMEXA 25da99fac2 fix(codex): preserve reset consume request body
(cherry picked from commit fc2dfb82d2)
2026-07-16 00:27:28 +08:00
elky 8616fe6ee2 refactor(workspace): enforce layered crate boundaries 2026-07-15 23:47:19 +08:00
MMEXA 9b8724453b test(pool): 使用正式 Gemini API 格式 2026-07-14 08:39:41 +08:00
MMEXA 01e104d86a fix(pool): 对齐账号批量配置语义 2026-07-14 08:07:28 +08:00
MMEXA 0acd1de29c fix(gateway): 保持密钥更新模块显式所有权 2026-07-14 05:19:04 +08:00
MMEXA a25fab371a feat(pool): add bulk key configuration management 2026-07-14 04:57:05 +08:00
MMEXA 93e2f95c47 fix(frontend): 按端点能力约束会话压缩映射 2026-07-14 02:05:10 +08:00
MMEXA f10d631a9c feat(frontend): 澄清模型映射适用范围 2026-07-14 00:30:39 +08:00
MMEXA cfc4894dab fix(usage): 保留最新进行态生命周期事件 2026-07-14 00:30:24 +08:00
MMEXA 3f86fdd6bc feat(usage): 展示压缩操作与进行态请求语义 2026-07-13 22:03:44 +08:00
MMEXA b09d1f1c33 fix(usage): expose pending reasoning and exact reset expiry 2026-07-13 22:03:44 +08:00
MMEXA 2fc604e047 feat(codex): 按操作语义路由 Responses V2 压缩 2026-07-13 22:03:34 +08:00
MMEXA e8afa03e45 fix(auth): 授权 Responses Compact 伴随端点 2026-07-13 06:07:54 +08:00
MMEXA fc2dfb82d2 fix(codex): preserve reset consume request body 2026-07-12 23:04:33 +08:00
elky a728c090a9 fix(gateway): scope concurrency helper to tests 2026-07-12 22:36:48 +08:00
elky e58621a735 Merge PR #669: align GPT-5.6 and Codex request protocols 2026-07-12 21:50:20 +08:00
MMEXA b1be370b2e fix(gateway): scope concurrency test helper to tests 2026-07-12 21:08:30 +08:00
MMEXA cf0d957ac7 Merge f127b67e73 into 7f61bb43c7 2026-07-12 20:34:39 +08:00
MMEXA f127b67e73 fix(openai): encode tool errors in Responses output 2026-07-12 20:34:31 +08:00
elky 7f61bb43c7 feat(security): harden gateway request and runtime controls 2026-07-12 14:10:54 +08:00
MMEXA 25c49dd804 fix(data): keep terminal usage state monotonic 2026-07-12 05:45:37 +08:00
MMEXA 72222d935c test(gateway): use valid tunnel relay envelopes 2026-07-12 05:45:32 +08:00
MMEXA 063d517306 test(gateway): compare timeout response numerically 2026-07-12 04:43:13 +08:00
MMEXA 02495ce28e fix(admin): preserve inactive endpoint key counts 2026-07-12 04:19:06 +08:00
MMEXA 63936aa110 fix(gateway): route format rules through serving facade 2026-07-12 03:56:53 +08:00
MMEXA 8d4d42a887 fix(auth): resolve group policy before key intersection 2026-07-12 03:35:42 +08:00
MMEXA 2316df5c9a feat(codex): align Search and execution protocol 2026-07-12 03:04:15 +08:00
MMEXA 59d37ae1dd fix(frontend): import structured models.dev pricing 2026-07-11 18:12:55 +08:00
MMEXA 3014fd50c6 fix(billing): preserve effective cache and tier facts 2026-07-11 18:12:55 +08:00
MMEXA 14c4e3a04e fix(codex): enforce provider request identity 2026-07-11 18:09:14 +08:00
MMEXA 8f1070a451 feat(frontend): expose processing tier pricing 2026-07-11 12:27:09 +08:00
MMEXA 0b30cc6b0f feat(openai): unify tier authorization and settlement 2026-07-11 12:27:05 +08:00
MMEXA b2f596b8f0 fix(gateway): route Codex header through serving facade 2026-07-11 09:43:12 +08:00
MMEXA 01a96fed74 fix(codex): simplify summary normalization 2026-07-11 09:20:07 +08:00
MMEXA 46a903aada fix(codex): align current reasoning request semantics 2026-07-11 09:15:08 +08:00
MMEXA dfa121dd5b feat(openai): align GPT-5.6 and Codex request contracts 2026-07-11 07:40:12 +08:00
elky bc1da3bf3f feat(security): harden client IP and admin controls 2026-07-10 15:13:12 +08:00
elky 6e0dc3b59e feat(frontend): refine global model pricing dialog 2026-07-10 15:13:12 +08:00
elky 4bf5d4c044 Fix cache token accounting and tiered pricing 2026-07-10 15:13:12 +08:00
fawney19 736fc76345 Merge pull request #668 from MMEXA/codex/antigravity-empty-output-retry-20260709
修复 Gemini 空输出按候选重试处理
2026-07-10 09:16:56 +08:00
MMEXA b6b2ca38f4 触发 CI 重跑 2026-07-10 00:41:22 +08:00
MMEXA f07eb25cfc 修复 usage 详情 body 引用解包 2026-07-10 00:23:31 +08:00
MMEXA d2ea437c1c 修复 Gemini 空输出按候选重试处理 2026-07-09 23:10:30 +08:00
fawney19 7bc7d0f8d8 Merge pull request #666 from xixiknow/main
Fix provider key response time counter overflow
2026-07-09 18:04:11 +08:00
fawney19 14cf639aba Merge pull request #667 from MMEXA/codex/antigravity-v1internal-query-20260709
修复 Antigravity v1internal 查询参数透传
2026-07-09 17:50:38 +08:00
yangrs 55cdab592c Remove redundant response time conversion 2026-07-09 16:26:09 +08:00
MMEXA ee0ec18283 修复 Antigravity v1internal 查询参数透传 2026-07-09 16:03:29 +08:00
Start f31c9e03e2 Merge branch 'fawney19:main' into main 2026-07-09 15:11:44 +08:00
fawney19 e50db10439 Merge pull request #663 from MMEXA/codex/gemini-interactions-antigravity-20260705
完善 Gemini Interactions 与 Antigravity 全链路兼容
2026-07-09 14:55:51 +08:00
yangrs 192dc6c20d Fix provider key response time overflow 2026-07-09 14:40:48 +08:00
elky 5e1d14f19b Fix timeline duration display from latency 2026-07-09 11:46:45 +08:00
MMEXA b8b89d21b7 fix: 同步提交本地 sync 错误上报 2026-07-08 23:49:18 +08:00
MMEXA 5eddf4f9ee 细化 Antigravity 测试模型项目元数据补全 2026-07-08 22:45:30 +08:00
MMEXA c7186e1720 完善 Antigravity 配额展示与 CI 断言 2026-07-08 22:34:29 +08:00
MMEXA 4866509938 移除 Antigravity 未知重置时间噪音 2026-07-08 22:34:29 +08:00
MMEXA 2122660a5c 对齐原生 Antigravity 控制面与显示模型 2026-07-08 22:34:29 +08:00
MMEXA 5c68ab896a 恢复历史 backfill 兼容 live 账本 2026-07-08 22:34:29 +08:00
MMEXA f9c8ec41f4 完善 Antigravity 与 Gemini 跨格式兼容 2026-07-08 22:34:29 +08:00
MMEXA b1ed6b24b0 触发 CI 复跑 2026-07-08 22:34:29 +08:00
MMEXA c17c78ad4b 修正 Antigravity Gemini 3.5 Flash 档位展示 2026-07-08 22:34:29 +08:00
MMEXA 9ec48ab6b9 优化 Antigravity 配额展示顺序 2026-07-08 22:34:29 +08:00
MMEXA accd250226 修正 Antigravity 配额模型标签 2026-07-08 22:34:29 +08:00
MMEXA 80a6579766 支持 Gemini Interactions 与 Antigravity 配额精细化 2026-07-08 22:34:29 +08:00
fawney19 1ca83ca3fb Merge pull request #664 from MMEXA/codex/wallet-auth-cache-delay-20260706
修复钱包余额变更后的鉴权缓存延迟
2026-07-07 01:59:44 +08:00
fawney19 a931da0764 Merge pull request #662 from MMEXA/codex/reset-credit-20260704
增加 Codex 重置次数功能
2026-07-07 01:58:44 +08:00
fawney19 a61374c595 Merge pull request #661 from MMEXA/codex/frontend-debug-20260704
修复前端调试与基础交互问题
2026-07-07 01:57:41 +08:00
MMEXA c3136126e5 修复钱包余额变更后的鉴权缓存延迟 2026-07-06 06:15:31 +08:00
MMEXA b23d299533 重跑 Codex 重置次数 CI 2026-07-05 01:37:31 +08:00
MMEXA b03aae18c3 修复 Codex 重置次数 CI 检查 2026-07-04 15:47:01 +08:00
MMEXA 99b6fe468f 简化 Codex 重置机会展示标签 2026-07-04 15:16:38 +08:00
MMEXA ef77ec04ca 增加 Codex 重置次数功能 2026-07-04 06:10:53 +08:00
MMEXA 242081433e 修复前端调试与基础交互问题 2026-07-04 05:24:40 +08:00
ZheFox b86d4e1f0c Merge pull request #660 from zhefox/main
refactor(frontend): unify mobile menu background styles
2026-07-03 13:17:59 +08:00
ZheFox a151f37d63 refactor(frontend): unify mobile menu background styles 2026-07-03 13:17:18 +08:00
ZheFox 42f7907740 Merge pull request #659 from zhefox/main
refactor(frontend): improve mobile overflow handling
2026-07-03 12:54:01 +08:00
ZheFox e72e25c59c refactor(frontend): improve mobile overflow handling 2026-07-03 12:53:22 +08:00
ZheFox 1b0440481b Merge pull request #658 from zhefox/main
修复管理端额度显示、节点表格显示与移动端滚动问题
2026-07-03 12:49:15 +08:00
ZheFox 26d85681f0 refactor(frontend): improve mobile overflow and proxy node table 2026-07-03 12:25:53 +08:00
ZheFox 1dcee77055 refactor(frontend): improve dialog and mobile overflow handling 2026-07-03 11:26:40 +08:00
elky 1ac16005f9 Stabilize usage worker autoscale tests 2026-07-02 17:28:25 +08:00
elky 2f1cdb6a0b Record exhausted usage failures synchronously 2026-07-02 16:08:04 +08:00
elky ac93851b2a Stabilize Gateway h2c transport test 2026-07-02 14:02:26 +08:00
elky 400b3125a4 Preserve terminal request candidate state 2026-07-02 01:40:57 +08:00
elky 2e5ff32e1a perf(frontend): 收敛导航预取并去重首屏请求
- 导航预取仅保留 pointerdown 触发,移除 mouseenter/focus,避免鼠标划过误触发
- 后台预取只做组件懒加载,不再预取各页业务数据,减少首屏资源争抢
- 版本状态检查增加 sessionStorage 缓存(正常 20 分钟 / 错误 5 分钟 TTL)
- fetchModules、必读公告拉取增加请求去重,避免并发重复请求
- 更新检查改用可清理的定时器,组件卸载时清理
- UsageRecordsTable 搜索防抖改为自定义实现,卸载时取消挂起 emit 并补充测试
2026-07-01 20:42:52 +08:00
elky a0f7074e59 chore: disable Redis persistence by default, document triage and policy 2026-07-01 14:15:16 +08:00
elky 7c32be46ca Mark sync usage active earlier 2026-07-01 02:21:20 +08:00
Entropy.Xu 6ed2f9bd0a fix: apply actual billing cost to wallet settlement 2026-07-01 01:12:40 +08:00
elky 778b106023 test: stabilize gateway nextest timing 2026-06-30 18:42:57 +08:00
elky f179ee72f9 chore: update gateway pressure observability 2026-06-30 17:01:39 +08:00
elky 974def5fef refactor(frontend): extract provider key identity block 2026-06-30 17:01:39 +08:00
elky e5351b7d9d refactor(frontend): extract provider key actions 2026-06-30 17:01:39 +08:00
elky ed83184d55 refactor(frontend): extract provider quota display components 2026-06-30 17:01:39 +08:00
elky 15b6606c82 refactor(frontend): extract pool key display panels 2026-06-30 17:01:39 +08:00
elky d7411a3104 refactor(frontend): extract pool header and theme toggle 2026-06-30 17:01:39 +08:00
elky 9f138d09e6 refactor(frontend): modularize i18n architecture 2026-06-30 17:01:39 +08:00
ZheFox bf29129a4b Merge pull request #654 from zhefox/main
Cancel upstream streams on client disconnect and void cancelled usage billing
2026-06-29 01:14:11 +08:00
zhefox f6293b6812 fix(usage): void cancelled usage and cancel dropped streams 2026-06-29 00:30:41 +08:00
elky 7e9424008f Add usage queue worker autoscaling 2026-06-26 14:02:57 +08:00
elky 6c5e70ccb1 fix monitoring error totals and counter health 2026-06-26 10:48:45 +08:00
elky 063834e95b Split admin operations dashboard route 2026-06-26 01:48:37 +08:00
elky c76d6b6396 Add admin operations dashboard and usage state fixes 2026-06-26 01:32:45 +08:00
elky 6f00e9fc67 Improve gateway transport and usage runtime 2026-06-25 22:36:27 +08:00
elky d336d1a7fa Improve gateway scheduling and runtime admission 2026-06-24 01:53:45 +08:00
ZheFox cf0af8fa1e Merge pull request #652 from zhefox/main
fix(usage): preserve token counts in body redaction
2026-06-23 14:40:59 +08:00
zhefox fd220b6c42 fix(usage): preserve token counts in body redaction 2026-06-23 14:38:39 +08:00
zhefox 3472bb75e7 ci: combine gateway clippy and nextest jobs 2026-06-23 14:06:43 +08:00
zhefox ba65c96c74 Merge branch 'main' of https://github.com/zhefox/Aether 2026-06-23 13:36:00 +08:00
zhefox c54b214657 ci: shard gateway tests and disable debug info in rust ci 2026-06-23 13:35:56 +08:00
ZheFox 4fcc17114f Merge pull request #651 from zhefox/main
fix(ai-formats): accept Claude context_management in responses conversion
2026-06-23 10:43:26 +08:00
zhefox deb5f55786 fix(ai-formats): clean up cross-format safety rules for Gemini requests 2026-06-23 10:30:13 +08:00
zhefox 1836c2b652 fix(ai-formats): accept Claude context_management in responses conversion 2026-06-23 10:03:55 +08:00
elky 5b7805181b perf: queue request candidate persistence 2026-06-22 02:49:17 +08:00
elky f75894acbb perf: reduce gateway db pressure under load 2026-06-22 00:08:48 +08:00
elky 541cc197c4 fix: preserve in-memory user export fallback 2026-06-22 00:08:48 +08:00
fawney19 363d1aba9a Merge pull request #615 from AAEE86/main
feat: 健康监控仪表盘与关联下钻优化,完善使用记录展示
2026-06-21 12:48:13 +08:00
fawney19 eb2cf662b7 Merge pull request #650 from stabey/pr/claude-system-responses-20260620
fix(ai-formats): preserve Claude in-message system guidance in Responses
2026-06-21 12:47:26 +08:00
elky 900f8a7163 fix(pool): allow zero cooldown settings 2026-06-21 12:20:08 +08:00
elky 61bdd304b7 Handle inactive PAT owner as invalid OAuth token 2026-06-21 11:39:20 +08:00
elky 279735ae7f Auto-size SQL pool defaults 2026-06-21 11:15:40 +08:00
elky cc2830f6ec Merge branch 'review/pr-639' 2026-06-21 10:48:49 +08:00
elky 8dbd730568 fix: respect imported oauth authorization headers 2026-06-21 02:27:06 +08:00
stabey bb6aa03485 fix(ai-formats): strip Claude billing headers from preserved guidance 2026-06-21 00:22:57 +08:00
stabey 6a22488698 fix(ai-formats): preserve Claude in-message system guidance in responses 2026-06-21 00:07:57 +08:00
elky f1c30439ff fix: preserve provider auth metadata 2026-06-20 22:11:42 +08:00
fawney19 1123095bb7 Merge pull request #624 from MMEXA/codex/fix-antigravity-oauth-quota
修复 Antigravity OAuth 导入后配额复检缺 project
2026-06-19 23:11:38 +08:00
MMEXA 938f11981d fix(ai-serving): route Antigravity auth enum through facade 2026-06-19 22:25:52 +08:00
MMEXA 6c4e730e60 修复 Antigravity OAuth 配额复检缺 project 2026-06-19 22:21:22 +08:00
elky 16584067d7 Add route-backed routing profile views 2026-06-18 02:06:34 +08:00
fawney19 6de0fe75a4 Merge pull request #641 from Kayphoon/codex/usage-cleanup-break-condition
fix(usage): align cleanup loop break conditions with candidate row count
2026-06-17 11:04:55 +08:00
fawney19 34f0913ed0 Merge pull request #645 from zhefox/main
修复 OpenAI Chat/Responses/Messages 转换兼容性并透传 Codex cyber_policy 错误
2026-06-17 11:03:29 +08:00
zhefox 5b305c64e1 fix(ai-formats): omit request tool call ids in OpenAI Responses input 2026-06-17 09:15:38 +08:00
zhefox 8ad97761e8 fix(ai-formats): preserve OpenAI Responses tool call item ids 2026-06-17 08:29:25 +08:00
zhefox 0f92ef664d fix(ai-formats): support OpenAI Responses custom tool/raw passthrough 2026-06-17 04:24:56 +08:00
zhefox 16a4fd3687 Merge branch 'main' of https://github.com/zhefox/Aether 2026-06-17 04:07:53 +08:00
zhefox 3a3fcbe46a fix(ai-formats): preserve OpenAI tool call item ids 2026-06-17 04:05:09 +08:00
zhefox 18d8ea2052 fix(ai-formats): preserve OpenAI tool call item ids 2026-06-17 04:03:40 +08:00
zhefox 6ab08f4014 fix(ai-formats): preserve Claude raw blocks, reasoning tokens, and test stack safety 2026-06-17 03:45:23 +08:00
zhefox 628a3a0d8d fix(ai-formats): support cyber policy failover and custom tool/audio passthrough 2026-06-17 02:33:14 +08:00
elky f52628e00b Handle OpenAI Responses keepalive stream events 2026-06-16 22:52:06 +08:00
zhefox f9d97ececb fix(ai-formats): ignore OpenAI Responses metadata events 2026-06-16 22:34:38 +08:00
fawney19 803e555022 Merge pull request #635 from zhefox/main
fix(gateway): 支持 OpenAI 图片编辑端点请求
2026-06-16 22:31:52 +08:00
zhefox b1bd727978 将 JSON 提示注入为 developer 输入 2026-06-16 21:40:15 +08:00
zhefox c2748dc868 忽略 OpenAI Responses keepalive 事件 2026-06-16 20:00:46 +08:00
ZheFox 302620cb94 Merge branch 'fawney19:main' into main 2026-06-16 12:17:28 +08:00
AAEE86 c255f29e98 Merge remote-tracking branch 'upstream/main' 2026-06-16 10:53:47 +08:00
Kayphoon 6d1b818414 fix(usage): align cleanup loop break conditions with candidate row count
The cleanup loop break condition used rows_affected() from the UPDATE
statement, but for rows that only had blob/audit refs (no inline
compressed body data), the UPDATE reported 0 affected rows. This caused
the loop to exit after the first batch, skipping the majority of
candidates.

Change the break condition in all 4 cleanup functions from:
  if cleaned == 0 || cleaned < batch_size
to:
  if rows.len() < batch_size

This ensures the loop continues as long as SELECT returns a full batch,
regardless of how many rows the UPDATE actually modified.

Affected functions:
- cleanup_usage_raw_body_fields
- cleanup_usage_compressed_body_fields
- cleanup_usage_header_fields
- cleanup_usage_stale_body_fields
2026-06-16 04:19:58 +08:00
elky 669636d3e4 Harden PII redaction format conversion 2026-06-14 20:36:57 +08:00
elky 68038c182b Distinguish expired OAuth token status 2026-06-12 19:43:59 +08:00
elky 308cc88ef7 Fix provider deletion cleanup 2026-06-12 16:25:11 +08:00
elky 30b545785f feat: improve failover rules and request timeline 2026-06-11 00:49:29 +08:00
elky 31fade82f6 Preserve OpenAI encrypted reasoning blocks 2026-06-10 20:02:03 +08:00
elky 0246ba93dd fix(transport): preserve safe accept encoding 2026-06-10 19:58:57 +08:00
elky ff7ec8575c fix(ai-formats): ignore null stream errors 2026-06-10 18:44:46 +08:00
elky aa58cb4a05 build: speed up release image linking 2026-06-10 18:37:23 +08:00
elky e9b4efc2d4 fix(ai-serving): preserve explicit request encoding 2026-06-10 18:16:55 +08:00
elky ea76f7bb0b Support OpenAI Responses builtin tool stream items 2026-06-10 14:49:13 +08:00
elky 8edcbdcb29 feat(usage): expose request timing details 2026-06-10 09:16:15 +08:00
ndllz 5249660e07 fix: respect oauth module disabled state 2026-06-09 18:13:07 +08:00
ndllz 84b99a641a fix: speed up usage activity heatmap render 2026-06-09 16:52:58 +08:00
AAEE86 4824e4a487 fix(frontend): 移除账号导入重复处理中提示 2026-06-09 16:40:45 +08:00
zhefox ba723ebe48 fix(usage): always use truncated body placeholder when limit exceeded 2026-06-09 09:59:10 +08:00
zhefox 04ba8cbe9e fix(gateway): support openai image accept negotiation 2026-06-08 20:40:30 +08:00
elky 84f41dae77 feat(format): audit same-format compatibility rewrites 2026-06-08 16:12:37 +08:00
zhefox 82040bfc21 fix(gateway): support OpenAI image edit requests 2026-06-08 13:22:08 +08:00
elky 6155ffefcc fix(format): avoid false cache-control conversion blocks 2026-06-08 00:52:48 +08:00
elky bf4279a590 Merge remote-tracking branch 'origin/main' into dev
# Conflicts:
#	crates/aether-ai-formats/src/formats/openai/chat/stream.rs
#	crates/aether-ai-formats/src/formats/openai/responses/response.rs
#	crates/aether-ai-formats/src/formats/shared/sync_products.rs
2026-06-08 00:17:30 +08:00
elky 77759fac54 feat: 新增提供商批量处理功能 2026-06-07 22:57:02 +08:00
elky 63a2fd4dcf Merge origin/main into dev 2026-06-06 03:11:38 +08:00
elky 7a19891c60 fix: distinguish unaudited conversion fields 2026-06-06 00:35:27 +08:00
AAEE86 85573d7980 Merge remote-tracking branch 'upstream/main' 2026-06-05 08:28:43 +08:00
zhefox ebd59246a8 fix(test): assert responses timestamps and output text in finalize tests 2026-06-04 17:35:10 +08:00
zhefox fd27f55fe5 fix(provider): normalize OpenAI Responses modern fields and stream events 2026-06-04 15:45:37 +08:00
fawney19 69b8b96fb8 Merge pull request #625 from stabey/pr/responses-call-items-cache-control-20260604
fix: 剥离 Codex cache_control 并完善 Responses 工具调用展示
2026-06-04 13:59:56 +08:00
fawney19 19d1d36043 Merge pull request #620 from zhefox/main
fix(provider): 修复 Chat reasoning_effort 值域与 Responses 扩展透传
2026-06-04 13:59:42 +08:00
stabey 9f19ca5754 fix(usage): keep streamed call args in responses completion
The response.completed fallback rebuilt every call item with responsesCallInput(), which returns '{}' for a function_call lacking arguments. Since '{}' is truthy, ensureToolCall overwrote arguments already collected from streamed delta events. Guard the completed branch with responsesCallHasInput (matching the output_item.done branch) so empty/default inputs no longer clobber streamed args, and align its dedupe key with the streaming phase to avoid duplicate tool-call rendering when an item has no id. Drop the now-dead '工具调用' fallbacks since responsesCallName never returns empty.
2026-06-04 13:30:20 +08:00
stabey 2de2a792f6 fix(codex): strip cache_control before responses upstream 2026-06-04 12:01:36 +08:00
stabey ada690624b fix(usage): render responses call items in conversation view 2026-06-04 11:06:36 +08:00
elky 465476985b fix: preserve provider schema drift safely 2026-06-03 22:29:24 +08:00
elky da5624c98e chore: add format field coverage generator 2026-06-03 21:44:33 +08:00
elky b2f68bbaf7 feat: enforce full format field coverage audit 2026-06-03 21:18:49 +08:00
elky 7507af5829 feat: audit strict format conversion contracts 2026-06-03 20:27:15 +08:00
zhefox 5e39801bba fix(provider): clamp reasoning effort and filter chat extensions 2026-06-03 10:52:26 +08:00
elky 5ac153a0bb Fix gateway nextest stack limit 2026-06-03 01:32:25 +08:00
elky c7a5155ce4 Fix sync CLI test stack overflow 2026-06-03 01:12:40 +08:00
elky 869c3d3037 Fix finalize local test stack overflow 2026-06-03 00:48:36 +08:00
elky ef6a11c146 fix(gateway): preserve heartbeat no-path fallback 2026-06-03 00:25:01 +08:00
elky 21432911de Merge remote-tracking branch 'origin/pr/605' 2026-06-03 00:16:22 +08:00
elky eb98340924 Merge remote-tracking branch 'origin/pr/604' 2026-06-02 23:34:59 +08:00
elky 746af0d93e Fix Kiro cache usage reporting 2026-06-02 23:06:29 +08:00
elky 08ac9c5c58 Merge remote-tracking branch 'origin/pr/614' 2026-06-02 22:10:41 +08:00
elky bce3bf2b6e Merge remote-tracking branch 'origin/pr/593' 2026-06-02 21:40:59 +08:00
elky 4ec9ca61cf Merge remote-tracking branch 'origin/pr/613'
# Conflicts:
#	apps/aether-gateway/src/tests/usage/direct.rs
2026-06-02 19:27:50 +08:00
elky 657e6aa672 Merge remote-tracking branch 'origin/pr/619' 2026-06-02 19:23:44 +08:00
AAEE86 7835840ebd feat(dashboard): 增加全站实时指标和自动刷新
- 管理员仪表盘新增全站 RPM/TPM 与在线/启用用户指标
- 合并今日请求/费用、全站 RPM/TPM、在线/启用用户卡片展示
- 在线用户按最近 5 分钟活跃请求去重统计
- 全站 RPM/TPM 按最近 60 秒请求与 Token 统计
- 新增仪表盘自动刷新按钮,开启后每 10 秒静默刷新数据
- 同步前端类型、空态占位和仪表盘测试
2026-06-02 18:16:24 +08:00
zhefox 6cabcd85aa fix(provider): preserve Claude messages defaults in responses conversion 2026-06-02 17:14:26 +08:00
elky 03e436707d Fix gateway usage nextest stack overflow 2026-06-02 16:59:13 +08:00
elky 781bc5ac58 Merge branch 'pr-617' 2026-06-02 10:40:34 +08:00
AAEE86 86f72da3d9 feat(health): 增加历史状态条指标 Tooltip
- 为健康监控时间轴返回 timeline_details 分段指标
- Hover 历史状态柱时展示总请求/成功/失败/可用率/状态
- 展示平均耗时/TTFB/速度和完整时间范围
- 修复历史状态柱 Tooltip 触发区域不可用的问题
- 补齐前端类型、详情抽屉透传和 mock 数据
2026-06-02 10:36:54 +08:00
elky 0a2c674ad8 Ignore tunnel release tags for app build version 2026-06-02 09:43:12 +08:00
zhefox 0daa8c196b fix(provider): preserve reasoning and Claude tool results in responses conversion 2026-06-02 09:04:55 +08:00
zhefox 98dc5925a5 fix(provider): preserve openai responses tool history in chat conversion 2026-06-02 00:35:19 +08:00
AAEE86 d5d3f09846 refactor(health): add dashboard overview and related drill-down
- Replace health monitor tabs with a dashboard layout
- Add related health drill-down for endpoint, model, and provider cards
- Render provider health as cards and hide empty monitors
2026-06-02 00:10:59 +08:00
AAEE86 0e6fc96eb1 test(gateway): run wallet usage settlement test on larger stack
Wrap the wallet settlement usage test with the large-stack async test helper to
avoid stack overflow in the default test thread.
2026-06-01 22:40:22 +08:00
AAEE86 b052f40ffb test(gateway): run base usage body capture test on larger stack
Wrap the request_record_level=base local gateway usage test with the existing
large-stack async test helper to avoid stack overflow in the default test thread.
2026-06-01 22:23:54 +08:00
AAEE86 2aef9d2478 Refine mobile usage record metadata layout 2026-06-01 21:59:45 +08:00
AAEE86 8627a18f2e test(gateway): run local usage report test on large stack
Wrap the local OpenAI chat sync usage-reporting test in the existing
large-stack harness to avoid stack overflows under nextest suite load.
2026-06-01 21:45:44 +08:00
AAEE86 21c478be22 fix(health): hide empty endpoint monitors
- Remove raw API format label from endpoint health cards
- Hide endpoint health cards with no requests
2026-06-01 21:24:20 +08:00
AAEE86 9d8f7d158b Refine mobile usage record details
- Move mobile usage actions into the card header
- Add compact user/provider metadata line on mobile
- Preserve hidden unknown toggle and auto refresh controls
2026-06-01 21:13:10 +08:00
AAEE86 c1649fe837 refactor(health): consolidate monitor components 2026-06-01 20:49:38 +08:00
AAEE86 d3c8317939 fix(health): align model health card layout 2026-06-01 18:54:56 +08:00
AAEE86 7ffe33f867 feat(health): refine health monitor metrics
- add TPS to model and provider health payloads

- exclude user-cancelled 499 requests from health statistics

- update model/provider health cards with average latency, average TTFB, TPS, and availability
2026-06-01 18:34:51 +08:00
elky 6c2a57f237 fix rust ci failures 2026-06-01 02:42:10 +08:00
github-actions[bot] 0f4141ef3f chore(tunnel): update download links for tunnel-v0.3.16 2026-05-31 17:46:42 +00:00
elky 37413c0211 Refactor tunnel stability protocol 2026-06-01 01:36:49 +08:00
Entropy.Xu d1b64b6748 修复:完善 Kiro 模拟缓存共享回收 2026-05-31 22:43:27 +08:00
Entropy.Xu c2bcfab7d4 修复:Kiro 模拟缓存接入共享运行时 2026-05-31 22:13:45 +08:00
elky 392353ffff Merge remote-tracking branch 'entropy-xu/codex/ccswitch-import' 2026-05-31 20:52:26 +08:00
Entropy.Xu 9734be31cf 修复:收敛 Kiro 模拟缓存断点语义 2026-05-31 20:45:16 +08:00
elky 905453d62b revert: remove usage elapsed clock calibration 2026-05-31 20:30:51 +08:00
Entropy.Xu a3b8a99709 修复:补齐 Kiro 模拟缓存 TTL 和消息级断点 2026-05-31 20:22:18 +08:00
Entropy.Xu 2e24e5f358 修复:扩大 Kiro 模拟缓存前缀读取范围 2026-05-31 19:52:18 +08:00
github-actions[bot] 40eb3cf6e1 chore(tunnel): update download links for tunnel-v0.3.15 2026-05-31 11:17:51 +00:00
elky 549463088c chore: bump aether-tunnel version to 0.3.15 2026-05-31 19:09:56 +08:00
elky f8b5651883 Support encoded tunnel node names 2026-05-31 16:33:06 +08:00
stabey de0a880ca6 test(gateway): 修复 usage wallet 测试栈溢出 2026-05-31 03:35:01 +08:00
stabey ba4e194cb5 test(gateway): 修复 usage base 记录测试栈溢出 2026-05-31 03:21:29 +08:00
stabey 1c05a722c1 test(gateway): 修复 usage local 同步测试栈溢出 2026-05-31 03:09:11 +08:00
stabey eda94913cf test(gateway): 修复 usage 同步测试栈溢出
CI 中 gateway_records_pending_usage_before_execution_runtime_sync_result_arrives 仍会在默认测试栈上溢出。

复用 large-stack tokio runtime 包装该测试,避免 gateway 全量测试在无业务失败时被 SIGABRT 中断。
2026-05-31 02:54:11 +08:00
stabey 3dfafbc379 fix(ai): 按 Responses 文本分片去重快照
upstream 已有 8abedecb 处理单个 OpenAI Responses 文本流中 delta 与 done/completed 快照重复输出的问题。

本提交保留该方向,并把去重状态从全局文本扩展为按 output_index/item_id 与 content_index 分片记录,避免多个 message item 或多个 text content part 共用同一段快照状态。
2026-05-31 02:54:11 +08:00
stabey 8d1e54eba6 fix(stream): 中途失败时不合成正常收尾
上游流式读取失败后,已经缓冲的局部转换状态可能是不完整的工具调用。

在 terminal failure 存在时跳过 normalizer 和 rewriter 的 finish 路径,避免把半截 tool_use 补成正常的 Claude message_stop。
2026-05-31 02:54:11 +08:00
stabey 6bfd56b54f fix(usage): 避免上游流式错误误记为成功
当上游流式响应中途失败时,sync error payload 可能同时包含合成错误体和部分上游流 body。

优先使用合成错误体生成 usage 终态,避免只因为上游先返回过 200 和部分 SSE 内容就把失败请求记录为 completed/settled。
2026-05-31 02:54:11 +08:00
elky 49f952692b Fix remaining sync chat stack overflows 2026-05-31 02:12:22 +08:00
elky fde15c9b60 Fix sync chat test stack overflow 2026-05-31 01:13:09 +08:00
elky 06f26cfacf Merge remote-tracking branch 'origin/pr/597' 2026-05-31 00:09:42 +08:00
elky 5360665432 test(gateway): avoid stack overflow in cors proxy test 2026-05-30 22:48:19 +08:00
elky a20ac1d31f fix(pool): align oauth status filter with visible state 2026-05-30 21:37:21 +08:00
elky 1bdd300606 Merge branch 'review-pr-612' 2026-05-30 21:26:18 +08:00
elky c56f0198ff Merge branch 'review-pr-611' 2026-05-30 21:26:12 +08:00
elky 02fc6bd4ef Merge branch 'review-pr-610' 2026-05-30 21:26:07 +08:00
elky c3a8352d76 Merge branch 'review-pr-603' 2026-05-30 21:25:58 +08:00
elky 463576915f Merge branch 'review-pr-602' 2026-05-30 21:25:52 +08:00
elky 1db6b9d307 Merge branch 'review-pr-596' 2026-05-30 21:25:46 +08:00
elky b5a02a118f Merge branch 'review-pr-595' 2026-05-30 21:25:39 +08:00
elky ae96d5d61b fix usage trace active key selection 2026-05-30 19:48:18 +08:00
cym ce1d532e3c fix(pool): align status filters with visible key state 2026-05-30 18:49:26 +08:00
Entropy.Xu f27485ec05 fix(kiro): 忽略图片 base64 token 估算 2026-05-30 02:59:04 +08:00
Entropy.Xu 9616f458de fix(kiro): 模拟缓存读取移动断点前缀 2026-05-30 00:42:14 +08:00
MMEXA 3455faf7da 修复格式转换优先级保持的首轮候选排序
让开启格式转换优先级保持的跨格式候选进入首轮候选页。

普通跨格式候选仍延后到后续页,保持原有兜底语义。
2026-05-29 22:01:11 +08:00
Entropy.Xu 7ed4b84654 feat(ccswitch): 添加一键导入和用量查询 2026-05-29 21:39:45 +08:00
github-actions[bot] 0d76a8e478 chore(tunnel): update download links for tunnel-v0.3.14 2026-05-29 13:32:35 +00:00
elky b9612fef9b chore: bump aether-tunnel version to 0.3.14 2026-05-29 21:21:30 +08:00
fawney19 92ae88f1be fix: avoid postgres migration version collision 2026-05-29 16:18:59 +08:00
ZheFox 91a5e58cec Merge branch 'fawney19:main' into main 2026-05-29 15:27:46 +08:00
fawney19 1658925f52 Disable key circuit breaker for pool providers 2026-05-29 15:16:06 +08:00
Entropy.Xu bb5a4454a5 feat(gateway): 添加标准文本非流式心跳 2026-05-29 14:35:16 +08:00
fawney19 9fb600df1b Fix PR 599 check regressions 2026-05-29 12:30:56 +08:00
ZheFox fff4fe4e20 Merge branch 'fawney19:main' into main 2026-05-29 12:27:38 +08:00
fawney19 3e4dfd2bac Merge branch 'pr-599' 2026-05-29 02:46:07 +08:00
fawney19 8bd82c8c95 Update endpoint base URL placeholders 2026-05-29 02:44:03 +08:00
fawney19 b59c724455 Normalize endpoint API root handling 2026-05-29 02:29:33 +08:00
ZheFox 3ee272fd53 Merge branch 'fawney19:main' into main 2026-05-29 01:48:50 +08:00
AAEE86 ab5d1f266f fix(usage): Optimize the billing layout of the request details page for mobile devices 2026-05-29 00:01:51 +08:00
Entropy.Xu 906742e3c4 fix(billing): 复用待支付套餐订单 2026-05-28 23:09:36 +08:00
AAEE86 0ee45f41e1 feat(mobile): Refine mobile usage record layout 2026-05-28 22:43:20 +08:00
RWDai 6d285410c2 Preserve dashboard daily breakdown rows 2026-05-28 22:02:42 +08:00
Entropy.Xu eaabfb83ed fix(tunnel): bound upstream clients and heartbeat deltas 2026-05-28 20:34:08 +08:00
fawney19 ef2953038e Fix usage records filtering and pool trace display 2026-05-28 20:24:48 +08:00
zhefox cc1a63bf01 fix: repair missing routing profiles snapshot 2026-05-28 18:59:04 +08:00
Novick Yuan 4b2d8cef3c fix(usage): calibrate active elapsed clock efficiently 2026-05-28 18:45:16 +08:00
fawney19 df518ad668 fix contracts usage server time header 2026-05-28 17:54:56 +08:00
fawney19 37b0c00701 Merge remote-tracking branch 'origin/main' 2026-05-28 17:19:04 +08:00
fawney19 88f03aaef2 Keep key circuit breaker out of pool scoring 2026-05-28 17:18:42 +08:00
fawney19 ffd8d273c4 Revert "Merge remote-tracking branch 'origin/pr/592'"
This reverts commit 3504875922, reversing
changes made to 5c3a1aecbe.
2026-05-28 17:10:27 +08:00
fawney19 ef2a96bcc4 Remove default hot pool size cap 2026-05-28 17:01:03 +08:00
Novick Yuan 734717899b Invalidate model routing cache after admin model writes 2026-05-28 16:57:28 +08:00
RWDai d2d28c30d9 Use bearer auth for OpenAI embedding passthrough 2026-05-28 16:53:00 +08:00
fawney19 10532e1a55 Merge pull request #594 from AAEE86/main
feat(usage): support output_config effort badge source
2026-05-28 16:48:02 +08:00
fawney19 47886abd2b Preserve streaming usage timing on refresh 2026-05-28 16:33:10 +08:00
fawney19 d076f64db3 Harden usage server timing header passthrough 2026-05-28 16:28:19 +08:00
AAEE86 60e3ffc402 feat(usage): support output_config effort badge source
- extract reasoning effort from provider request body output_config.effort
- include output_config.effort in usage list fallback SQL
- cover the new request body shape in usage metadata tests
2026-05-28 16:14:37 +08:00
fawney19 b21be24faa Merge remote-tracking branch 'origin/pr/591' 2026-05-28 16:07:08 +08:00
fawney19 3504875922 Merge remote-tracking branch 'origin/pr/592' 2026-05-28 16:07:07 +08:00
Entropy.Xu 0f6d4b9146 feat(embedding): 接入阿里云多模态向量端点 2026-05-28 16:05:36 +08:00
Mas0nShi 6ebd39ed0b Stabilize stream first-byte usage test 2026-05-28 15:33:28 +08:00
fawney19 5c3a1aecbe Merge remote-tracking branch 'origin/pr/591' 2026-05-28 15:23:26 +08:00
Mas0nShi 1a45ec9386 Fix Gemini CLI streaming policy for OpenAI chat 2026-05-28 15:05:05 +08:00
fawney19 97133f657f style: 突出路由策略选中标签样式 2026-05-28 15:03:14 +08:00
Novick Yuan b108dc5ea6 Assert usage server timing over HTTP 2026-05-28 15:01:59 +08:00
Novick Yuan 35cf44b38e Extract active usage elapsed clock 2026-05-28 14:30:15 +08:00
Novick Yuan 6412294262 Use header-only usage server timing 2026-05-28 14:30:15 +08:00
Novick Yuan 01c8592ca6 Align admin user usage timing samples 2026-05-28 14:30:15 +08:00
Novick Yuan 9b5c3ecd23 Use shared clock for active usage timers 2026-05-28 14:30:15 +08:00
Novick Yuan 6de684df59 Track server clock offset for usage data 2026-05-28 14:30:15 +08:00
Novick Yuan 8aca1f8b93 Add server time to usage responses 2026-05-28 14:30:15 +08:00
fawney19 bb2fc2ec00 Optimize health monitor database reads 2026-05-28 14:24:37 +08:00
fawney19 18566b5837 Merge remote-tracking branch 'origin/pr/587' 2026-05-28 13:56:04 +08:00
fawney19 069e1c1e60 Fix merged PR check regressions 2026-05-28 13:53:04 +08:00
fawney19 2c28d9979c Merge commit 'refs/pr/585'
# Conflicts:
#	apps/aether-gateway/src/ai_serving/planner/passthrough/provider/family/request.rs
#	apps/aether-gateway/src/tests/ai_execute/stream_provider_gemini/local_cli.rs
#	apps/aether-gateway/src/tests/ai_execute/sync/gemini/cli.rs
#	apps/aether-gateway/src/tests/control/admin/provider_query.rs
#	crates/aether-provider-transport/src/gemini_cli/mod.rs
#	crates/aether-provider-transport/src/gemini_cli/request.rs
#	crates/aether-provider-transport/src/gemini_cli/url.rs
#	crates/aether-provider-transport/src/lib.rs
2026-05-28 13:19:42 +08:00
fawney19 0efb3d340d Merge commit 'refs/pr/530' 2026-05-28 12:53:45 +08:00
fawney19 535039c29e Merge remote-tracking branch 'origin/pr/584' 2026-05-28 12:16:51 +08:00
fawney19 93d3de1644 feat: improve routing policy diagnostics 2026-05-28 12:11:43 +08:00
AAEE86 4ce056fe45 feat(health): add model and provider health monitoring
- Rename the original health monitor to endpoint health monitor
- Add tab navigation for endpoint, model, and provider health views
- Add model health monitor cards with availability, latency, first-byte latency, and 60-point history
- Add admin-only provider health monitor with collapsible active-provider sections
- Show per-provider model health cards after expanding a provider
- Add backend model health and provider health monitor payload builders
- Add admin endpoint for provider health monitoring
- Add provider-scoped usage breakdown filtering for per-provider model statistics
- Add frontend API types and request helpers for model/provider health data
- Add demo mock data for model and provider health monitoring
- Fix model health timeline time-unit handling so request history segments render correctly

Verification:
- cargo fmt
- npm run type-check
- npm run build
- cargo test -p aether-gateway health_models
- cargo test -p aether-gateway health_providers
- cargo test -p aether-gateway gateway_exposes_frontdoor_manifest_without_proxying_upstream
2026-05-28 12:00:20 +08:00
Mas0nShi 9ad9858ac2 Merge origin/main into fix/gemini-cli-v1internal 2026-05-28 11:58:00 +08:00
MMEXA adca142d1e fix: adapt gemini cli to v1internal endpoint 2026-05-28 00:24:56 +08:00
stabey 739e39e1ca fix: 修复缓存 token usage 转换语义
统一 OpenAI、Gemini、Claude 之间缓存 token 的 usage 语义,避免 Claude 侧重复统计缓存输入 token。

同时补充 stream 合并逻辑、字段注释和覆盖转换链路的测试。
2026-05-27 23:49:07 +08:00
fawney19 14ad6e9b75 Merge remote-tracking branch 'origin/pr/583' 2026-05-27 18:42:57 +08:00
fawney19 d46d225a90 暗色模式下交换流式徽章填充与描边样式 2026-05-27 18:38:59 +08:00
ZheFox c05d227df2 Merge branch 'fawney19:main' into main 2026-05-27 17:06:31 +08:00
fawney19 42e723ff7c Clarify API key concurrency skip reasons 2026-05-27 17:00:09 +08:00
ZheFox b02d62642a Merge branch 'fawney19:main' into main 2026-05-27 16:18:21 +08:00
zhefox 8abedecb16 fix(ai): dedupe OpenAI responses text snapshot deltas 2026-05-27 16:11:02 +08:00
fawney19 d77a572dc7 fix: cast usage provider body before jsonb type checks 2026-05-27 15:48:12 +08:00
fawney19 8606455355 Merge pull request #581 from zhefox/main
fix(gateway): preserve JSON mode chat hints in responses normalization
2026-05-27 15:40:17 +08:00
fawney19 21e52722e6 Prefer provider request body for usage badges 2026-05-27 15:39:39 +08:00
fawney19 6673ab6d4a Add fast model directive service tier 2026-05-27 15:08:23 +08:00
fawney19 d488b1a680 fix auth refresh request body 2026-05-27 15:06:51 +08:00
fawney19 b9ac97ebc3 Simplify request detail cost overview 2026-05-27 14:24:57 +08:00
zhefox e09d3199c1 fix(gateway): update codex prompt cache key test 2026-05-27 13:57:48 +08:00
fawney19 ccfc4cbddc Cache provider catalog lookups 2026-05-27 13:56:39 +08:00
zhefox 41ad422002 fix(gateway): preserve JSON mode chat hints in responses normalization 2026-05-27 13:35:16 +08:00
fawney19 674cc85005 Stabilize stream runtime nextest timing 2026-05-27 11:00:04 +08:00
fawney19 dd2da69361 Merge pull request #580 from AAEE86/main
fix(mobile): improve usage and pool management layouts
2026-05-27 10:23:17 +08:00
fawney19 0ee6e393ce Record stream first byte on upstream event 2026-05-27 10:19:49 +08:00
fawney19 433a4d3c7d test(gateway): run claude pii redaction cases on large stack 2026-05-27 09:21:57 +08:00
AAEE86 049f26c03b fix(mobile): improve usage and pool management layouts
- Fix pool account batch dialog scrolling on mobile
- Rework usage records mobile filters into clearer rows
- Align user filter styling with other select filters
- Improve request detail drawer metric layout on mobile
2026-05-27 09:20:10 +08:00
fawney19 cf8372c8cb fix(usage): record visible stream first byte timing 2026-05-27 02:48:01 +08:00
fawney19 f03550415b style(usage): make fast badge white 2026-05-27 02:11:59 +08:00
fawney19 5a710c4f5e Merge remote-tracking branch 'origin/pr/578' 2026-05-27 02:07:52 +08:00
fawney19 56901f91ce Merge remote-tracking branch 'origin/pr/576' 2026-05-27 02:06:22 +08:00
fawney19 1109c3547c Merge branch 'pr-577' 2026-05-27 01:35:25 +08:00
fawney19 d24ead234d Merge branch 'pr-575'
# Conflicts:
#	apps/aether-gateway/src/ai_serving/planner/passthrough/provider/family/payload.rs
#	apps/aether-gateway/src/ai_serving/planner/passthrough/provider/family/request.rs
#	apps/aether-gateway/src/ai_serving/planner/standard/family/payload.rs
#	apps/aether-gateway/src/ai_serving/planner/standard/family/request.rs
#	apps/aether-gateway/src/ai_serving/planner/standard/openai/chat/decision/request.rs
#	apps/aether-gateway/src/ai_serving/planner/standard/openai/responses/decision/payload.rs
#	apps/aether-gateway/src/ai_serving/planner/standard/openai/responses/decision/request.rs
2026-05-27 01:34:19 +08:00
fawney19 d816ae5c88 Merge remote-tracking branch 'origin/pr/573' 2026-05-27 01:06:57 +08:00
fawney19 8c6e586063 Merge remote-tracking branch 'zhefox/main' 2026-05-27 01:01:34 +08:00
fawney19 c632ec616d Merge remote-tracking branch 'origin/pr/564' 2026-05-27 00:52:15 +08:00
fawney19 bd71a46c25 Merge pull request #561 from Kayphoon/codex/s3-integrated-backup 2026-05-27 00:48:59 +08:00
fawney19 e2b5c3acc8 docs: remove simple query inventory 2026-05-27 00:45:03 +08:00
AAEE86 e27ca671fd feat(usage): show reasoning and fast badges in usage records
- extract provider reasoning effort from request body metadata
- extract priority service tier and expose it as service_tier
- show reasoning level and fast badges after model names
- include badges in active request updates and usage list payloads
- add targeted backend and frontend coverage
2026-05-27 00:36:52 +08:00
fawney19 614c999871 feat(admin): expose s3 backup as module 2026-05-27 00:30:37 +08:00
fawney19 42693c2c52 chore: remove s3 backup docs 2026-05-27 00:07:37 +08:00
fawney19 e21cd72181 fix: preserve pool scan budget for exhausted accounts 2026-05-26 23:49:26 +08:00
MMEXA a9e6a7d644 fix(frontend): recover login redirect navigation 2026-05-26 23:22:30 +08:00
yangrs ba72770cab fix: wire windsurf oauth runtime scheduling 2026-05-26 22:40:46 +08:00
Kayphoon 7530bec7de test(gateway): cover chat pii redaction formats 2026-05-26 22:29:45 +08:00
zhefox 1173a4d9d5 fix(provider): split partial model fetch warnings from errors 2026-05-26 17:45:06 +08:00
zhefox aa409a8a9c fix(provider): refine endpoint default paths for openai and claude roots 2026-05-26 16:50:36 +08:00
AAEE86 949e251b2e fix(provider): 模型测试按 Key 模型权限过滤
测试模型前检查 provider key 的 allowed_models:
- 空权限视为允许所有模型
- 非空权限需匹配请求模型或映射后的实际模型
- 不匹配的 key 标记为跳过,避免发起测试请求

同时补充相关单测和前端跳过原因文案。
2026-05-26 16:46:21 +08:00
fawney19 4933ae9014 Merge pull request #572 from AAEE86/main
fix(admin): add User-Agent for Done-hub provider ops
2026-05-26 16:19:02 +08:00
AAEE86 1793443b09 fix(admin): add User-Agent for Done-hub provider ops
- Done-hub Cookie 请求增加浏览器 User-Agent
- 覆盖认证验证和余额查询的共享请求头
- 补充请求头测试,确认 Cookie 与 User-Agent 同时发送
2026-05-26 16:07:02 +08:00
ZheFox 23a36e37bb Merge branch 'fawney19:main' into main 2026-05-26 15:56:17 +08:00
zhefox c9cf1d458a fix(provider): factor model fetch route test type alias 2026-05-26 15:56:03 +08:00
zhefox d28a389a93 fix(provider): support unversioned API roots in model fetch 2026-05-26 15:39:56 +08:00
fawney19 7e76c9763d Clarify stream first byte timeout message 2026-05-26 15:02:50 +08:00
Kayphoon 9e029462aa test(gateway): stabilize stream timeout regression 2026-05-26 14:41:56 +08:00
Kayphoon 4523a2c67b fix(data): cast MySQL usage aggregates 2026-05-26 14:41:56 +08:00
Kayphoon 84c8bc960e feat(admin): add configurable S3 backups 2026-05-26 14:41:56 +08:00
zhefox c4927162b7 Merge branch 'main' of https://github.com/zhefox/Aether 2026-05-26 13:57:27 +08:00
zhefox 1ebe0aeadf fix(users): allow clearing explicit admin group memberships 2026-05-26 13:57:22 +08:00
ZheFox 992c58f2bd Merge branch 'fawney19:main' into main 2026-05-26 13:08:59 +08:00
zhefox 0bf63cc80e fix(provider): support multi-key selection in model tests 2026-05-26 13:08:35 +08:00
zhefox 5fc6dc8019 Merge branch 'main' of https://github.com/zhefox/Aether 2026-05-26 12:31:03 +08:00
zhefox 96184caa48 fix(codex): strip unsupported OpenAI responses body fields 2026-05-26 12:30:58 +08:00
fawney19 12ff87949d fix(ai): preserve combined Gemini builtin tools 2026-05-26 11:14:24 +08:00
fawney19 b75953bf4c Merge remote-tracking branch 'origin/pr/569' 2026-05-26 11:11:13 +08:00
MMEXA c733139091 fix(ai): normalize Gemini search grounding tools 2026-05-26 05:33:34 +08:00
fawney19 331d37be26 test: run sub2api balance provider ops on larger stack 2026-05-26 02:30:39 +08:00
fawney19 57ccd44b89 fix: fill provider quota execution timeout defaults 2026-05-26 02:11:10 +08:00
fawney19 d60b6e7454 test: run gemini image bridge case on larger stack 2026-05-26 01:52:01 +08:00
fawney19 50e4f27276 Merge remote-tracking branch 'origin/pr/567' 2026-05-26 01:21:03 +08:00
fawney19 a0f22ae659 fix: tighten pr 566 claude and deepseek handling 2026-05-26 00:57:28 +08:00
fawney19 235f32e10e Merge remote-tracking branch 'origin/pr/566' into review/pr-566-fix 2026-05-26 00:43:45 +08:00
fawney19 e7b3acdec3 fix(provider): format oauth import tests 2026-05-25 23:48:04 +08:00
fawney19 f3a367b02d Merge commit 'refs/pull/563/head' of github-fawney19:fawney19/Aether into review/pr-562 2026-05-25 23:44:02 +08:00
fawney19 c03aebba3f Merge branch 'pr-562' into review/pr-562 2026-05-25 23:10:29 +08:00
fawney19 4fb8955bc2 fix(provider): move key model auto-match into dialog 2026-05-25 23:03:52 +08:00
Novick Yuan 5dfccdec3e Fix stream candidate watchdog timeout semantics 2026-05-25 21:43:13 +08:00
fawney19 8b386b0aac Merge remote-tracking branch 'origin/pr/555' 2026-05-25 21:14:45 +08:00
hemo94931 9010f0806a fix(ai): sanitize Claude Read pages passthrough 2026-05-25 21:01:10 +08:00
hemo94931 495795327c fix(ai): sanitize empty Read pages for Claude tools 2026-05-25 21:01:10 +08:00
root 686311eabf test: align OpenAI image stream keepalive expectation 2026-05-25 21:01:10 +08:00
root e68b843875 Add DeepSeek thinking compatibility 2026-05-25 21:00:08 +08:00
root b46028cb85 refactor: clarify OpenAI SSE control policy 2026-05-25 21:00:08 +08:00
root fa172ecb95 fix: avoid synthetic keepalive for OpenAI streams 2026-05-25 21:00:08 +08:00
root 431311979a fix: handle split streaming terminal events 2026-05-25 20:58:17 +08:00
fawney19 d3249485fa Fix stream timeout semantics 2026-05-25 20:09:37 +08:00
zhefox 4c22a819f9 fix(provider): always show batch assign models action 2026-05-25 20:05:13 +08:00
zhefox f4d66021e4 Merge remote-tracking branch 'upstream/main'
# Conflicts:
#	crates/aether-data/src/repository/usage/mysql.rs
2026-05-25 17:29:12 +08:00
fawney19 54c5d5803b fix: cast mysql usage aggregate counters 2026-05-25 15:36:13 +08:00
fawney19 ae138ddb56 feat: update admin config import and runtime handling 2026-05-25 15:23:02 +08:00
zhefox b72abec2fc fix(gateway): spawn oauth account refresh asynchronously 2026-05-25 15:09:41 +08:00
zhefox db4f3fd210 fix(usage): cast mysql usage aggregates to numeric types 2026-05-25 13:56:12 +08:00
zhefox 230ce5df5f Merge branch 'main' of https://github.com/zhefox/Aether 2026-05-25 13:38:38 +08:00
zhefox ec681335e8 fix(usage): treat empty body_state as missing terminal event 2026-05-25 13:38:18 +08:00
Entropy.Xu aaad113190 feat(admin-users): 支持按创建时间排序 2026-05-25 12:21:58 +08:00
calida-tec 63681b4be3 fix(provider): accept common OAuth token JSON aliases 2026-05-25 10:02:47 +08:00
MMEXA b347f1816d Fix native Antigravity stream envelope handling 2026-05-25 09:39:55 +08:00
ZheFox e9efc5c42a Merge branch 'fawney19:main' into main 2026-05-25 09:15:13 +08:00
MMEXA 28c3a5dbe4 Add Antigravity v1internal gateway adapter 2026-05-25 06:58:15 +08:00
fawney19 505d9fd8bc Merge remote-tracking branch 'origin/main' 2026-05-25 01:56:34 +08:00
fawney19 932397d1b3 fix(gateway): defer ChatGPT web image quota decrement 2026-05-25 01:56:25 +08:00
fawney19 1be445423c Merge remote-tracking branch 'origin/pr/558' 2026-05-25 01:22:03 +08:00
fawney19 2df9615fb9 Merge pull request #560 from Kayphoon/codex/remove-install-migration
fix(install): remove pg single-node migration entrypoint
2026-05-25 01:21:30 +08:00
fawney19 fe7fb17ff5 fix(gateway): align admin key health circuit summary 2026-05-25 01:18:07 +08:00
Kayphoon 72d43878ef fix(install): remove pg single-node migration entrypoint 2026-05-25 01:14:37 +08:00
fawney19 cc3ce8b5d7 Merge remote-tracking branch 'origin/pr/557' 2026-05-25 01:08:22 +08:00
fawney19 d4ae6e0e64 fix: read imported usage aggregates in dashboards 2026-05-25 00:51:46 +08:00
MMEXA 480579a0d5 fix(gateway): expire provider key circuit cooldowns 2026-05-25 00:40:38 +08:00
ZheFox ba188aea92 Merge branch 'fawney19:main' into main 2026-05-24 23:40:40 +08:00
fawney19 40b4e52508 Filter format-scoped key fields on import 2026-05-24 22:44:12 +08:00
fawney19 e2d5fc9dfb Preserve usage data in system imports 2026-05-24 21:40:48 +08:00
zhefox c92bdfba16 Merge branch 'main' of https://github.com/zhefox/Aether 2026-05-24 18:51:28 +08:00
ZheFox 83a2609344 Merge branch 'fawney19:main' into main 2026-05-24 18:51:00 +08:00
fawney19 18d9004f22 fix docker app logging permissions 2026-05-24 18:50:39 +08:00
Codex 74c8bfc59f 调整 ChatGPT Web 生图 token 估算口径 2026-05-24 18:50:14 +08:00
Codex 3bf7469d30 修复 ChatGPT Web 生图 usage 估算 2026-05-24 18:50:14 +08:00
Codex 66837b7d7f 修复 ChatGPT Web 生图发起即扣额度 2026-05-24 18:50:14 +08:00
Codex 14b182d09b 修复 ChatGPT Web 生图调用起始预扣额度 2026-05-24 18:50:14 +08:00
Codex 9dba6ec1d9 修复 ChatGPT Web 生图预扣与 Free 限额继承 2026-05-24 18:50:14 +08:00
Codex a52b513533 调整 ChatGPT Web 生图请求预扣额度 2026-05-24 18:50:14 +08:00
Codex 218ca8e6eb 修复 ChatGPT Web 生图额度递减显示 2026-05-24 18:50:14 +08:00
Codex 2b2754b779 修复 ChatGPT Web 生图成功后额度同步 2026-05-24 18:50:13 +08:00
Codex 952d1c840d 修复 ChatGPT Web 额度刷新 403 误判 2026-05-24 18:50:13 +08:00
zhefox 2207b60834 fix(usage): preserve failed status for active request refreshes 2026-05-24 18:48:50 +08:00
ZheFox c09bb28d16 Merge branch 'fawney19:main' into main 2026-05-24 18:07:09 +08:00
github-actions[bot] 13e0759d5a chore(tunnel): update download links for tunnel-v0.3.13 2026-05-24 08:43:41 +00:00
fawney19 80054276eb chore: bump aether-tunnel version to 0.3.13 2026-05-24 16:36:57 +08:00
fawney19 517c9e5108 Merge pull request #552 from RWDai/issue-549-fix
Bound models route data reads
2026-05-24 16:17:41 +08:00
fawney19 576918daa5 Optimize provider scheduler database hotspots 2026-05-24 15:47:56 +08:00
zhefox bbd4338e8e Merge branch 'main' of https://github.com/zhefox/Aether 2026-05-24 15:21:27 +08:00
zhefox e6423a91aa feat(provider): auto-match batch assign models from key 2026-05-24 15:19:37 +08:00
fawney19 78523f122d Limit pool score interest feedback writes 2026-05-24 12:35:55 +08:00
fawney19 e1df06f06c revert compose data layout to legacy paths 2026-05-24 00:37:03 +08:00
fawney19 ecfe04f48a Merge pull request #554 from zhefox/main 2026-05-24 00:15:47 +08:00
ZheFox ff7f27d4f8 Merge branch 'fawney19:main' into main 2026-05-23 23:46:18 +08:00
fawney19 dd07425d21 Merge remote-tracking branch 'origin/pr/550'
# Conflicts:
#	frontend/src/features/usage/components/UsageRecordsTable.vue
2026-05-23 23:40:10 +08:00
ZheFox ba9aa7c1bd Merge branch 'fawney19:main' into main 2026-05-23 23:32:40 +08:00
zhefox 92fd253cae Merge branch 'main' of https://github.com/zhefox/Aether 2026-05-23 22:24:09 +08:00
zhefox 9f6fac418e feat(usage): normalize provider stats across usage and cost views 2026-05-23 22:22:37 +08:00
fawney19 e53a2757eb Merge branch 'pr-548' 2026-05-23 22:19:51 +08:00
fawney19 db32b1d982 fix(usage): ignore truncated stream captures for terminal inference 2026-05-23 22:16:02 +08:00
fawney19 6b1c1e4f50 Merge remote-tracking branch 'origin/pr/547' 2026-05-23 22:03:17 +08:00
fawney19 3eb9614e68 Merge pull request #546 from novcky/fix/pool-scheduler-skip-invalid-oauth-accounts
修复 Provider Pool 热池反复调度已失效 OAuth 账号的问题
2026-05-23 21:38:18 +08:00
fawney19 8087a98c9d Merge remote-tracking branch 'origin/pr/475'
# Conflicts:
#	apps/aether-gateway/src/handlers/admin/provider/oauth/dispatch/refresh/execution.rs
#	apps/aether-gateway/src/handlers/admin/provider/oauth/dispatch/refresh/response.rs
#	apps/aether-gateway/src/handlers/admin/provider/oauth/errors.rs
#	apps/aether-gateway/src/handlers/admin/provider/oauth/quota/shared.rs
#	apps/aether-gateway/src/state/oauth.rs
#	apps/aether-gateway/src/tests/control/admin/oauth.rs
#	crates/aether-admin/src/provider/quota.rs
2026-05-23 21:26:22 +08:00
fawney19 5643b2c901 feat: add compose data layout migration helper 2026-05-23 20:51:02 +08:00
fawney19 18eac2dd7a feat: clarify deployment update strategies 2026-05-23 20:14:26 +08:00
ZheFox 0f2a96554f Merge branch 'fawney19:main' into main 2026-05-23 19:54:24 +08:00
RWDai 0655e868a2 Bound models route data reads 2026-05-23 19:36:20 +08:00
fawney19 4b66cadf15 Merge remote-tracking branch 'origin/pr/544' 2026-05-23 18:41:52 +08:00
Kayphoon 4ee64339ba fix(usage): show retry marker with fallback 2026-05-23 17:54:48 +08:00
Kayphoon babe328565 fix(usage): include embedding formats in filters 2026-05-23 16:19:23 +08:00
fawney19 6447fda852 fix: harden tunnel security and timeout handling 2026-05-23 14:17:04 +08:00
fawney19 74f7348529 Merge remote-tracking branch 'origin/pr/535' 2026-05-23 13:00:18 +08:00
wzw bf456450d7 feat: 代理池均衡分发&批量添加代理节点 2026-05-23 10:40:25 +08:00
zhefox 91cb2bbbcc fix(usage): refine stream terminal capture gating for OpenAI responses 2026-05-23 03:31:15 +08:00
Novick Yuan c0252387b4 修复热池调度已失效 OAuth 账号 2026-05-23 03:17:09 +08:00
zhefox bd1e155332 fix(provider): normalize OpenAI chat tool history for Claude messages 2026-05-23 02:45:04 +08:00
fawney19 ab048b8a03 fix monitoring trace lookup fallback 2026-05-23 01:22:24 +08:00
zhefox e5ce2ac7a4 fix(usage): detect missing terminal events in stream reporting 2026-05-23 00:47:36 +08:00
fawney19 c7641dad0a fix: propagate build version through local deploy 2026-05-23 00:26:07 +08:00
fawney19 b5e942ca9d fix: increase postgres shared memory for dashboard queries 2026-05-23 00:21:34 +08:00
fawney19 74c82d9948 Merge remote-tracking branch 'origin/main' 2026-05-22 23:59:05 +08:00
fawney19 d18b13a91a fix: harden frontdoor and usage ingestion 2026-05-22 23:57:38 +08:00
Mas0nShi 0d80db8a8d Refactor Gemini CLI v1internal planner request builder 2026-05-22 18:57:03 +08:00
Mas0nShi 8cc6888c5b Fix Gemini CLI OpenAI conversion envelope 2026-05-22 18:40:11 +08:00
fawney19 9af1507238 Merge pull request #545 from RWDai/feat/expand-client-types
feat: expand client type recognition
2026-05-22 18:36:53 +08:00
Mas0nShi c67818ee86 Fix Gemini CLI standard conversion envelope 2026-05-22 18:24:40 +08:00
fawney19 6ef6cbade2 Fallback dashboard aggregate reads on schema mismatch 2026-05-22 17:52:36 +08:00
Mas0nShi 8df0e1790d Fix Gemini CLI batch import parse error entry 2026-05-22 17:25:13 +08:00
Mas0nShi 8b7643e150 Merge remote-tracking branch 'origin/main' into fix/gemini-cli-v1internal
# Conflicts:
#	apps/aether-gateway/src/ai_serving/transport.rs
#	apps/aether-gateway/src/handlers/admin/provider/oauth/dispatch/batch/parse.rs
#	apps/aether-gateway/src/handlers/shared/catalog.rs
#	crates/aether-admin/src/provider/quota.rs
#	crates/aether-model-fetch/src/strategy.rs
#	crates/aether-provider-pool/src/lib.rs
#	crates/aether-provider-pool/src/service.rs
#	crates/aether-provider-transport/src/provider_types.rs
#	frontend/src/features/providers/components/ProviderDetailDrawer.vue
#	frontend/src/utils/__tests__/providerKeyQuota.spec.ts
#	frontend/src/utils/providerKeyQuota.ts
#	frontend/src/views/admin/PoolManagement.vue
2026-05-22 17:13:57 +08:00
fawney19 ef04f4b0fb Add automatic B/T compact unit formatting 2026-05-22 17:13:16 +08:00
Mas0nShi ce02f1ae8c Add Gemini CLI v1internal quota support 2026-05-22 16:58:04 +08:00
RWDai c2d0f60784 fix(gateway): accept header sessions for unknown clients 2026-05-22 16:10:27 +08:00
zhiqicloud 781830a202 fix: resolve gateway clippy regressions 2026-05-22 15:54:51 +08:00
RWDai 9dd545353c Preserve omitted tunnel security in CLI mode 2026-05-22 15:54:10 +08:00
zhiqicloud 4c7ebf8b8d style: format admin update settings 2026-05-22 15:32:49 +08:00
zhiqicloud a4a5f70a10 Merge upstream/main into feat/one-click-update 2026-05-22 15:28:27 +08:00
RWDai 9633bce2e1 feat(frontend): centralize client family labels 2026-05-22 15:28:19 +08:00
RWDai ca341703bc feat(usage): infer additional client families 2026-05-22 15:27:39 +08:00
RWDai cb2ff61bbc feat(gateway): expand client session family detection 2026-05-22 15:26:56 +08:00
RWDai 08b27806a7 Respect explicit tunnel security off 2026-05-22 15:20:36 +08:00
zhiqicloud b59c3a9e3b feat: support admin online update and deploy flow 2026-05-22 15:15:17 +08:00
fawney19 ecf6019ccb Merge pull request #543 from zhefox/main
Accept Claude message bodies in OpenAI chat endpoints
2026-05-22 14:53:25 +08:00
zhefox 2a298de971 fix(provider): serialize Claude tool results as JSON strings 2026-05-22 14:41:27 +08:00
RWDai 33633637e5 Fix secure tunnel session handling 2026-05-22 14:35:43 +08:00
fawney19 8ede01ad4e Merge remote-tracking branch 'origin/main' 2026-05-22 14:16:07 +08:00
fawney19 8966fd6aac Protect background workers under DB pool pressure 2026-05-22 14:11:47 +08:00
ZheFox 2b8ff8a743 Merge branch 'fawney19:main' into main 2026-05-22 14:11:00 +08:00
zhefox 97de4ff8a3 fix(gateway): accept Claude Messages bodies on OpenAI chat endpoints 2026-05-22 14:09:51 +08:00
fawney19 f6c3ebf7d3 Merge pull request #542 from zhefox/main
Handle OpenAI chat body responses and SSE passthrough
2026-05-22 12:05:36 +08:00
ZheFox 56abfdf39d Merge branch 'fawney19:main' into main 2026-05-22 11:51:53 +08:00
zhefox 9b95fa4d95 fix(gateway): handle responses-shaped OpenAI chat bodies and SSE passthrough 2026-05-22 11:39:38 +08:00
fawney19 f341c573eb Merge pull request #541 from AAEE86/main
feat(notification): add Bark push support
2026-05-22 11:24:11 +08:00
fawney19 b227669985 Merge pull request #540 from zhefox/main
fix(usage): treat stream terminal failures as failures on HTTP 200
2026-05-22 11:23:53 +08:00
AAEE86 ce44d35eb6 feat(notification): add Bark push support
Add Bark as a notification-service delivery channel, including encrypted Device Key configuration, server URL/template settings, module status integration, and admin UI support.
2026-05-22 11:08:00 +08:00
RWDai b05f2a270a Fix gateway secure tunnel Clippy warning 2026-05-22 10:13:06 +08:00
RWDai 2e701a90c9 Complete secure tunnel encryption support 2026-05-22 09:47:28 +08:00
zhefox 507f2f8250 fix(usage): treat stream terminal failures as failures on HTTP 200 2026-05-22 09:34:22 +08:00
Mas0nShi 9533bd7043 fix: route Gemini CLI generateContent through stream 2026-05-22 09:30:55 +08:00
fawney19 2b32b9a445 Fix system data import export flows 2026-05-22 02:46:49 +08:00
fawney19 3714c211dc Merge pull request #534 from RWDai/fix/issue-528-cache-affinity-health
fix(gateway): preserve cache affinity during health updates
2026-05-22 02:10:45 +08:00
fawney19 504c1ccb37 feat: restructure notification services 2026-05-22 01:45:46 +08:00
fawney19 d5e64d6ad9 Merge branch 'pr-503'
# Conflicts:
#	apps/aether-gateway/src/handlers/admin/provider/summary/value.rs
#	apps/aether-gateway/src/lib.rs
#	apps/aether-gateway/src/maintenance/mod.rs
#	apps/aether-gateway/src/maintenance/runtime/workers.rs
#	frontend/src/api/endpoints/types/provider.ts
2026-05-22 00:19:23 +08:00
RWDai 9859aec16c fix(gateway): evict pooled affinity after sibling key failures 2026-05-21 23:50:34 +08:00
fawney19 0e6d7539ad Merge remote-tracking branch 'origin/pr/538' 2026-05-21 23:12:42 +08:00
fawney19 d6eb41aa78 Merge remote-tracking branch 'origin/pr/536'
# Conflicts:
#	apps/aether-gateway/src/execution_runtime/stream/execution.rs
2026-05-21 23:03:14 +08:00
fawney19 97997685b5 Merge remote-tracking branch 'origin/pr/498' 2026-05-21 22:56:43 +08:00
fawney19 ab0a90de97 fix(runtime-state): govern redis connections 2026-05-21 22:53:37 +08:00
ZheFox eeb7995214 Merge branch 'fawney19:main' into main 2026-05-21 22:27:07 +08:00
zhefox 68d8f86dc6 fix(gateway): stop stream polling on downstream disconnect and preserve Codex cache keys 2026-05-21 22:26:14 +08:00
RWDai 18fe5a4f11 fix(gateway): preserve affinity during adaptive retries 2026-05-21 21:56:26 +08:00
RWDai 26ee1a9958 fix(gateway): preserve affinity during quota telemetry 2026-05-21 21:56:26 +08:00
zhefox 77c2d91eb0 fix(gateway): drain downstream-disconnected streams and stop inferring cancelled usage 2026-05-21 20:30:44 +08:00
fawney19 b8a65cbdec Merge pull request #532 from RWDai/opencode/cosmic-nebula
fix: raise group rate limits by access tier
2026-05-21 19:39:50 +08:00
zhefox 71f9afc526 fix(gateway): move openai responses helper import to ai_serving module 2026-05-21 18:09:57 +08:00
zhefox 8ca4a10f24 fix(gateway): require terminal events for OpenAI responses streams 2026-05-21 17:46:21 +08:00
Mas0nShi 66f21de50e fix: unwrap Gemini CLI model test envelopes 2026-05-21 17:39:10 +08:00
Mas0nShi 3e6ce6cf4a fix: hydrate Gemini CLI project metadata 2026-05-21 17:10:17 +08:00
stabeyandClaude Opus 4.7 cadc45c5b8 fix(data): cleanup uses failed candidate status instead of 504
The stale-pending cleanup task previously hardcoded status_code=504 and a
generic timeout message for every usage row it finalized. When a request
had already been observed as failing — e.g. upstream Connection reset by
peer, watchdog 504, or an authenticated 4xx — the cleanup overwrote that
context with a misleading "服务器超时" outcome and 504 status, hiding the
real cause from the dashboards and customer.

Pull the most recent failed/cancelled candidate per stale request_id and,
if present, finalize the usage row with the candidate's status_code
(defaulting to 502 when none was recorded) and error_message. Requests
that have no terminal candidate (truly stuck pending/streaming) keep the
existing 504 + timeout-message behavior, since they really are timeouts
from the cleanup's perspective. Applied to all three SQL backends with
parameterized UPDATE statements.

The Postgres failed-candidate lookup orders by
COALESCE(finished_at, started_at, created_at) DESC, matching the MySQL
and SQLite ORDER BY clauses so the three backends pick the same
"most recent terminal candidate" under every NULL combination of timing
columns.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
2026-05-21 16:56:56 +08:00
zhefox b7b7b4f718 fix(gateway): report terminal stream failure errors consistently 2026-05-21 16:15:03 +08:00
RWDai 4f49dd5943 Document tunnel security MVP config 2026-05-21 16:09:38 +08:00
RWDai 888414c41b Forward proxy tunnel security aliases 2026-05-21 16:09:00 +08:00
RWDai 8f41bc1558 Generate secure tunnel install sessions 2026-05-21 16:08:37 +08:00
RWDai a543ca9e07 Add tunnel installer security envs 2026-05-21 16:08:14 +08:00
RWDai 40b9db3545 Add tunnel security setup fields 2026-05-21 16:07:56 +08:00
RWDai bd4f6b9206 Add tunnel security config fields 2026-05-21 16:07:27 +08:00
RWDai 776f95b1ab chore: format auth rate limit tests for rustfmt 1.95 2026-05-21 15:53:40 +08:00
zhefox 12abde2aeb fix(usage): handle terminal stream failures and preserve usage updates 2026-05-21 15:52:51 +08:00
RWDai 965a8c79af fix(gateway): preserve cache affinity during health updates 2026-05-21 15:51:56 +08:00
RWDai d33043288d fix(frontend): clarify user group policy help 2026-05-21 15:38:53 +08:00
RWDai a2ad556f2b fix(gateway): raise group rate limits by tier 2026-05-21 15:38:42 +08:00
Mas0nShi e53d5f07e8 feat: support Gemini CLI v1internal quota 2026-05-21 15:38:10 +08:00
stabeyandClaude Opus 4.7 40434005c0 fix(gateway): use total_ms for non-stream upstream watchdog
When the endpoint forces upstream_stream_policy=force_non_stream while
the client streams, the local stream candidate watchdog still preferred
timeouts.first_byte_ms — a non-stream upstream produces no early first
byte, so the watchdog fired before the HTTP request_timeout and aborted
otherwise-healthy attempts at ~300s.

Read upstream_is_stream from report_context and invert the priority:
non-stream upstreams use total_ms first, falling back to first_byte_ms
and then the default; streaming upstreams keep the previous order.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
2026-05-21 15:26:42 +08:00
stabeyandClaude Opus 4.7 3330b2ac4c refactor(report-context): extract UPSTREAM_IS_STREAM_KEY constant
The "upstream_is_stream" JSON key flows from the AI execution report
context producer (aether-ai-serving::report_context) through several
consumers — usage runtime metadata copy/move, gateway watchdog, sync
execution decision, observability handlers, and the per-driver usage
repositories. Each site spelled the key as a bare string literal, so a
producer-side rename would silently degrade every consumer to its
fallback (typically assuming streaming) with no compile-time signal.

Introduce a single pub const UPSTREAM_IS_STREAM_KEY in
aether-ai-formats (the lowest crate every consumer already depends on),
re-export from the crate root, and route producer + all map-style
consumers through it. The change is purely a string-literal → constant
swap; behaviour is identical.

Sites left as literals (intentional):
- `json!({"upstream_is_stream": ...})` macro keys, which must be string
  literals at the macro layer; these are also API-response payload
  field names (an external contract that should not silently track
  internal report-context renames).
- SQL column accessors (`try_get::<...>("upstream_is_stream")`), which
  refer to the database schema column, not the JSON key.
- Test fixtures and assertions, which validate the on-the-wire contract
  and should keep verifying the actual string.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
2026-05-21 15:26:42 +08:00
zhefox be6e49b9c2 Merge branch 'main' of https://github.com/zhefox/Aether 2026-05-21 12:30:48 +08:00
zhefox e59e6c3797 fix(gateway): sanitize Claude thinking and handle missing stream finish 2026-05-21 12:30:42 +08:00
Entropy.Xu 6b04a0a3a6 fix(windsurf): 修复 native 工具流式回程 2026-05-21 02:55:05 +08:00
Entropy.Xu 4112a8b2ea fix(provider): 修复 Windsurf PR CI 失败 2026-05-21 02:21:35 +08:00
fawney19 b84e4a96e2 chore: tune default postgres settings for 2c4g 2026-05-21 01:36:05 +08:00
fawney19 e7f8b259ac Revert "Merge pull request #517 from zhiqicloud/feat/provider-balance-query"
This reverts commit 7e95e769d5, reversing
changes made to 490306c242.
2026-05-21 01:24:17 +08:00
Entropy.Xu 65c361115a fix(provider): 修复 Windsurf PR 冲突残留 2026-05-21 01:02:02 +08:00
Entropy.Xu 129c7c90c0 fix(provider): 修复 Windsurf 原生工具桥接 2026-05-21 01:02:02 +08:00
Entropy.Xu 82637ad882 fix(provider): 修复 Windsurf Connect 请求与端点计数 2026-05-21 01:02:02 +08:00
Entropy.Xu 931c577345 fix(provider): 接入 Windsurf 模型测试链路 2026-05-21 01:02:02 +08:00
Entropy.Xu 9466d92a7a fix(provider): 接入 Windsurf 模型拉取和格式转换 2026-05-21 01:02:02 +08:00
Entropy.Xu 0a0a8b31c7 fix(provider): 隐藏 Windsurf refresh token 刷新入口 2026-05-21 01:02:02 +08:00
Entropy.Xu 208c77a062 fix(provider): 对齐 Windsurf PostAuth 登录链路 2026-05-21 01:02:02 +08:00
Entropy.Xu 02d1343436 feat(provider): 补充 Windsurf 邮箱密码导入表单 2026-05-21 01:02:02 +08:00
Entropy.Xu 0226e14251 feat(provider): 原生接入 Windsurf provider 2026-05-21 01:02:02 +08:00
fawney19 923515ab28 fix: align image generation checks 2026-05-21 00:45:02 +08:00
fawney19 4d0c654822 Merge remote-tracking branch 'origin/pr/524' 2026-05-20 23:17:31 +08:00
ZheFox 779877acd0 Merge branch 'fawney19:main' into main 2026-05-20 22:49:49 +08:00
fawney19 d49b0a8a45 Make extension modules reorderable 2026-05-20 22:45:32 +08:00
ZheFox 5a9f19cbf2 fix(gateway): filter upstream SSE control-only blocks 2026-05-20 22:33:41 +08:00
zhiqicloud 9562295d8b feat: 添加在线更新功能 2026-05-20 22:29:11 +08:00
ZheFox 3c6924238f feat(usage): include cache token details in stream usage payloads 2026-05-20 21:20:10 +08:00
ZheFox 64ad0f694b feat(usage): include cache token details in stream usage payloads 2026-05-20 21:06:51 +08:00
fawney19 754f672ee2 Merge pull request #526 from MMEXA/fix/codex-responses-pending-recovery-20260520
修复 Codex Responses 工具字段和成功请求回收标记
2026-05-20 21:01:54 +08:00
fawney19 e50ceeba5a Merge pull request #525 from final0920/fix/issue-505-priority-order
fix: 修复优先级管理重新打开顺序回退
2026-05-20 21:00:06 +08:00
fawney19 57910f906d Preserve usage provider identity 2026-05-20 20:56:36 +08:00
ZheFox 8838e9289b fix(provider): remove stale image preview block from model test dialog 2026-05-20 20:53:13 +08:00
ZheFox d3355a8a09 fix(gateway): decode stream-encoded provider response JSON 2026-05-20 20:40:48 +08:00
fawney19 c972bbd397 Fix provider pool exhaustion scheduling 2026-05-20 20:16:57 +08:00
MMEXA fed9bdf01a Fix Codex Responses tool schema and pending recovery 2026-05-20 12:02:00 +00:00
ZheFox ab2287202d feat(provider): show image previews in model test dialog 2026-05-20 19:59:20 +08:00
流云 b47282fe4c fix: 修复优先级管理重新打开顺序回退
优先级管理弹窗依赖 grouped-by-format 接口回显格式优先级,但该接口此前读取 summary key 行。summary 查询会清空 global_priority_by_format 和 internal_priority 等路由字段,导致保存后的数据库顺序存在,重新打开页面却回退为前端占位顺序。

改为使用完整 key 查询,并增加 summary 字段被清空时仍能回显真实优先级的回归测试。

Fixes #505

Constraint: grouped-by-format 是优先级管理弹窗的数据源,必须返回真实 per-format priority 字段。
Rejected: 修改前端继续猜测顺序 | 无法区分真实数据库优先级与占位回退。
Confidence: high
Scope-risk: narrow
Tested: cargo fmt --check; git diff --check
Not-tested: cargo test on local Windows blocked by missing NASM for boring-sys2
2026-05-20 19:44:05 +08:00
ZheFox a31e237cc3 Merge upstream main 2026-05-20 19:28:49 +08:00
ZheFox cfa32a2b4d fix(gateway): preserve openai image 200 responses and sync success reporting 2026-05-20 19:17:42 +08:00
ZheFox 2cacf66a37 fix(gateway): route openai image streams with images surface 2026-05-20 18:13:34 +08:00
fawney19 d0981c2fd5 Merge pull request #522 from RWDai/fix/admin-users-server-pagination
Fix admin users server-side pagination
2026-05-20 18:03:11 +08:00
RWDai 3e12c06627 Fix user group options cache busting 2026-05-20 17:51:29 +08:00
RWDai 74a3df3f1e Fix user group options cache invalidation 2026-05-20 17:46:16 +08:00
fawney19 cb894208a1 Merge pull request #521 from Avilianb/fix/provider-query-responses-compact-body
Fix provider model test compact request bodies
2026-05-20 17:42:53 +08:00
fawney19 76752beca6 chore(postgres): 支持通过环境变量配置 PG 性能参数 2026-05-20 17:41:52 +08:00
RWDai e80e7b0cfa Fix admin users pagination review issues 2026-05-20 17:41:06 +08:00
RWDai 77051e6245 Fix admin users pagination follow-ups 2026-05-20 17:32:47 +08:00
ZheFox de7be4f15b fix(gateway): route openai image api requests with mapped models 2026-05-20 17:16:38 +08:00
RWDai 44fb1af287 Fix admin user count clippy lint 2026-05-20 17:06:33 +08:00
fawney19 e7a76b0510 chore(docker): postgres 启用 pg_stat_statements 扩展 2026-05-20 17:00:44 +08:00
fawney19 2881ff097a feat(usage): 管理员用量统计支持筛选刷新与失败保留旧数据
- 用量统计/聚合接口新增 skipCache 选项与 120s 超时,便于强制绕过缓存
- loadStats 增加 force/preserveOnFailure 选项,背景刷新失败时保留旧数据
- 管理员页面在用户/模型/Provider 筛选变化时强制刷新统计,并将筛选条件传入统计接口
- 手动刷新与自动刷新分离,自动刷新不再重载长期热力图相关聚合
2026-05-20 16:51:52 +08:00
RWDai 462c3dde79 Load admin users with server-side pagination 2026-05-20 16:51:46 +08:00
RWDai 1be703b56e Track admin users pagination in frontend data layer 2026-05-20 16:51:46 +08:00
RWDai 5130da9710 Return paginated admin users metadata 2026-05-20 16:51:46 +08:00
RWDai 8440846bae Expose admin user export counts through gateway state 2026-05-20 16:51:46 +08:00
RWDai 831554f11d Add admin user export count queries 2026-05-20 16:51:46 +08:00
Avilianb 97fb588a4a Apply rustfmt to compact provider test 2026-05-20 16:47:30 +08:00
Avilianb cd994d57d2 Fix provider model test compact request bodies 2026-05-20 16:29:46 +08:00
fawney19 70f2882a43 refactor(api-keys): 将独立余额 Key 表单的额度、IP 限制、敏感信息保护移至左侧列 2026-05-20 16:28:58 +08:00
fawney19 fa5d26ce38 Merge remote-tracking branch 'origin/main' 2026-05-20 16:14:09 +08:00
fawney19 f76bbaab52 feat: support api key ip restriction rules 2026-05-20 16:11:49 +08:00
ZheFox cde2062618 chore(gateway): make openai image intent import local 2026-05-20 16:09:24 +08:00
ZheFox 9673fc4c01 fix(codex): trigger image override only on explicit tool_choice 2026-05-20 15:34:31 +08:00
fawney19 19ae7c8902 Merge pull request #519 from RWDai/feat/aether-tunnel-ip-family-options
feat(tunnel): add tunnel IP family controls
2026-05-20 15:30:17 +08:00
RWDai eb63838f6a fix(proxy): keep legacy installer URLs working 2026-05-20 14:43:24 +08:00
RWDai 232006f71d fix(tunnel): restore IP family flag test builds 2026-05-20 14:42:52 +08:00
fawney19 b6bdc08267 Merge branch 'pr-501' 2026-05-20 14:05:00 +08:00
fawney19 7e95e769d5 Merge pull request #517 from zhiqicloud/feat/provider-balance-query
支持 Provider Key 余额查询与自动刷新
2026-05-20 13:59:24 +08:00
RWDai f4c79c80ac fix(tunnel): allow explicit false IP family flags 2026-05-20 13:50:58 +08:00
RWDai edded777e7 docs(tunnel): document IP family controls 2026-05-20 13:43:22 +08:00
RWDai 7284165f39 feat(tunnel): add tunnel IP family controls 2026-05-20 13:42:55 +08:00
RWDai 1604a6d87d fix(gateway): allow clearing API key IP whitelists 2026-05-20 13:41:14 +08:00
ZheFox 7d5ad1e70e chore(gateway): silence dead code warnings in openai image bridge 2026-05-20 13:32:14 +08:00
ZheFox 89860bec97 fix(codex): restrict openai image routing to codex responses 2026-05-20 13:17:24 +08:00
zhiqicloud ebc1774300 修正 new_api 验证测试断言 2026-05-20 13:04:22 +08:00
zhiqicloud 122daf0f87 支持 Provider Key 余额查询与自动刷新 2026-05-20 12:33:31 +08:00
RWDai 149651f831 test(gateway): expect image bridge tools 2026-05-20 11:22:45 +08:00
fawney19 490306c242 Merge commit 'refs/pull/511/head' of github-fawney19:fawney19/Aether 2026-05-20 11:19:58 +08:00
RWDai 316b1e3207 Merge remote-tracking branch 'upstream/main' into feat/500-api-key-ip-whitelist 2026-05-20 11:14:43 +08:00
RWDai 84c4c2f9c2 fix(gateway): preserve image generation tools 2026-05-20 11:02:41 +08:00
fawney19 4d856f3deb Merge remote-tracking branch 'origin/pr/516' 2026-05-20 10:54:32 +08:00
fawney19 61bcbe826a fix: tighten OAuth auto cleanup signals 2026-05-20 10:34:22 +08:00
RWDai bdc848b19e Merge upstream main into feat/500-api-key-ip-whitelist 2026-05-20 10:26:56 +08:00
mayrain 65e3b6f3da fix(codex): trigger image override only on explicit tool_choice
The codex `apply_codex_openai_responses_special_body_edits` override
previously triggered whenever the `tools` array contained an
`image_generation` entry, regardless of whether the caller actually
asked to use it. Codex CLI advertises `image_generation` alongside
~20 other tools under `tool_choice: "auto"`, so every routine codex
conversation was being rewritten into image-generation-only form:

  - `model` forced to `gpt-5.4-mini` (CODEX_OPENAI_IMAGE_INTERNAL_MODEL)
  - `stream` forced to `true`
  - `tools` truncated to a single `image_generation` entry
  - `tool_choice` overwritten to `{"type":"image_generation"}`

The upstream ChatGPT codex backend then rejected the request with
`400 Tool choice 'image_generation' not found in 'tools' parameter`,
which the gateway surfaced as a retryable 503 to clients. The bug
reproduced on every codex CLI session that included the image tool
in its tool catalogue, even though the user never requested image
generation.

Narrow the trigger to the caller's actual selection. The new helper
`codex_openai_responses_tool_choice_references_image_generation`
matches only the explicit string `"image_generation"` or the object
form `{"type":"image_generation"}`. The pre-existing
`is_openai_image_request(provider_api_format)` branch still handles
genuine `openai:image` traffic, so true image-generation flows are
unaffected.

Tests:
  - lock the regression: `tool_choice: "auto"` with image_generation
    in tools must not trigger the override (model/tools preserved)
  - lock variants: string `"image_generation"` and object form both
    still trigger; other tool_choice values and an absent
    `tool_choice` do not
2026-05-20 09:52:39 +08:00
zhiqicloud 97b05e744f 支持官方直连支付、退款配置与套餐联动 2026-05-20 08:16:52 +08:00
fawney19 fbda210b84 Merge pull request #511 2026-05-20 01:22:55 +08:00
fawney19 ed75ae6d56 Merge pull request #509 from beilo/feat/key-ranking-usage
Add paginated API key usage leaderboard
2026-05-20 01:06:58 +08:00
fawney19 d1ad1815f7 Merge pull request #513 from mayrainnn/fix/request-body-content-encoding
feat: normalize compressed request bodies
2026-05-20 01:05:59 +08:00
fawney19 b1a3a26815 Merge pull request #512 from Entropy-Xu/codex/fix-wallet-overdraft-settlement
[codex] 修复钱包余额不足后重复消费
2026-05-20 01:05:26 +08:00
fawney19 94760dbc14 refactor(tunnel): rename aether-proxy to aether-tunnel 2026-05-20 01:02:01 +08:00
zhiqicloud 3a318a86b2 支持官方直连支付、退款配置与套餐联动 2026-05-20 00:21:56 +08:00
fawney19 f4d0d5904a Remove image_generation tools from OpenAI image bridges 2026-05-20 00:20:56 +08:00
fawney19 25c7bb935e chore(frontend): simplify concurrent limit hint text 2026-05-19 23:52:59 +08:00
fawney19 f5deed8709 refactor(proxy): improve tunnel throughput and observability 2026-05-19 23:49:36 +08:00
beilo fe3a848eb5 feat(admin): add paginated API key usage leaderboard 2026-05-19 23:17:10 +08:00
mayrain 8f4f4d2d82 refactor(gateway): route decoded body access through ai_serving 2026-05-19 22:58:03 +08:00
mayrain 66a54cc39e feat(gateway): normalize compressed request bodies 2026-05-19 22:57:45 +08:00
Entropy.Xu 7ace958710 fix(wallet): 修复余额不足后重复消费
- 有限钱包结算允许扣成负数,先扣充值余额再扣赠送余额,缺口回写到充值余额
- 日额度钱包补扣路径在存在钱包时不再把余额不足标成 insufficient_quota
- 补充内存和 SQLite 结算回归覆盖,三种数据库实现保持一致
验证:
- cargo fmt --all --check
- cargo clippy -p aether-data --all-targets -- -D warnings
- cargo clippy -p aether-gateway --all-targets -- -D warnings
- cargo clippy --workspace --exclude aether-gateway --exclude aether-data --all-targets -- -D warnings
- cargo nextest run -p aether-gateway
- cargo nextest run -p aether-data
- cargo nextest run --workspace --exclude aether-gateway --exclude aether-data
- cargo test -p aether-data sqlite --lib
- Postgres/MySQL data_db_smoke commands from rust-ci.yml
2026-05-19 19:29:22 +08:00
fawney19 57655bdb25 Hide capability tags from UI 2026-05-19 19:10:30 +08:00
fawney19 124077a0a1 Merge branch 'pr-506' 2026-05-19 18:36:36 +08:00
fawney19 1b570daf72 Revert "Merge PR #504"
This reverts commit d216a9e219, reversing
changes made to 21e82abd54.
2026-05-19 17:23:57 +08:00
fawney19 8bcd5b8189 Revert "Merge remote-tracking branch 'origin/pr-473'"
This reverts commit f2cdb74ed8, reversing
changes made to 3f0fd15395.
2026-05-19 16:31:58 +08:00
fawney19 63202a63ef Merge branch 'codex/pool-hot-trace-fix'
# Conflicts:
#	apps/aether-gateway/src/ai_serving/planner/candidate_materialization.rs
2026-05-19 14:47:37 +08:00
fawney19 7e9ca88e00 fix: restore provider key circuit breaker backoff 2026-05-19 13:57:36 +08:00
beilo 75aa3dc0cc fix: refine oauth auto-removal behavior 2026-05-19 13:41:18 +08:00
fawney19 6a1da6a5ff fix: speed up admin pool loading 2026-05-19 12:05:36 +08:00
mayrain d88f092dd1 feat(kiro): add simulated cache provider toggle 2026-05-19 10:47:17 +08:00
mayrain b4d17a392a feat(kiro): simulate prompt cache usage accounting 2026-05-19 10:47:17 +08:00
fawney19 c6a408d4e8 Fix local gateway Docker native deps 2026-05-19 10:43:04 +08:00
RWDai d19bf71343 Refresh migration cutoff expectations 2026-05-19 10:25:06 +08:00
RWDai 02f09c2056 Regenerate API key schema baselines 2026-05-19 10:24:47 +08:00
RWDai 6a104d5736 Add allowed IPs to data schema sources 2026-05-19 10:24:27 +08:00
RWDai 95af482ffe Update auth snapshot observability fixture 2026-05-19 10:23:54 +08:00
RWDai b05264ff74 Stabilize wallet today usage test timing 2026-05-19 10:23:32 +08:00
RWDai 2aab1ea97b Clean up gateway API key whitelist handlers 2026-05-19 10:23:15 +08:00
RWDai f1687017e6 Fix provider endpoint format normalization path 2026-05-19 10:22:50 +08:00
fawney19 052de6b96e test: stabilize merged PR checks 2026-05-19 10:18:25 +08:00
fawney19 d2b42e91d2 test: align cancelled sync billing status 2026-05-19 08:25:39 +08:00
fawney19 d8a9d7eb5e chore: format merged PR changes 2026-05-19 08:13:56 +08:00
fawney19 d216a9e219 Merge PR #504 2026-05-19 08:10:57 +08:00
fawney19 21e82abd54 Merge PR #502 2026-05-19 08:10:48 +08:00
fawney19 18ed9a57c2 Merge PR #499 2026-05-19 08:10:37 +08:00
fawney19 702dc3ceb4 Merge PR #489 (ours: keep current main) 2026-05-19 03:38:26 +08:00
fawney19 3180ca2bf9 Merge PR #488 (ours: keep current main) 2026-05-19 03:38:22 +08:00
fawney19 69af74b1e0 Merge PR #488 and #489 2026-05-19 03:16:35 +08:00
MMEXA ef9c0ebbd4 Merge latest origin/main into codex/gemini-embedding-batch
# Conflicts:
#	apps/aether-gateway/src/ai_serving/planner/standard/openai/mod.rs
#	apps/aether-gateway/src/execution_runtime/fallback.rs
2026-05-18 19:11:42 +00:00
MMEXA ca4d0dc819 Merge origin/main into codex/gemini-embedding-batch
# Conflicts:
#	apps/aether-gateway/src/ai_serving/api.rs
#	apps/aether-gateway/src/ai_serving/planner/passthrough/provider/family/request.rs
#	apps/aether-gateway/src/ai_serving/planner/standard/family/request.rs
#	apps/aether-gateway/src/ai_serving/transport.rs
#	apps/aether-gateway/src/handlers/admin/provider/query/models/model_test/summary.rs
#	apps/aether-gateway/src/handlers/admin/provider/query/models/model_test/tests.rs
#	crates/aether-data/src/repository/candidate_selection/postgres.rs
#	crates/aether-model-fetch/src/strategy.rs
2026-05-18 19:02:19 +00:00
fawney19 cd1aa92931 Merge branch 'merge-pr-483' 2026-05-19 02:28:43 +08:00
fawney19 9fc9c334c9 Merge remote-tracking branch 'origin/pr/487'
# Conflicts:
#	crates/aether-data/src/lifecycle/bootstrap/postgres.rs
#	crates/aether-data/src/lifecycle/migrate/tests.rs
#	crates/aether-data/src/repository/oauth_providers/postgres.rs
#	crates/aether-data/src/repository/oauth_providers/sqlite.rs
#	frontend/src/views/admin/OAuthSettings.vue
2026-05-19 02:27:39 +08:00
fawney19 c563c192b7 Merge remote-tracking branch 'origin/pr-483' into merge-pr-483
# Conflicts:
#	apps/aether-gateway/src/ai_serving/planner/standard/openai/chat/decision/payload.rs
#	apps/aether-gateway/src/ai_serving/planner/standard/openai/chat/decision/request.rs
#	apps/aether-gateway/src/ai_serving/planner/standard/openai/responses/decision/payload.rs
#	apps/aether-gateway/src/ai_serving/planner/standard/openai/responses/decision/request.rs
#	apps/aether-gateway/src/execution_runtime/chatgpt_web_image.rs
2026-05-19 02:23:14 +08:00
fawney19 afedd90c80 Merge commit 'refs/pull/481/head' of github-fawney19:fawney19/Aether
# Conflicts:
#	apps/aether-gateway/src/ai_serving/api.rs
#	apps/aether-gateway/src/ai_serving/planner/standard/family/request.rs
#	apps/aether-gateway/src/ai_serving/planner/standard/openai/chat/decision/request.rs
#	apps/aether-gateway/src/ai_serving/planner/standard/openai/responses/decision/request.rs
2026-05-19 01:46:41 +08:00
MMEXA be2e8e594c fix(frontend): align Vertex Gemini endpoint controls 2026-05-18 17:36:04 +00:00
fawney19 d392681c58 Merge branch 'pr-478'
# Conflicts:
#	apps/aether-gateway/src/data/state/mod.rs
#	apps/aether-gateway/src/handlers/admin/mod.rs
#	apps/aether-gateway/src/handlers/admin/routes.rs
#	crates/aether-data/src/lifecycle/bootstrap/postgres.rs
#	crates/aether-data/src/lifecycle/migrate/tests.rs
#	crates/aether-data/src/repository/announcements/postgres.rs
#	frontend/src/features/auth/components/RegisterDialog.vue
2026-05-19 01:27:42 +08:00
MMEXA 5ed8325592 fix(gateway): use Vertex model garden catalog endpoint 2026-05-18 17:06:42 +00:00
fawney19 ed1d9fdb57 Fix postgres bigint counter decoding 2026-05-19 00:58:09 +08:00
ZheFox c58ce63fc3 Merge branch 'fawney19:main' into main 2026-05-19 00:50:52 +08:00
fawney19 5eed329916 Rootfix usage counter outbox 2026-05-19 00:44:37 +08:00
fawney19 19c8688eb1 Merge pull request #477 from RWDai/feat/356-usage-record-columns
Add configurable usage record columns
2026-05-19 00:35:30 +08:00
MMEXA 4317ff78b1 Expose local scheduling failures in usage UI 2026-05-18 16:28:37 +00:00
yangrsandClaude Opus 4.7 6c16f399d4 feat: 重要通知模块、Server 酱独立配置与额度提醒
- 新增重要通知统一模块(邮件 + Server 酱)作为后台任务通知出口
- 拆出独立的 Server 酱 配置页(SendKey + Markdown 模板,支持 {title}/{body} 变量替换),通过仪表盘内置工具入口进入
- 新增提供商额度提醒后台 worker:余额低于阈值时通过重要通知推送,提供商配置页加入额度提醒开关与阈值
- 重要通知页加入配置可用性守卫:未配置任一通道时禁用总开关,未配置邮件/SendKey 时禁用对应通道开关
- 测试通知端点支持 channel 过滤(all/email/server_chan),并绕过总开关与通道开关,便于配置阶段先验证通道
- 修复:测试通知路由未在 buffered-body 白名单导致 channel 参数丢失、测试时邮件分支被误触发
- 修复:sub2api 验证响应中 username 为 null 时正确回退到 email,避免误报"验证响应缺少: 用户信息"

Co-Authored-By: Claude Opus 4.7 <[email protected]>
2026-05-18 23:51:14 +08:00
MMEXA b480f3aaff fix(gateway): normalize Gemini Vertex embedding transport 2026-05-18 15:39:29 +00:00
MMEXA 84f312fa4c fix(gateway): close dropped sync attempts 2026-05-18 14:15:22 +00:00
ZheFox 61d5fdb0ec fix(frontend): preserve fixed provider model test key inheritance 2026-05-18 22:06:27 +08:00
ZheFox 995ab302be Merge remote-tracking branch 'upstream/main'
# Conflicts:
#	apps/aether-gateway/src/handlers/admin/provider/endpoints_admin/payloads.rs
#	apps/aether-gateway/src/handlers/admin/provider/endpoints_admin/reads.rs
#	apps/aether-gateway/src/handlers/admin/provider/endpoints_admin/update.rs
#	apps/aether-gateway/src/tests/control/admin/endpoints/routes.rs
#	frontend/src/features/models/components/GlobalModelFormDialog.vue
#	frontend/src/features/providers/components/ProviderModelFormDialog.vue
#	frontend/src/features/providers/components/provider-tabs/__tests__/model-test-request.spec.ts
#	frontend/src/features/providers/components/provider-tabs/model-test-request.ts
2026-05-18 22:02:31 +08:00
RWDai c6ee558180 Add admin user key IP whitelist UI 2026-05-18 20:48:28 +08:00
RWDai 460cd63d3a Add user API key IP whitelist UI 2026-05-18 20:48:21 +08:00
RWDai 2a3593d9c5 Update planner auth snapshot fixtures 2026-05-18 20:48:14 +08:00
RWDai bcca8d295a Update auth context test fixtures 2026-05-18 20:48:05 +08:00
RWDai d8f68c1d9a Preserve allowed IP defaults in admin key imports 2026-05-18 20:47:57 +08:00
RWDai ab53326865 Support allowed IPs in admin user key endpoints 2026-05-18 20:47:46 +08:00
RWDai 96b857f642 Support allowed IPs in user API key endpoints 2026-05-18 20:47:37 +08:00
RWDai fc12cc8a36 Enforce API key IP restrictions in proxy auth 2026-05-18 20:47:28 +08:00
RWDai 276d19b63c Persist allowed IPs in auth repositories 2026-05-18 20:47:17 +08:00
RWDai 437024cdb1 Add API key allowed IP data types 2026-05-18 20:47:03 +08:00
RWDai 64b41434ce Add API key allowed IP migrations 2026-05-18 20:46:53 +08:00
RWDai 24129d7f12 Merge upstream/main into feat/356-usage-record-columns 2026-05-18 20:33:05 +08:00
fawney19 d51b44d642 Merge remote-tracking branch 'origin/pr-485'
# Conflicts:
#	crates/aether-data/src/repository/provider_catalog/sqlite.rs
2026-05-18 19:44:18 +08:00
RWDai e8c55e8f1f Label OpenAI JS SDK in cache monitoring 2026-05-18 19:30:39 +08:00
RWDai 9179516b19 Label OpenAI JS SDK in usage records 2026-05-18 19:30:19 +08:00
RWDai 37abfe66f0 Label OpenAI JS SDK in user usage 2026-05-18 19:29:59 +08:00
RWDai ae9d4038b1 Prefer typed admin usage client family 2026-05-18 19:29:36 +08:00
RWDai b6f558d10b Project usage client family in Postgres reads 2026-05-18 19:29:10 +08:00
RWDai 6d994917a0 Add usage client family read model field 2026-05-18 19:28:36 +08:00
fawney19 b99f43783c Merge remote-tracking branch 'origin/pr-482'
# Conflicts:
#	.github/workflows/release.yml
2026-05-18 18:55:11 +08:00
fawney19 7f76eff827 Merge remote-tracking branch 'origin/pr-484' 2026-05-18 18:01:57 +08:00
fawney19 be939f7e63 Merge remote-tracking branch 'origin/pr-493' 2026-05-18 18:01:43 +08:00
fawney19 a8a87d2b41 Merge remote-tracking branch 'origin/pr-494' 2026-05-18 18:00:47 +08:00
RWDai 0a26accca4 Merge remote-tracking branch 'upstream/main' into feat/356-usage-record-columns 2026-05-18 17:49:35 +08:00
fawney19 40e925680c Merge remote-tracking branch 'origin/pr-480' 2026-05-18 16:46:35 +08:00
fawney19 f2cdb74ed8 Merge remote-tracking branch 'origin/pr-473'
# Conflicts:
#	frontend/src/features/providers/components/ProviderModelFormDialog.vue
2026-05-18 16:46:22 +08:00
Codex 7ac8159728 fix: use Vertex Model Garden models endpoint 2026-05-18 08:17:09 +00:00
fawney19 3f0fd15395 Merge remote-tracking branch 'origin/pr-491' 2026-05-18 16:14:07 +08:00
fawney19 90dbc279fd Merge remote-tracking branch 'origin/pr-476' 2026-05-18 16:13:45 +08:00
fawney19 3723f165bc Merge remote-tracking branch 'origin/pr-496' 2026-05-18 16:13:03 +08:00
fawney19 3bffecddf2 fix: route access log sanitizer through gateway api 2026-05-18 15:47:02 +08:00
fawney19 a818af7833 Merge remote-tracking branch 'origin/pr-492' 2026-05-18 14:52:29 +08:00
fawney19 187b3a08fb Merge remote-tracking branch 'origin/pr-495' 2026-05-18 14:51:50 +08:00
fawney19 2405ccc0f2 Merge remote-tracking branch 'origin/pr-490' 2026-05-18 14:51:14 +08:00
fawney19 0fcfcae9c8 Merge remote-tracking branch 'origin/pr-486' 2026-05-18 14:50:48 +08:00
ZheFox 2de0cbbafc Merge branch 'fawney19:main' into main 2026-05-18 13:13:37 +08:00
ZheFox a4012ad353 Merge upstream/main 2026-05-18 13:11:11 +08:00
fawney19 e4315fbbf0 fix: refine frontend admin and auth UI 2026-05-18 12:41:57 +08:00
ZheFox a10c02ef63 feat(billing): add image output range pricing support 2026-05-18 11:52:01 +08:00
MMEXA 66f154a251 fix(gateway): avoid low OpenAI-compatible test token cap 2026-05-18 03:06:46 +00:00
fawney19 92813e6122 feat: add routing profile scheduling policies 2026-05-18 11:03:49 +08:00
MMEXA f50f26e599 fix(gateway): cover Google OpenAI-compatible roots 2026-05-18 02:15:23 +00:00
ZheFox a3094fda53 fix(gateway): ignore OpenAI tools for image intent routing 2026-05-18 09:54:10 +08:00
MMEXA b004a02e4a fix(gateway): harden Gemini endpoint routing 2026-05-18 00:53:34 +00:00
MMEXA 9586f5158e Fix fixed-provider endpoint key counts 2026-05-17 20:55:36 +00:00
ZheFox f5ace4fd6d fix(frontend): stabilize image generation overrides in model forms 2026-05-18 03:39:55 +08:00
ZheFox a05ae94cea fix(frontend): update checkbox bindings to checked events 2026-05-18 03:27:31 +08:00
ZheFox dff17b6cb1 feat(billing): support image output pricing in model forms and details 2026-05-18 03:19:58 +08:00
ZheFox 0b2a8fafce feat(billing): add image output pricing and usage tracking 2026-05-18 02:49:56 +08:00
ZheFox 691ccaaa04 fix(usage): track OpenAI image SSE completion and usage estimates 2026-05-18 01:32:45 +08:00
RWDai 26900c8c9e Show only client family in usage client column 2026-05-18 00:46:41 +08:00
MMEXA 226ce45d0d fix: pass explicit local build version 2026-05-17 16:38:22 +00:00
Kayphoon ae472d6744 fix(data): require provider id for provider usage aggregation 2026-05-18 00:36:29 +08:00
MMEXA 89d08d9953 fix: align provider model fetch state 2026-05-17 16:17:47 +00:00
ZheFox c024c782e4 feat(billing): add image quality pricing and usage tracking 2026-05-18 00:11:30 +08:00
MMEXA 84fc1e35e2 fix(frontend): support login autofill and reliable redirect 2026-05-17 16:03:57 +00:00
MMEXA 995be3781a fix(frontend): align usage APIs with Rust routes 2026-05-17 15:37:52 +00:00
MMEXA ed570155a1 fix(gateway): redact credential query values in access logs 2026-05-17 15:26:56 +00:00
ZheFox 680b617b00 feat(gateway): route OpenAI image streams through chat bridge 2026-05-17 23:09:50 +08:00
RWDai 0a62e4bc77 Remove usage request path column 2026-05-17 23:05:02 +08:00
RWDai 96d40dd21d Infer usage client family from User-Agent 2026-05-17 23:04:31 +08:00
MMEXA ff83b54c3b fix(gateway): reject empty Gemini success responses 2026-05-17 14:55:33 +00:00
MMEXA 81ff375bfd fix(gateway): route Gemini embedding batches correctly 2026-05-17 14:24:32 +00:00
dalamudx b0fc6e68ef fix: add icon_url to mysql generated baseline 2026-05-17 22:00:28 +08:00
dalamudx e1a73be2e1 fix: update schema baselines, logical schema, and snapshot cutoff for icon_url 2026-05-17 21:52:36 +08:00
dalamudx cf2c74e8e8 fix: add new migration version to test whitelist 2026-05-17 21:42:56 +08:00
dalamudx 2cb01f7a69 fix: add missing icon_url arg in test helper 2026-05-17 21:40:54 +08:00
dalamudx 48deff15c4 fix(oauth): fix login failures and add provider icon_url config
- Fix FIND_OAUTH_LINKED_USER_SQL missing allowed_providers_mode columns
- Fix TOUCH_OAUTH_LINK_SQL json/jsonb type mismatch in COALESCE
- Add icon_url field to OAuth provider config (DB, API, frontend)
- Fix admin OAuth test: accept 404 as reachable, use system proxy
2026-05-17 21:33:02 +08:00
ZheFox d6c8c14de7 feat(gateway): route OpenAI image intents through image bridge 2026-05-17 21:19:17 +08:00
RWDai b2266b588e Preserve usage origin metadata in Postgres lists 2026-05-17 21:09:59 +08:00
RWDai fcaafb3939 chore(ci): retrigger flaky sqlite smoke 2026-05-17 20:38:55 +08:00
fawney19 7d569127ae Fix active probe pool fallback tracing 2026-05-17 20:34:06 +08:00
RWDai 981020a5ab fix(ci): apply rustfmt to usage metadata handlers 2026-05-17 20:28:08 +08:00
ZheFox d9c8119bda fix(routing): match provider model names in admin routing counts 2026-05-17 18:59:41 +08:00
ZheFox 485d166912 fix(provider): count inherited endpoint formats for model tests 2026-05-17 18:04:16 +08:00
Kayphoon 5060532c51 fix: package release sqlite compose template 2026-05-17 16:33:35 +08:00
Kayphoon f29cca72ba refactor(data): limit query abstraction to postgres and sqlite 2026-05-17 15:40:44 +08:00
Kayphoon 77640d51a6 refactor(data): add select query abstraction 2026-05-17 15:04:35 +08:00
HsungKayphoon f0a6fffa87 refactor(data): introduce simple query helper 2026-05-17 14:07:23 +08:00
Entropy.Xu 1b24e1c22a fix(wallet): 修复额度耗尽后仍可消费 2026-05-17 11:49:08 +08:00
ZheFox ab0d766f47 fix(usage): stop inferring cache reads from prompt_cache_key 2026-05-17 03:04:33 +08:00
ZheFox 41ffb18604 fix(billing): avoid double counting cache read in OpenAI cache hit context 2026-05-17 02:40:01 +08:00
ZheFox b903f8ff7d fix(usage): estimate cache read tokens for cancelled requests 2026-05-17 02:13:13 +08:00
ZheFox 0483d001b4 fix(usage): bill cancelled terminal usage and preserve total token estimates 2026-05-17 01:11:42 +08:00
HsungKayphoon d290a1fdb9 fix: satisfy rust 1.95 clippy 2026-05-17 00:00:46 +08:00
ZheFox 2803e9317d fix(usage): bill cancelled terminal usage and preserve terminal telemetry 2026-05-16 23:43:40 +08:00
HsungKayphoon c5e26a1ed6 fix: align sqlite repositories with postgres behavior 2026-05-16 23:43:40 +08:00
fawney19 a2f91b4108 Cancel upstream stream on downstream disconnect 2026-05-16 22:04:11 +08:00
fawney19 664bd98056 Merge pull request #479 from Entropy-Xu/codex/fix-balance-cost-estimate
fix(gateway): 修复额度预检误判余额不足
2026-05-16 21:23:23 +08:00
mayrain 5bf236957e feat(grok): add runtime image surfaces 2026-05-16 21:15:38 +08:00
mayrain 936e1ae37b feat(grok): add admin oauth and quota support 2026-05-16 21:15:38 +08:00
mayrain cbfe1d378f feat(grok): add provider pool and transport support 2026-05-16 21:15:38 +08:00
mayrain e5f1f52759 feat(model-test): wire image previews into provider tests 2026-05-16 21:15:27 +08:00
mayrain edacc5a7d0 feat(model-test): add image-aware request helpers 2026-05-16 21:15:27 +08:00
fawney19 ed9267562b Adjust provider detail key pagination 2026-05-16 21:08:25 +08:00
Entropy.Xu bddae47454 fix(gateway): 修复额度预检误判余额不足 2026-05-16 20:45:40 +08:00
fawney19 3a5922d4ee fix: use codex quota refresh for account checks 2026-05-16 20:16:23 +08:00
fawney19 56994d4c29 fix(frontend): relax system import limits 2026-05-16 19:00:20 +08:00
Entropy.Xu 973eb1a614 feat(referrals): 添加邀请返利和注册确认功能 2026-05-16 17:41:52 +08:00
HsungKayphoon f9f1fa928a refactor: drive pg sqlite copy from target schema 2026-05-16 17:29:14 +08:00
fawney19 328ac721ce Merge pull request #472 from Kayphoon/codex/manual-usage-cleanup-origin-main
feat: add scoped manual usage cleanup
2026-05-16 16:25:40 +08:00
HsungKayphoon 527feb69db fix: cover portable migration tables in logical schema 2026-05-16 15:58:41 +08:00
RWDai ba661f1b3c Add usage record column controls 2026-05-16 15:55:38 +08:00
RWDai 4f584a71df Add usage metadata frontend plumbing 2026-05-16 15:55:38 +08:00
RWDai 97cd92a1a2 Expose usage record metadata fields 2026-05-16 15:55:38 +08:00
RWDai df7b2824a6 Persist client family usage metadata 2026-05-16 15:55:38 +08:00
RWDai 03ba1f94d7 Show execution failure reasons in request details 2026-05-16 15:42:13 +08:00
RWDai 6dd5d2fe14 Add failure notice resolver for usage records 2026-05-16 15:42:13 +08:00
RWDai a2649718ea Expose scheduling failure details in usage payload 2026-05-16 15:42:13 +08:00
RWDai 9325c2ad9d fix(providers): remove unreachable manual model add flow 2026-05-16 15:35:26 +08:00
RWDai 590151f40b feat(models): allow manual global model creation 2026-05-16 15:27:34 +08:00
HsungKayphoon 4b12ec8913 feat: add postgres to single-node migration 2026-05-16 15:26:43 +08:00
HsungKayphoon 74a1e5ad7d feat: add scoped manual usage cleanup 2026-05-16 15:23:44 +08:00
fawney19 75b7319465 Merge pull request #471 from Kayphoon/fix/usage-provider-id-null 2026-05-16 15:19:30 +08:00
fawney19 6a608b8e3f fix: recover finalized usage provider links safely 2026-05-16 15:15:57 +08:00
beilo 2ca4b486ec fix: auto-remove invalid oauth pool keys 2026-05-16 15:08:16 +08:00
fawney19 c2cdcefdc8 fix(public): 恢复 support.rs 的 system_config_bool 引入 2026-05-16 14:28:19 +08:00
fawney19 893ac18d60 Merge PR #469: 修复用户可见性、额度、验证与 Codex 探测 2026-05-16 14:28:19 +08:00
fawney19 74abb50bdc fix(public): keep original GitHub links visible
(cherry picked from commit 7c93552697c9c35b77df35e117900ff8e9b62994)
2026-05-16 13:50:46 +08:00
Kayphoon f817f856c8 fix: allow upsert to backfill NULL provider link fields after billing finalizes
The ON CONFLICT update guard for provider_id, provider_endpoint_id, and
provider_api_key_id previously required billing_status = 'pending'. Once a
usage row left pending state with these fields still NULL, subsequent upserts
could never fill them. Add an OR IS NULL clause so missing provider links are
always recoverable regardless of billing status.
2026-05-16 13:11:59 +08:00
fawney19 3a23eaa572 Merge pull request #470 from yao177/fix/provider-keep-priority-on-conversion-load-balance
fix: preserve conversion priority beneath scheduler modes
2026-05-16 12:56:57 +08:00
fawney19 a7fdce493b feat: paginate provider keys from backend 2026-05-16 12:52:49 +08:00
yao177 232976c14a fix: keep conversion priority below load balance 2026-05-16 04:41:07 +00:00
yao177 dc1009798d test: cover cache affinity conversion priority ordering 2026-05-16 04:32:56 +00:00
fawney19 b6d74249a4 fix(public): keep original GitHub links visible
(cherry picked from commit 7c93552697c9c35b77df35e117900ff8e9b62994)
2026-05-16 11:38:49 +08:00
fawney19 f72ab383c9 Refine OAuth auto cleanup conditions
(cherry picked from commit 78826eae47ab18ace6931dd190a41070416b5e10)
2026-05-16 11:33:45 +08:00
fawney19 f59cf1090d feat(admin): unify system data management and aggregate import/export 2026-05-16 02:16:53 +08:00
Entropy.Xu 1a50c5e112 merge: 同步主线并解决用户侧验证冲突 2026-05-16 01:25:42 +08:00
fawney19 48da062251 Merge pull request #468 from fawney19/revert-pr-466
Revert PR #466
2026-05-16 01:01:44 +08:00
Entropy.Xu bbd3c30b0e fix(public): 修复用户可见性、额度、验证与 Codex 探测 2026-05-16 00:51:44 +08:00
fawney19 d6c320bf06 Revert "Merge remote-tracking branch 'origin/pr/466'"
This reverts commit 0e0a24862f, reversing
changes made to b09fd48eee.
2026-05-16 00:50:00 +08:00
fawney19 d53546d56f Make pool probing request-driven 2026-05-16 00:18:47 +08:00
fawney19 43d891bee1 Remove legacy Python tests 2026-05-16 00:06:21 +08:00
fawney19 0e0a24862f Merge remote-tracking branch 'origin/pr/466' 2026-05-15 22:49:35 +08:00
ZheFox 2345df0e38 fix(usage): bill cancelled terminal usage 2026-05-15 22:38:34 +08:00
fawney19 b09fd48eee Merge remote-tracking branch 'origin/pr/462' 2026-05-15 22:28:40 +08:00
fawney19 c916e76bd2 Merge branch 'pr-461'
# Conflicts:
#	.github/workflows/rust-ci.yml
#	crates/aether-data/src/lifecycle/bootstrap/postgres.rs
#	crates/aether-data/src/lifecycle/migrate/tests.rs
2026-05-15 21:52:37 +08:00
fawney19 8eb4c029b2 Merge pull request #467 from AAEE86/main
fix(usage): prevent detail view from overriding active request status
2026-05-15 21:08:48 +08:00
fawney19 87e44479cc Add proxy tunnel diagnostics and default logging 2026-05-15 19:43:59 +08:00
AAEE86 9c7757f801 fix(usage): prevent detail view from overriding active request status
- Keep pending/streaming lifecycle status authoritative for active requests
- Prevent detail status code or trace state from misclassifying in-flight requests as stream/failed
- Add regression coverage for status resolution and timeline state emission
2026-05-15 19:04:50 +08:00
fawney19 1503986d40 Fix compose installer defaults 2026-05-15 18:43:47 +08:00
fawney19 0bd3e2fa88 Merge remote-tracking branch 'origin/main' 2026-05-15 18:13:26 +08:00
fawney19 0f0b9a6118 chore: add db maintenance make targets 2026-05-15 18:10:49 +08:00
fawney19 e4f427f921 Merge remote-tracking branch 'entropy-xu/payment-billing-plans'
# Conflicts:
#	crates/aether-data/src/lifecycle/bootstrap/postgres.rs
#	crates/aether-data/src/lifecycle/migrate/tests.rs
#	frontend/src/views/admin/Users.vue
2026-05-15 16:37:17 +08:00
fawney19 2006a3e678 Merge pull request #464 from RWDai/fix/issue-455-codex-compact-include
fix: strip include from codex compact requests
2026-05-15 16:10:33 +08:00
fawney19 38240f77ff chore: replace dev script with make targets 2026-05-15 16:09:11 +08:00
RWDai 6014925e60 style: format codex compact regression tests 2026-05-15 14:40:27 +08:00
RWDai 1dc9b505e4 fix: strip include from codex compact requests 2026-05-15 14:20:16 +08:00
fawney19 b0ba1f250d Merge pull request #463 from AAEE86/main
chore(deps): update npm lockfiles
2026-05-15 14:13:14 +08:00
fawney19 2e9feaa60a Add local dev.sh script 2026-05-15 13:51:52 +08:00
AAEE86 8538364930 chore(deps): update npm lockfiles 2026-05-15 13:46:32 +08:00
fawney19 fb2662b877 Merge pull request #460 from RWDai/manual-provider-model-save-20260515
Allow manual provider model save without online discovery
2026-05-15 13:33:32 +08:00
fawney19 170218bb26 Merge pull request #458 from RWDai/opencode/sunny-orchid
Add one-click proxy node installation
2026-05-15 12:49:08 +08:00
fawney19 582504d11a Allow deploy to pull missing images 2026-05-15 11:29:16 +08:00
RWDai 88d8a8b79f Pin Rust CI component installs to 1.95.0 2026-05-15 10:14:50 +08:00
RWDai 8e4fc40be5 Refresh Rust CI for audit admin enum fix 2026-05-15 10:02:01 +08:00
fawney19 01a982bdf6 fix: preserve codex include fields 2026-05-15 03:25:06 +08:00
fawney19 daf33a82a6 deploy: restore local build and sqlite compose 2026-05-15 03:17:14 +08:00
fawney19 cc5a44e373 Merge branch 'pr-453' 2026-05-15 02:25:39 +08:00
fawney19 8e0f8003a9 fix: simplify account self-check config 2026-05-15 02:21:11 +08:00
fawney19 54a8312e46 feat: add adaptive pool metrics and self-check 2026-05-15 02:21:11 +08:00
Entropy.Xu 85f48123b6 feat(auth): 添加 Turnstile 注册防护 2026-05-15 01:54:20 +08:00
RWDai fdea51c7b1 Simplify proxy install command copy 2026-05-15 01:21:31 +08:00
RWDai 870bb19798 Update proxy installer branch references 2026-05-15 01:21:08 +08:00
RWDai dbec85344d Fix Rust CI toolchain setup 2026-05-15 01:20:36 +08:00
RWDai 4db01244ec Update Postgres bootstrap for audit admin role 2026-05-15 01:20:29 +08:00
RWDai 6021110fb2 Add audit admin Postgres userrole migration 2026-05-15 01:20:29 +08:00
RWDai 1216fa940e Allow manual provider model save without online discovery 2026-05-15 00:51:30 +08:00
RWDai bcf4adf944 Merge branch 'main' into opencode/sunny-orchid
# Conflicts:
#	README.md
2026-05-14 20:22:36 +08:00
RWDai 07ea745f56 Document proxy one-click installation 2026-05-14 19:35:41 +08:00
RWDai 3e122c84ef Add proxy node script install UI 2026-05-14 19:35:16 +08:00
RWDai 98a54b4633 Add proxy install session client API 2026-05-14 19:34:56 +08:00
RWDai 2db85b1c53 Add proxy one-click installer scripts 2026-05-14 19:34:33 +08:00
RWDai 91545cf906 Add admin proxy install session creation 2026-05-14 19:34:09 +08:00
RWDai 5a15822ce0 Add public proxy install session delivery 2026-05-14 19:33:47 +08:00
ZheFox eef21b6c34 Merge branch 'fawney19:main' into main 2026-05-14 17:47:47 +08:00
zhefox 498b3b1226 fix(auth): correct postgres api key create field ordering 2026-05-14 17:41:13 +08:00
zhefox 3b77686cee fix(auth): cast standalone api key expires_at update to timestamptz 2026-05-14 16:28:38 +08:00
fawney19 bf511f9f8c Merge pull request #449 from RWDai/feat/audit-admin-readonly
Add read-only audit administrator role
2026-05-14 15:51:27 +08:00
ZheFox a0eed2cc51 fix: split Codex chat and responses defaults 2026-05-14 15:33:03 +08:00
fawney19 e61aa46dad deploy: limit installer to supported modes 2026-05-14 15:26:58 +08:00
fawney19 d2831ec6f0 deploy: remove local build compose path 2026-05-14 14:40:25 +08:00
fawney19 a7e71624e3 ci: align release channels and speed rust checks 2026-05-14 14:09:27 +08:00
RWDai bc0017a1b4 Use audit admin permissions in proxy auth 2026-05-14 13:39:57 +08:00
RWDai 4cb7b2d494 Add audit admin internal read permissions 2026-05-14 13:39:57 +08:00
RWDai eb7c8a3ad5 Format audit admin role helpers 2026-05-14 13:39:57 +08:00
RWDai 6e2f90aba8 Keep audit admins on read-only admin views 2026-05-14 13:39:57 +08:00
RWDai 9c59c0e1a2 Add audit role to user management UI 2026-05-14 13:39:57 +08:00
RWDai 6a9a54cad6 Show audit admin labels in shared layouts 2026-05-14 13:39:57 +08:00
RWDai e768145961 Update public auth redirects for audit admins 2026-05-14 13:38:45 +08:00
RWDai a4e040a7d7 Route audit admins into admin console 2026-05-14 13:38:45 +08:00
RWDai e818443841 Add frontend audit admin auth semantics 2026-05-14 13:38:45 +08:00
RWDai 337d0af136 Accept audit role in user administration 2026-05-14 13:38:45 +08:00
RWDai 4d2667764f Enforce read-only admin route permissions 2026-05-14 13:38:45 +08:00
RWDai feb676b66f Allow audit admins through backend admin auth 2026-05-14 13:38:45 +08:00
RWDai 14871c2255 Add audit administrator role helpers 2026-05-14 13:38:45 +08:00
fawney19 48fe52b207 Update guide setup commands 2026-05-14 13:02:25 +08:00
fawney19 509bd30252 Redesign sensitive info protection settings 2026-05-14 11:14:20 +08:00
fawney19 91955ad1e0 Merge remote-tracking branch 'origin/pr/451' into aether-rust-pioneer 2026-05-14 02:10:37 +08:00
fawney19 b41a4a000f Merge remote-tracking branch 'origin/pr/435' into aether-rust-pioneer 2026-05-14 02:02:26 +08:00
fawney19 d29d1cf63b Remove deprecated Python source tree 2026-05-14 01:58:05 +08:00
fawney19 1f8ff7f6d2 Fix PR 434 check failures 2026-05-14 01:51:23 +08:00
fawney19 e251a63cb3 Merge remote-tracking branch 'pr-434/fix-provider-model-test-compat' into aether-rust-pioneer 2026-05-14 01:28:33 +08:00
fawney19 eb654c2fe7 Merge pull request #441 from zhefox/aether-rust-pioneer
fix: add codex reasoning defaults and stream rewrite tests
2026-05-14 00:35:45 +08:00
fawney19 697b6e0653 fix: stabilize openai responses reasoning streams 2026-05-14 00:27:10 +08:00
fawney19 acd44328d6 feat: add version update flow 2026-05-13 22:28:05 +08:00
fawney19 8b813f645f Merge branch 'pr-438' into aether-rust-pioneer 2026-05-13 22:28:05 +08:00
zhefox 2d354fa294 fix: preserve passthrough for same-format stream rewrites 2026-05-13 19:42:22 +08:00
fawney19 cfb22d3f06 Merge pull request #439 from Kayphoon/feat/macos-one-click-install
feat(compose): add solo sqlite compose deployment
2026-05-13 19:03:49 +08:00
fawney19 1a196c8cf2 Merge pull request #443 from AAEE86/rust
feat(admin): add Done-hub provider ops template
2026-05-13 18:42:28 +08:00
fawney19 f847a71747 Merge pull request #448 from RWDai/fix/disable-hidden-user-policies
Disable hidden per-user policy fields
2026-05-13 18:40:46 +08:00
RWDai 87c0a915b8 Align rate limit monitoring test with group policy 2026-05-13 18:27:56 +08:00
Kayphoon 2958041dc7 feat(gateway): add reversible chat pii redaction 2026-05-13 18:25:13 +08:00
fawney19 5d1460e051 refactor: extract provider pool abstractions 2026-05-13 18:19:15 +08:00
RWDai 6a9017bfce Cover disabled user policy API behavior 2026-05-13 17:56:18 +08:00
RWDai a1b0db38f5 Reject hidden user policy payloads 2026-05-13 17:56:18 +08:00
RWDai a99546b390 Disable user policy during auth resolution 2026-05-13 17:56:17 +08:00
zhefox 1adbf23be4 feat(ai-formats): add reasoning summary boundaries for streams 2026-05-13 17:41:34 +08:00
AAEE86 afbb656510 feat(admin): add Done-hub provider ops template
- 新增 Done-hub Cookie 认证架构
- 使用 /api/user/profile 查询余额,按 quota / 500000 换算
- 补充余额解析、认证头和校验相关测试
2026-05-13 16:10:59 +08:00
zhefox 1726df1169 fix: add codex reasoning defaults and stream rewrite tests 2026-05-13 14:34:16 +08:00
Kayphoon d69d862034 feat(compose): add solo sqlite compose deployment 2026-05-13 11:21:35 +08:00
RWDai a03cab4a09 Show update status on admin dashboard 2026-05-13 10:28:18 +08:00
RWDai c90ed14a24 Fetch Aether releases for update checks 2026-05-13 10:27:45 +08:00
RWDai e00df5f7a2 Add admin update payload builder 2026-05-13 10:27:21 +08:00
fawney19 3c2497f019 Fix admin pool sorting and OAuth refresh 2026-05-13 09:16:52 +08:00
Entropy.Xu 3fb007a56d Merge remote-tracking branch 'origin/aether-rust-pioneer' into payment-billing-plans
# Conflicts:
#	crates/aether-data/src/lifecycle/bootstrap/postgres.rs
#	crates/aether-data/src/lifecycle/migrate/tests.rs
2026-05-13 01:29:18 +08:00
Entropy.Xu 10285c5eb9 feat: add payment gateway and billing plans 2026-05-13 01:18:38 +08:00
Kayphoon 15800d7a80 feat(admin): manual request-records cleanup with typed confirmation 2026-05-13 00:36:43 +08:00
fawney19 4387a9cdd5 Sync Cargo.lock for release build 2026-05-13 00:07:55 +08:00
fawney19 8714d93d4b Stabilize Codex OAuth import tests 2026-05-12 23:36:43 +08:00
fawney19 68256ece2d Fix Kiro manual OAuth refresh test stack 2026-05-12 23:09:09 +08:00
fawney19 339808d55b 更新aether-proxy版本号 2026-05-12 22:46:49 +08:00
fawney19 e7471f44b0 fix provider oauth endpoint reconciliation 2026-05-12 22:46:20 +08:00
fawney19 d1a47c068e feat: update gateway pool and usage flows 2026-05-12 21:05:11 +08:00
mayrain 43c476d54a fix: refine provider model test dialog 2026-05-12 20:11:37 +08:00
mayrain 9f26383de5 fix: route provider model tests through candidates 2026-05-12 20:11:16 +08:00
mayrain 8f082674d7 fix: align embedding provider request formats 2026-05-12 20:10:52 +08:00
mayrain fca3f24d91 fix: preserve legacy admin config imports 2026-05-12 20:10:27 +08:00
fawney19 38012c62ff Allow full management tokens to access token management 2026-05-12 19:53:31 +08:00
fawney19 63149fe281 Add tunnel overload protection 2026-05-12 18:36:41 +08:00
fawney19 5509f70ad4 Bump proxy version to 0.3.9 2026-05-12 18:36:41 +08:00
github-actions[bot] 110cb147d1 chore(proxy): update download links for proxy-v0.3.9 2026-05-12 09:04:17 +00:00
fawney19 92e4066977 Merge pull request #432 from RWDai/fix/cli-install-copy
Add admin API key CLI install sessions
2026-05-12 16:50:24 +08:00
fawney19 ba5e802174 Merge pull request #433 from Kayphoon/feat/macos-one-click-install
feat(install): add macOS native one-click install with launchd
2026-05-12 16:49:50 +08:00
fawney19 4b2507b155 Fix score popover duplication 2026-05-12 16:04:00 +08:00
Kayphoon 6d2fcf12cd feat(install): add macOS native one-click install with launchd
- Release workflow builds macos-amd64/macos-arm64 tarballs on native runners
- install.sh detects Darwin, downloads macos-* assets, installs LaunchDaemon
- Dedicated _aether service account (dscl), env root:_aether 0640
- Launchd stdout/stderr in /var/log/aether (root-owned dir, service-writable files)
- Wrapper script parses env literally without shell expansion
- Auto-prune old releases (default keep 3)
- README documents macOS launchd commands
2026-05-12 15:36:29 +08:00
RWDai f60fc1cd7a fix: improve api key install copy flow 2026-05-12 15:34:16 +08:00
RWDai 9fc270fe69 feat: add admin api key install dialog 2026-05-12 15:34:08 +08:00
RWDai a8ff050518 fix: narrow api key install session types 2026-05-12 15:33:53 +08:00
RWDai a7bab0ebd2 feat: add admin api key install sessions 2026-05-12 15:33:45 +08:00
RWDai 8eda0932b0 fix: preserve install base URLs 2026-05-12 15:33:30 +08:00
fawney19 4e563e3385 Adjust proxy tunnel sizing defaults 2026-05-12 15:30:55 +08:00
fawney19 9c05b5f4e0 Unify pool score display as account health 2026-05-12 15:27:21 +08:00
fawney19 fa73655134 refactor: isolate dispatch scheduling core 2026-05-12 13:15:41 +08:00
fawney19 81ee27cdea Merge remote-tracking branch 'origin/codex/pool-member-scores' into aether-rust-pioneer 2026-05-12 09:17:25 +08:00
fawney19 fad28eee2c Improve pool score probing rules 2026-05-12 09:08:10 +08:00
fawney19 c578689356 Fix pool score CI regressions 2026-05-12 02:17:19 +08:00
fawney19 b9e62d1667 Implement generic pool member scoring and probing 2026-05-12 01:46:22 +08:00
fawney19 09146f8cdd fix: tighten model candidate matching scopes 2026-05-12 00:52:19 +08:00
fawney19 0fa97595bf Fix reasoning model directive response identity 2026-05-11 23:03:11 +08:00
fawney19 7ae38b6c43 Merge pull request #429 from AAEE86/rust
fix(usage): 统一用户排行榜 Token 统计口径
2026-05-11 22:46:56 +08:00
AAEE86 bfbf7a4663 fix(usage): 统一用户排行榜 Token 统计口径
- 将用户排行榜 Tokens 调整为与仪表盘今日 Token 一致
- 统一 effective input、cache creation fallback、cache read 计算规则
- 补齐 Postgres 聚合、admin 内存构建和内存仓库回退路径
- 增加缓存命中场景的统计口径测试
2026-05-11 22:34:37 +08:00
fawney19 cb0ccb9cdb fix postgres user role enum casts in migrations 2026-05-11 20:31:47 +08:00
fawney19 ce70780851 Fix proxy node uptime bucket rendering 2026-05-11 20:22:40 +08:00
fawney19 30b18d3310 fix: use gateway healthcheck flag in compose 2026-05-11 19:21:46 +08:00
fawney19 1d33e2c51b Fix Gemini model denial test fixture 2026-05-11 18:40:28 +08:00
fawney19 fed676f54f fix: exempt admins from default user group limits 2026-05-11 18:20:01 +08:00
fawney19 9bed5e9f83 fix: use intersection for user group policies 2026-05-11 17:00:07 +08:00
fawney19 e16a225eb3 Merge pull request #428 from AAEE86/rust
fix(gateway): 修复正则模型映射未生效到出站请求
2026-05-11 16:04:56 +08:00
AAEE86 67ca47afd4 fix(gateway): 修复正则模型映射未生效到出站请求
修复 key allowed_models 通过 global_model_mappings 正则命中时,
候选选择仍使用 provider model mapping 作为出站模型的问题。

正则命中后将 allowed model 写入 selected_provider_model_name,
确保最终 execution runtime 请求体中的 model 使用映射后的模型。

补充三层回归测试:
- scheduler-core 正则映射解析
- gateway candidate/data 候选选择输出
- gateway ai_execute 出站请求 model 捕获
2026-05-11 15:58:13 +08:00
fawney19 9057537ab8 fix: preserve model associations on refresh 2026-05-11 14:52:50 +08:00
fawney19 247ea9d1bd Restrict scheduler affinity to cache affinity mode 2026-05-11 14:06:49 +08:00
fawney19 e91c874863 fix(migrations): explicitly set timestamps in system_configs insert
老库的 created_at/updated_at 没有 DEFAULT now(),依赖默认值会写入 NULL
触发 NOT NULL 约束失败,改为显式传入 now() 与 sqlite/mysql 版本保持一致
2026-05-11 03:22:46 +08:00
github-actions[bot] 40470d8ca5 chore(proxy): update download links for proxy-v0.3.8 2026-05-10 19:10:49 +00:00
fawney19 94c0076778 chore(proxy): bump version to 0.3.8 2026-05-11 03:03:04 +08:00
fawney19 b813498e40 feat(proxy): surface node resource diagnostics 2026-05-11 02:59:18 +08:00
fawney19 cc4512fbbb Refine load balance candidate ranking 2026-05-11 02:32:11 +08:00
fawney19 ef4cc55c9a Record per-candidate upstream error bodies 2026-05-11 02:30:45 +08:00
fawney19 e3574e1918 Track scheduler affinity epochs and key sorting 2026-05-11 01:45:49 +08:00
fawney19 7b81c77424 Fix regex model mapping direction 2026-05-11 01:43:14 +08:00
fawney19 c9c968c2e9 fix: avoid duplicate user policy migration version 2026-05-11 00:39:34 +08:00
fawney19 e3f8fef30c Merge remote-tracking branch 'origin/pr/425' into aether-rust-pioneer 2026-05-11 00:19:50 +08:00
fawney19 ceda0635e4 Merge remote-tracking branch 'origin/pr/424' into aether-rust-pioneer
# Conflicts:
#	crates/aether-provider-transport/src/vertex/auth.rs
#	crates/aether-provider-transport/src/vertex/mod.rs
#	crates/aether-provider-transport/src/vertex/policy.rs
2026-05-11 00:19:04 +08:00
fawney19 bacb14e5f0 refactor: lazy pool key scheduling 2026-05-11 00:12:05 +08:00
Entropy.Xu 9e705ff603 Fix legacy user policy modes for empty lists 2026-05-10 19:35:09 +08:00
fawney19 1a0f1a7b72 Merge branch 'codex/pr-416-420-integration' into aether-rust-pioneer 2026-05-10 19:22:28 +08:00
fawney19 3201851667 Merge remote-tracking branch 'origin/pr/416' into codex/pr-416-420-integration 2026-05-10 19:13:42 +08:00
Codex 75df21932a 修复 Gemini 工具结果透传 Vertex 格式 2026-05-10 19:08:35 +08:00
Codex fb96771b56 修复 Vertex AI 服务账号访问 2026-05-10 19:08:27 +08:00
fawney19 c4b484fb43 fix: satisfy vertex transport lint checks 2026-05-10 19:07:51 +08:00
fawney19 37fb79fb87 fix: drop stray auth modules migration 2026-05-10 18:57:58 +08:00
fawney19 f03039d846 Merge remote-tracking branch 'origin/pr/420' into codex/pr-416-420-integration 2026-05-10 18:44:29 +08:00
fawney19 69476f69b9 Merge remote-tracking branch 'origin/pr/416' into codex/pr-416-420-integration
# Conflicts:
#	crates/aether-data/src/lifecycle/bootstrap/postgres.rs
#	crates/aether-data/src/lifecycle/migrate/tests.rs
2026-05-10 18:44:19 +08:00
Entropy.Xu bb22978574 Merge remote-tracking branch 'upstream/aether-rust-pioneer' into fix-management-token-oauth-jsonb
# Conflicts:
#	crates/aether-data/src/lifecycle/bootstrap/postgres.rs
#	crates/aether-data/src/lifecycle/migrate/tests.rs
2026-05-10 18:36:53 +08:00
fawney19 228253c166 Merge pull request #410 from Entropy-Xu/codex/codex-image-progress-heartbeat
Add Codex image progress heartbeat
2026-05-10 17:57:33 +08:00
fawney19 d26321006b Merge remote-tracking branch 'origin/aether-rust-pioneer' into aether-rust-pioneer 2026-05-10 17:48:54 +08:00
fawney19 377dd52805 style: polish multi select dropdown 2026-05-10 17:40:32 +08:00
fawney19 d246f6b42c fix: align user group access controls 2026-05-10 17:28:23 +08:00
Entropy.Xu 59d16ebb4b Merge remote-tracking branch 'upstream/aether-rust-pioneer' into fix-management-token-oauth-jsonb
# Conflicts:
#	crates/aether-data/src/lifecycle/bootstrap/postgres.rs
#	crates/aether-data/src/lifecycle/migrate/tests.rs
2026-05-10 16:21:09 +08:00
Entropy.Xu 948a173f39 fix: avoid management token migration version collision 2026-05-10 16:16:11 +08:00
Entropy.Xu 56857280d3 fix: tolerate legacy management token json columns 2026-05-10 15:59:13 +08:00
Codex 7e804c408f 修复 Vertex AI 服务账号访问 2026-05-10 15:51:47 +08:00
Entropy.Xu 5268f62a64 feat: add sync image heartbeat toggle 2026-05-10 12:40:52 +08:00
github-actions[bot] 7f101431c5 chore(proxy): update download links for proxy-v0.3.7 2026-05-10 03:27:15 +00:00
fawney19 a8ac944959 Merge remote-tracking branch 'entropy-xu/codex/user-groups-default-permissions' into aether-rust-pioneer 2026-05-10 11:26:29 +08:00
fawney19 e8259a76f0 chore(proxy): bump version to 0.3.7 2026-05-10 11:19:26 +08:00
fawney19 3983b5a5cf Merge pull request #419 from wendaochangsheng/aether-rust-pioneer
修复 Vertex AI 服务账号 JSON 导入入口
2026-05-10 09:27:22 +08:00
fawney19 79b3a76dc1 Merge pull request #411 from buniakinazach-dev/feat/codex-spark-quota
新增 Codex Spark 额度展示
2026-05-10 09:25:44 +08:00
fawney19 c2bf17b4dd Fix Codex Spark quota formatting 2026-05-10 09:14:19 +08:00
Codex be3afbd279 修复 Vertex AI 服务账号 JSON 导入入口 2026-05-10 04:21:29 +08:00
fawney19 18690ceed2 Merge pull request #414 from stabey/fix/claude-tool-results-openai-chat
fix: preserve Claude tool results in OpenAI chat conversion
2026-05-10 02:50:36 +08:00
fawney19 0f42a6ed82 Fix Codex image progress heartbeat merge regressions 2026-05-10 02:10:23 +08:00
Entropy.Xu 545299fc62 fix: align management token oauth permissions and jsonb schema 2026-05-10 01:47:48 +08:00
fawney19 3c1456706a Merge pull request #405 from Entropy-Xu/codex/async-cleanup-records
Add async request body cleanup records
2026-05-10 01:12:12 +08:00
fawney19 a81053e6ff fix(admin): run destructive purges as cleanup tasks 2026-05-10 00:41:21 +08:00
fawney19 391c2fbe5d Merge remote-tracking branch 'origin/aether-rust-pioneer' into codex/async-cleanup-records
# Conflicts:
#	apps/aether-gateway/src/maintenance/mod.rs
#	apps/aether-gateway/src/maintenance/runtime/runners.rs
2026-05-10 00:28:39 +08:00
Entropy.Xu 121bdbd614 fix(data): avoid duplicate user group migration version 2026-05-09 22:26:31 +08:00
stabey dcfdba0a97 fix: preserve claude tool results in openai chat conversion
Preserve all Claude tool_result blocks when emitting OpenAI Chat messages, including multimodal image/file content and is_error markers.
2026-05-09 22:16:27 +08:00
Entropy.Xu a68c690874 Merge remote-tracking branch 'upstream/aether-rust-pioneer' into codex/user-groups-default-permissions 2026-05-09 22:04:09 +08:00
Entropy.Xu 3a814f3d1f feat: add user groups and inherited access policies 2026-05-09 21:47:33 +08:00
fawney19 209322b499 feat(gateway): unify background task runtime and storage 2026-05-09 21:19:29 +08:00
fawney19 4a64d078f3 Merge pull request #407 from stabey/pr/provider-stream-policy
fix: enforce provider upstream stream policy
2026-05-09 13:43:36 +08:00
stabey 5f4fa4ce1f fix: preserve upstream stream accept negotiation 2026-05-09 12:54:47 +08:00
stabey 7e5a08e09d fix: enforce provider upstream stream policy 2026-05-09 12:30:56 +08:00
fawney19 8958bf5e08 Merge pull request #409 from RWDai/feat/issue-373-cli-install
feat: add API key CLI install flow
2026-05-09 12:06:58 +08:00
fawney19 8b67120964 Merge pull request #402 from AAEE86/rust
feat(model-fetch): fetch Kiro models from upstream
2026-05-09 12:03:15 +08:00
fawney19 79d07ac79a Improve request trace upstream diagnostics 2026-05-09 10:46:04 +08:00
Alice 757c264e3e 新增 Codex Spark 额度展示
解析 GPT-5.3-Codex-Spark 额度窗口,并在号池页面和提供商账号抽屉中展示 Spark 周额度与 5H 额度。Spark 额度仅用于展示,不影响普通 Codex 周额度和 5H 额度的调度与筛选逻辑。
2026-05-09 10:44:45 +08:00
RWDai a72bf19454 fix(users): buffer API key install session body 2026-05-09 10:29:36 +08:00
Entropy.Xu eda3738d59 Merge branch 'aether-rust-pioneer' of https://github.com/fawney19/Aether into codex/codex-image-progress-heartbeat
# Conflicts:
#	frontend/src/features/usage/components/__tests__/HorizontalRequestTimeline.spec.ts
2026-05-09 01:26:44 +08:00
Entropy.Xu 3b4f27f767 Add Codex image progress heartbeat 2026-05-09 01:21:26 +08:00
fawney19 4cf0de681a feat(proxy): add node metrics expansion panel 2026-05-08 23:20:50 +08:00
RWDai 2c865ede35 feat(users): add CLI install modal 2026-05-08 23:16:16 +08:00
RWDai 46f44f4ea7 feat(users): add install session client 2026-05-08 23:16:16 +08:00
RWDai da46eb4791 feat(users): add API key install sessions 2026-05-08 23:16:16 +08:00
fawney19 e21fb58479 chore(proxy): guard metrics retention cleanup 2026-05-08 22:57:00 +08:00
fawney19 a703acd1fe feat(proxy): record tunnel stability metrics 2026-05-08 22:03:17 +08:00
fawney19 9a84a6ff6c refactor(ai-formats): group formats by provider
Move protocol/request/response format modules under provider-oriented formats modules and update registry, transport, and architecture paths.
2026-05-08 15:51:14 +08:00
fawney19 84a84e3f31 Merge pull request #404 from RWDai/fix/python-v063-user-import-compat
Preserve Python user import compatibility
2026-05-08 14:20:08 +08:00
Entropy.Xu 3f29335fd6 Add async request body cleanup records 2026-05-08 13:50:18 +08:00
RWDai 141a81d4a3 Preserve Python user import compatibility 2026-05-08 13:37:32 +08:00
AAEE86 2543676437 feat(model-fetch): fetch Kiro models from upstream
- add Kiro ListAvailableModels request planning and headers
- route Kiro model refresh through upstream fetch
- normalize Kiro model payloads and default model metadata
- remove profileArn requirement from ListAvailableModels
2026-05-08 13:24:27 +08:00
fawney19 fa22384f24 Merge remote-tracking branch 'origin/pr/399' into aether-rust-pioneer
# Conflicts:
#	crates/aether-data/src/lifecycle/bootstrap/postgres.rs
#	crates/aether-data/src/lifecycle/migrate/tests.rs
2026-05-08 02:52:13 +08:00
fawney19 2a603fa1e4 Merge remote-tracking branch 'origin/pr/397' into aether-rust-pioneer 2026-05-08 02:47:13 +08:00
fawney19 31d142effc Merge remote-tracking branch 'origin/pr/395' into aether-rust-pioneer 2026-05-08 02:47:07 +08:00
fawney19 66d8e90647 Merge remote-tracking branch 'origin/pr/394' into aether-rust-pioneer 2026-05-08 02:46:35 +08:00
fawney19 080784cd9f Refactor provider transport modules 2026-05-08 02:34:45 +08:00
fawney19 c52ef1993f Improve provider OAuth device flow 2026-05-08 01:33:23 +08:00
fawney19 6247ac3edc refactor: extract runtime state backends 2026-05-08 00:18:12 +08:00
Entropy.Xu cc5cb3475e feat: add management token permissions 2026-05-07 23:48:29 +08:00
RWDai 71742d5ad2 Fix pool cycle usage display 2026-05-07 22:25:52 +08:00
RWDai 1bc0822ce6 fix(users): align batch selection checkbox 2026-05-07 22:22:55 +08:00
RWDai fcefa4d198 feat(users): refine batch action dialog 2026-05-07 22:22:55 +08:00
RWDai 1e2ff26e86 fix(users): harden batch role and quota updates 2026-05-07 22:22:55 +08:00
Entropy.Xu dd8c2ebec6 fix(admin): repair system maintenance controls
Implement server-side searchable user filtering for usage records, including backend search parameters and a shared frontend selector with loading, empty, and pinned-selected states.

Fix announcement deletion by cascading announcement read rows through a Postgres migration and defensive repository cleanup across supported backends.

Wire admin system purge and cleanup endpoints to real data deletion/maintenance flows, improve DataManagement messages, rebuild stats after stats purge, and add runtime OAuth token refresh maintenance when enabled.

Verified with rust-ci equivalent checks: cargo fmt, split clippy, split cargo tests, SQLite/Postgres/MySQL smoke tests, plus frontend npm ci, build, pages build, type-check, and targeted usage selector tests.
2026-05-07 21:26:40 +08:00
fawney19 6f620d92be Skip release checksum verification in installer 2026-05-07 19:32:39 +08:00
RWDai 61473bfb77 feat(users): wire batch actions into users page 2026-05-07 19:16:49 +08:00
RWDai b7172092e8 feat(users): add batch action dialog 2026-05-07 19:16:49 +08:00
RWDai ea014e0d89 feat(users): add batch API store methods 2026-05-07 19:16:49 +08:00
RWDai 8fa90e8ecf refactor(users): share access control options 2026-05-07 19:16:49 +08:00
RWDai 9eb17eca32 refactor(pool): share batch selection helpers 2026-05-07 19:16:49 +08:00
RWDai 18b247de4c test(users): cover batch admin endpoints 2026-05-07 19:16:49 +08:00
RWDai 94852ce60e feat(users): classify batch admin routes 2026-05-07 19:16:49 +08:00
RWDai 59312ebe73 feat(users): add batch admin handlers 2026-05-07 19:16:49 +08:00
RWDai 738031696b fix(users): thread rate limit presence through gateway 2026-05-07 19:16:49 +08:00
RWDai a44667d2a9 fix(users): preserve nullable rate limit updates 2026-05-07 19:16:49 +08:00
fawney19 b8fc36033b Improve installer language and download source prompts 2026-05-07 18:53:09 +08:00
fawney19 6ef0bd9488 fix(ci): clean stale release drafts before publish 2026-05-07 16:30:39 +08:00
fawney19 f3d9523502 Expose upstream proxy in proxy setup TUI 2026-05-07 16:20:20 +08:00
fawney19 2b439094c5 Fix endpoint condition source handling 2026-05-07 15:56:28 +08:00
fawney19 6317587a6b Merge pull request #391 from RWDai/fix/affinity-list-display
Fix affinity list display
2026-05-07 15:54:53 +08:00
github-actions[bot] 7fdbd0c808 chore(proxy): update download links for proxy-v0.3.6 2026-05-07 07:26:05 +00:00
RWDai 44c2534f46 Fix affinity monitoring CI checks 2026-05-07 15:24:17 +08:00
fawney19 af8f9e9057 chore(proxy): bump version to 0.3.6 2026-05-07 15:18:43 +08:00
RWDai cb9d9944e3 Improve affinity list display 2026-05-07 15:08:30 +08:00
RWDai 2cfeb14d88 Add session-scoped affinity deletion 2026-05-07 15:08:14 +08:00
RWDai d5d93eda11 Fix affinity key parsing and counters 2026-05-07 15:08:07 +08:00
fawney19 9bbdf889d4 feat(proxy): support upstream egress proxy 2026-05-07 15:01:18 +08:00
fawney19 96f9be26da polish usage output rate and installer progress 2026-05-07 13:39:25 +08:00
fawney19 0c03c188f1 fix codex window usage stats 2026-05-07 13:35:25 +08:00
fawney19 64af7b1d8a fix: preserve selected keys for quota refresh 2026-05-07 12:55:39 +08:00
github-actions[bot] a345bb8c0d chore(proxy): update download links for proxy-v0.3.5 2026-05-07 04:01:32 +00:00
4028 changed files with 966871 additions and 553016 deletions
+6
View File
@@ -1,12 +1,18 @@
# Build artifacts
build/
target/
*.so
*.egg
*.egg-info/
# Frontend
frontend/node_modules/
frontend/dist/
frontend/.vite/
aether-vscodex/web/node_modules/
aether-vscodex/web/dist/
aether-vscodex/vscode-extension/node_modules/
aether-vscodex/vscode-extension/dist/
# Development
.git/
+89 -12
View File
@@ -4,6 +4,17 @@
# 应用端口(默认 8084)
APP_PORT=8084
# 对外访问地址,用于一键安装、CC Switch 导入、支付回调等需要生成公网 URL 的场景。
# 生产环境建议显式配置为不带内部端口的公网域名,例如 https://aether.example.com
# AETHER_PUBLIC_BASE_URL=https://aether.example.com
# Docker Compose 镜像(默认正式版 latest;提前测试可改 rc/beta;也可固定具体版本)
# 示例:
# APP_IMAGE=ghcr.io/fawney19/aether:latest
# APP_IMAGE=ghcr.io/fawney19/aether:rc
# APP_IMAGE=ghcr.io/fawney19/aether:beta
# APP_IMAGE=ghcr.io/fawney19/aether:0.7.0-rc.1
# API Key 前缀(默认 sk)
API_KEY_PREFIX=sk
@@ -15,21 +26,25 @@ RUST_LOG=aether_gateway=info
# 示例: http://localhost:5173,https://app.example.com
# CORS_ORIGINS=http://localhost:5173
# CORS_ALLOW_CREDENTIALS=true
# 如果前后端跨站并依赖登录刷新 Cookie,还要配合:
# 登录刷新 Cookie 对同源浏览器请求和可信反代自动适配 HTTP/HTTPS。
# HTTP 自动使用兼容的 SameSite=Lax(显式 Strict 保留);HTTPS 保留原有 SameSite 配置。
# 无法确认访问协议时保留安全默认值;HTTPS 反代请正确传递 X-Forwarded-Proto。
# AUTH_REFRESH_COOKIE_SECURE 可显式覆盖自动判断,公网部署仍建议使用 HTTPS。
# 如果前后端跨站并依赖登录刷新 Cookie,必须使用 HTTPS,并配合:
# AUTH_REFRESH_COOKIE_SAMESITE=None
# AUTH_REFRESH_COOKIE_SECURE=true
# 数据库配置
# 数据库配置(仅支持 PostgreSQL)
DB_HOST=localhost
DB_PORT=5432
DB_USER=postgres
DB_NAME=aether
DB_PASSWORD=aether
DB_PASSWORD=
# Redis 配置
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_PASSWORD=aether
REDIS_PASSWORD=
# JWT密钥(使用 ./generate_keys.sh 生成)
# 用于用户登录 token 签名,更换后所有用户需重新登录
@@ -39,18 +54,80 @@ JWT_SECRET_KEY=change-this-to-a-secure-random-string
# 注意:更换此密钥后需要在管理面板重新配置所有 Provider API Key
ENCRYPTION_KEY=change-this-to-another-secure-random-string
# S3 备份的独立加密密钥(推荐)。未配置时为兼容旧部署,会回退到 ENCRYPTION_KEY。
# 密钥轮换前必须保留旧值,离线恢复工具需要它解密历史备份。
# AETHER_BACKUP_ENCRYPTION_KEY=change-this-to-a-dedicated-secure-random-string
# 启动自举管理员(仅在当前库里还没有活动管理员时生效)
# 手动部署时取消注释并设置;install.sh 首次生成配置时会提示输入。
# 首次启动前必须设置 ADMIN_PASSWORD;install.sh 首次生成配置时会提示输入。
ADMIN_EMAIL=[email protected]
ADMIN_USERNAME=admin
ADMIN_USERNAME=admin123456
# ADMIN_PASSWORD=
# ==================== 可选配置(有默认值) ====================
# docker compose 下 app 启动前自动执行 pending migration/backfill(默认 true)
# AETHER_GATEWAY_AUTO_PREPARE_DATABASE=true
# 可信反向代理 IP/CIDR,只有这些来源发送的 X-Real-IP / X-Forwarded-For 会被采用。
# 默认仅信任本机回环代理:127.0.0.0/8,::1/128。
# Docker/Nginx 位于独立容器时,请按实际容器网络设置,例如:172.16.0.0/12。
# AETHER_TRUSTED_PROXY_CIDRS=127.0.0.0/8,::1/128,172.16.0.0/12
# PostgreSQL 连接池配置(默认适合单实例/小型部署;高并发可按需调大)
# AETHER_GATEWAY_DATA_POSTGRES_MIN_CONNECTIONS=1
# AETHER_GATEWAY_DATA_POSTGRES_MAX_CONNECTIONS=20
# AETHER_GATEWAY_DATA_POSTGRES_IDLE_TIMEOUT_MS=30000
# VS Code Codex 云端协同(仅在叠加 aether-vscodex/docker-compose.aether.yml 时需要)
# 内部 token 至少 24 字节,建议使用:openssl rand -base64 32
# AETHER_VSCODEX_INTERNAL_TOKEN=replace-with-a-long-random-secret
# AETHER_VSCODEX_PUBLIC_WS_URL=wss://aether.example.com/api/vscodex/ws
# AETHER_VSCODEX_ALLOWED_ORIGINS=https://aether.example.com
# 启动时的数据库准备策略:auto(默认)或 verify-only
# AETHER_GATEWAY_DATABASE_MODE=auto
# PostgreSQL 连接池配置(默认每核 4 条、总池至少 32 条且最多 100 条;多实例部署应显式分配每实例预算)
# AETHER_GATEWAY_DATA_POSTGRES_MIN_CONNECTIONS=12
# AETHER_GATEWAY_DATA_POSTGRES_MAX_CONNECTIONS=80
# AETHER_GATEWAY_MAX_IN_FLIGHT_REQUESTS=2048
# AETHER_GATEWAY_REQUEST_BODY_BUFFER_BUDGET_MB=256
# 请求体完整读取总超时默认关闭;确需限制时配置 1000-600000 毫秒的非零值。
# AETHER_GATEWAY_REQUEST_BODY_READ_TIMEOUT_MS=0
# 单请求解压后 Payload 上限(MiB),默认 256;显式设为 0 才表示不限制。
# AETHER_MAX_REQUEST_BODY_MB=256
# AETHER_GATEWAY_SECURITY_CACHE_TTL_MS=1000
# AETHER_MAX_REDACTED_SYNC_RESPONSE_BODY_MB=64
# AETHER_MAX_INTERNAL_BUFFERED_BODY_MB=64
# AETHER_TUNNEL_NODE_STATUS_QUEUE_CAPACITY=1024
# Tunnel relay 使用的独立 HMAC 密钥。启用 HTTP tunnel relay 或多网关 owner 转发时必须配置,
# 所有网关实例必须使用同一个至少 32 字节的随机值;不要复用 JWT 或数据加密密钥。
# AETHER_TUNNEL_RELAY_AUTH_SECRET=
# 旧版 /api/internal/gateway/* 控制面默认关闭。确需独立服务调用时,配置至少 32 字节的
# 独立 HMAC 密钥;不要复用 JWT、数据加密或 tunnel relay 密钥。多节点必须使用相同值和共享 Redis。
# AETHER_INTERNAL_GATEWAY_AUTH_SECRET=
# 远程 relay 地址必须使用 HTTPS;HTTP 仅允许 localhost 或回环 IP。
# AETHER_TUNNEL_RELAY_BASE_URL=https://gateway-a.example.com
# 跨网关 relay 解析到受控私有地址时才显式开启;默认关闭以防止被篡改的 attachment
# 记录诱导网关向内网转发 relay 凭据。该开关不放宽普通 provider 的目标地址策略。
# AETHER_TUNNEL_RELAY_ALLOW_PRIVATE_TARGETS=false
# 更推荐按 relay 主机名精确放行私网部署(逗号分隔,大小写不敏感);不支持通配符/后缀。
# AETHER_TUNNEL_RELAY_PRIVATE_HOST_ALLOWLIST=gateway-a.internal,gateway-b.internal
# Bark 自建服务默认仅允许公网 HTTPS。确需明文 HTTP 或内网目标时分别显式开启:
# AETHER_BARK_ALLOW_HTTP=false
# AETHER_BARK_ALLOW_PRIVATE_TARGETS=false
# 普通 Provider 反代(包括 Provider OAuth)不按 DNS 地址过滤上游,兼容任意
# Fake-IP 域名及内网 DNS。仅信任管理员配置的上游;没有严格 DNS 过滤开关。
# URL 协议、字面 IP、TLS 证书,以及隧道中继和登录 OAuth 的校验仍保留。
# 可选 Provider OAuth 客户端。Gemini CLI 和 Antigravity 默认使用内置 native-app
# 客户端凭据;自定义 client ID 时必须同时配置对应的 client secret。
# 显式配置的 client secret 优先于默认值。
# AETHER_GEMINI_CLI_OAUTH_CLIENT_ID=
# AETHER_GEMINI_CLI_OAUTH_CLIENT_SECRET=
# AETHER_ANTIGRAVITY_OAUTH_CLIENT_ID=
# AETHER_ANTIGRAVITY_OAUTH_CLIENT_SECRET=
# PostgreSQL 容器调优:docker-compose.yml 已内置通用默认值,通常不用配置。
# 只有在 Postgres 独占大内存、或压测显示 DB 缓存/排序/维护任务成为瓶颈时再覆盖。
# 内置默认:shared_buffers=1GB, effective_cache_size=3GB, shm_size=512mb,
# work_mem=16MB, maintenance_work_mem=256MB。
# POSTGRES_SHARED_BUFFERS=8GB
# POSTGRES_EFFECTIVE_CACHE_SIZE=24GB
# POSTGRES_SHM_SIZE=2gb
# POSTGRES_WORK_MEM=16MB
# POSTGRES_MAINTENANCE_WORK_MEM=1GB
-213
View File
@@ -1,213 +0,0 @@
name: Build aether-proxy
on:
push:
tags: ['proxy-v*']
workflow_dispatch:
permissions:
contents: write
jobs:
preflight:
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/')
steps:
- uses: actions/checkout@v5
- name: Ensure proxy tag matches Cargo version
shell: bash
run: |
TAG="${GITHUB_REF_NAME}"
EXPECTED="${TAG#proxy-v}"
ACTUAL="$(cargo metadata --manifest-path apps/aether-proxy/Cargo.toml --locked --no-deps --format-version 1 | jq -r '.packages[] | select(.name == "aether-proxy") | .version')"
echo "tag version: ${EXPECTED}"
echo "cargo version: ${ACTUAL}"
if [ -z "${ACTUAL}" ]; then
echo "Could not resolve aether-proxy package version" >&2
exit 1
fi
if [ "${EXPECTED}" != "${ACTUAL}" ]; then
echo "proxy tag ${TAG} does not match apps/aether-proxy/Cargo.toml version ${ACTUAL}" >&2
exit 1
fi
build:
needs: preflight
if: always() && (needs.preflight.result == 'success' || needs.preflight.result == 'skipped')
name: ${{ matrix.name }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
include:
- name: linux-amd64
target: x86_64-unknown-linux-gnu
os: ubuntu-latest
use_cross: true
- name: linux-arm64
target: aarch64-unknown-linux-gnu
os: ubuntu-latest
use_cross: true
- name: linux-musl-amd64
target: x86_64-unknown-linux-musl
os: ubuntu-latest
use_cross: true
- name: linux-musl-arm64
target: aarch64-unknown-linux-musl
os: ubuntu-latest
use_cross: true
- name: macos-amd64
target: x86_64-apple-darwin
os: macos-15-intel
use_cross: false
- name: macos-arm64
target: aarch64-apple-darwin
os: macos-15
use_cross: false
- name: windows-amd64
target: x86_64-pc-windows-msvc
os: windows-latest
use_cross: false
steps:
- uses: actions/checkout@v5
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
with:
targets: ${{ matrix.target }}
- name: Ensure Rust target is installed
run: rustup target add ${{ matrix.target }}
- name: Rust cache
uses: Swatinem/rust-cache@v2
with:
workspaces: apps/aether-proxy -> target
key: ${{ matrix.target }}
- name: Install cross
if: matrix.use_cross
uses: taiki-e/install-action@cross
- name: Build
working-directory: apps/aether-proxy
shell: bash
run: |
if [ "${{ matrix.use_cross }}" = "true" ]; then
cross build --release --target ${{ matrix.target }}
else
cargo build --release --target ${{ matrix.target }}
fi
- name: Package (Unix)
if: runner.os != 'Windows'
shell: bash
run: |
cd target/${{ matrix.target }}/release
chmod +x aether-proxy
tar czf ../../../aether-proxy-${{ matrix.name }}.tar.gz aether-proxy
- name: Package (Windows)
if: runner.os == 'Windows'
shell: bash
run: |
cd target/${{ matrix.target }}/release
7z a ../../../aether-proxy-${{ matrix.name }}.zip aether-proxy.exe
- name: Upload artifact
uses: actions/upload-artifact@v5
with:
name: aether-proxy-${{ matrix.name }}
path: |
aether-proxy-*.tar.gz
aether-proxy-*.zip
if-no-files-found: error
release:
needs: build
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/')
steps:
- name: Download all artifacts
uses: actions/download-artifact@v5
with:
merge-multiple: true
path: artifacts
- name: Generate checksums
working-directory: artifacts
run: sha256sum aether-proxy-* > SHA256SUMS.txt
- name: Create GitHub Release
uses: softprops/action-gh-release@v2
with:
name: "${{ github.ref_name }}"
generate_release_notes: true
files: |
artifacts/aether-proxy-*
artifacts/SHA256SUMS.txt
fail_on_unmatched_files: true
update-readme:
needs: release
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/')
steps:
- uses: actions/checkout@v5
with:
ref: aether-rust-pioneer
- name: Update README download links
env:
TAG: ${{ github.ref_name }}
run: |
VERSION="${TAG#proxy-v}"
BASE="https://github.com/fawney19/Aether/releases/download/${TAG}"
if [ -d apps/aether-proxy ]; then
PROXY_DIR="apps/aether-proxy"
else
PROXY_DIR="aether-proxy"
fi
cd "$PROXY_DIR"
TABLE="| Platform | Download |\n|----------|----------|\n"
TABLE+="| Linux x86_64 (GNU) | [aether-proxy-linux-amd64.tar.gz](${BASE}/aether-proxy-linux-amd64.tar.gz) |\n"
TABLE+="| Linux ARM64 (GNU) | [aether-proxy-linux-arm64.tar.gz](${BASE}/aether-proxy-linux-arm64.tar.gz) |\n"
TABLE+="| Linux x86_64 (musl) | [aether-proxy-linux-musl-amd64.tar.gz](${BASE}/aether-proxy-linux-musl-amd64.tar.gz) |\n"
TABLE+="| Linux ARM64 (musl) | [aether-proxy-linux-musl-arm64.tar.gz](${BASE}/aether-proxy-linux-musl-arm64.tar.gz) |\n"
TABLE+="| macOS x86_64 | [aether-proxy-macos-amd64.tar.gz](${BASE}/aether-proxy-macos-amd64.tar.gz) |\n"
TABLE+="| macOS ARM64 | [aether-proxy-macos-arm64.tar.gz](${BASE}/aether-proxy-macos-arm64.tar.gz) |\n"
TABLE+="| Windows x86_64 | [aether-proxy-windows-amd64.zip](${BASE}/aether-proxy-windows-amd64.zip) |"
# Replace content between markers
if grep -q '<!-- DOWNLOAD_TABLE_START -->' README.md; then
awk -v table="$TABLE" '
/<!-- DOWNLOAD_TABLE_START -->/ { print; printf "%s\n", table; skip=1; next }
/<!-- DOWNLOAD_TABLE_END -->/ { skip=0 }
!skip { print }
' README.md > README.tmp && mv README.tmp README.md
fi
- name: Commit and push
run: |
if [ -d apps/aether-proxy ]; then
PROXY_DIR="apps/aether-proxy"
else
PROXY_DIR="aether-proxy"
fi
cd "$PROXY_DIR"
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add README.md
git diff --cached --quiet && exit 0
TAG="${GITHUB_REF#refs/tags/}"
git commit -m "chore(proxy): update download links for ${TAG}"
git push
+263
View File
@@ -0,0 +1,263 @@
name: Build aether-tunnel
on:
push:
tags: ['tunnel-v*']
workflow_dispatch:
permissions:
actions: read
contents: read
concurrency:
group: build-tunnel-${{ github.ref }}
cancel-in-progress: false
jobs:
preflight:
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/')
steps:
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
- name: Ensure tunnel tag matches Cargo version
shell: bash
run: |
TAG="${GITHUB_REF_NAME}"
EXPECTED="${TAG#tunnel-v}"
ACTUAL="$(cargo metadata --manifest-path apps/aether-tunnel/Cargo.toml --locked --no-deps --format-version 1 | jq -r '.packages[] | select(.name == "aether-tunnel") | .version')"
echo "tag version: ${EXPECTED}"
echo "cargo version: ${ACTUAL}"
if [ -z "${ACTUAL}" ]; then
echo "Could not resolve aether-tunnel package version" >&2
exit 1
fi
if [ "${EXPECTED}" != "${ACTUAL}" ]; then
echo "tunnel tag ${TAG} does not match apps/aether-tunnel/Cargo.toml version ${ACTUAL}" >&2
exit 1
fi
build:
needs: preflight
if: always() && (needs.preflight.result == 'success' || needs.preflight.result == 'skipped')
name: ${{ matrix.name }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
include:
- name: linux-amd64
target: x86_64-unknown-linux-gnu
os: ubuntu-latest
use_cross: true
- name: linux-arm64
target: aarch64-unknown-linux-gnu
os: ubuntu-latest
use_cross: true
- name: linux-musl-amd64
target: x86_64-unknown-linux-musl
os: ubuntu-latest
use_cross: true
- name: linux-musl-arm64
target: aarch64-unknown-linux-musl
os: ubuntu-latest
use_cross: true
- name: macos-amd64
target: x86_64-apple-darwin
os: macos-15-intel
use_cross: false
- name: macos-arm64
target: aarch64-apple-darwin
os: macos-15
use_cross: false
- name: windows-amd64
target: x86_64-pc-windows-msvc
os: windows-latest
use_cross: false
steps:
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@4360b52568e2003a75bf9bc1d59f33a8e3fc893c # stable
with:
targets: ${{ matrix.target }}
- name: Ensure Rust target is installed
run: rustup target add ${{ matrix.target }}
- name: Rust cache
uses: Swatinem/rust-cache@49a0bdc70d2e1b713ca9e2869b211fcce03d3c1c # v2
with:
workspaces: apps/aether-tunnel -> target
key: ${{ matrix.target }}
- name: Install cross
if: matrix.use_cross
uses: taiki-e/install-action@1ae7257be536a92d9218a6b343dc6e6ba650f7e1 # cross
- name: Build
working-directory: apps/aether-tunnel
shell: bash
run: |
if [ "${{ matrix.use_cross }}" = "true" ]; then
cross build --release --locked --target ${{ matrix.target }}
else
cargo build --release --locked --target ${{ matrix.target }}
fi
- name: Package (Unix)
if: runner.os != 'Windows'
shell: bash
run: |
cd target/${{ matrix.target }}/release
chmod +x aether-tunnel
tar czf ../../../aether-tunnel-${{ matrix.name }}.tar.gz aether-tunnel
- name: Package (Windows)
if: runner.os == 'Windows'
shell: bash
run: |
cd target/${{ matrix.target }}/release
7z a ../../../aether-tunnel-${{ matrix.name }}.zip aether-tunnel.exe
tar czf ../../../aether-tunnel-${{ matrix.name }}.tar.gz aether-tunnel.exe
- name: Upload artifact
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5
with:
name: aether-tunnel-${{ matrix.name }}
path: |
aether-tunnel-*.tar.gz
aether-tunnel-*.zip
if-no-files-found: error
retention-days: 1
release:
needs: build
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/')
permissions:
actions: read
attestations: write
contents: write
id-token: write
steps:
- name: Download all artifacts
uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5
with:
merge-multiple: true
path: artifacts
- name: Generate checksums
working-directory: artifacts
run: sha256sum aether-tunnel-* > SHA256SUMS.txt
- name: Attest tunnel release provenance
id: attest-release
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-path: |
artifacts/aether-tunnel-*.tar.gz
artifacts/aether-tunnel-*.zip
artifacts/SHA256SUMS.txt
- name: Bundle tunnel release provenance
env:
ATTESTATION_BUNDLE: ${{ steps.attest-release.outputs.bundle-path }}
run: install -m 0644 "${ATTESTATION_BUNDLE}" artifacts/AETHER_TUNNEL_RELEASE_PROVENANCE.sigstore.json
- name: Delete stale draft releases for tag
env:
GH_TOKEN: ${{ github.token }}
RELEASE_TAG: ${{ github.ref_name }}
REPOSITORY: ${{ github.repository }}
shell: bash
run: |
set -euo pipefail
draft_ids="$(gh api "repos/${REPOSITORY}/releases" --paginate --jq '.[] | select(.tag_name == env.RELEASE_TAG and .draft == true) | .id')"
if [[ -z "${draft_ids}" ]]; then
echo "No stale draft releases for ${RELEASE_TAG}"
exit 0
fi
while IFS= read -r release_id; do
[[ -z "${release_id}" ]] && continue
echo "Deleting stale draft release ${release_id} for ${RELEASE_TAG}"
gh api -X DELETE "repos/${REPOSITORY}/releases/${release_id}"
done <<< "${draft_ids}"
- name: Create GitHub Release
uses: softprops/action-gh-release@3bb12739c298aeb8a4eeaf626c5b8d85266b0e65 # v2
with:
name: "${{ github.ref_name }}"
generate_release_notes: true
files: |
artifacts/aether-tunnel-*
artifacts/AETHER_TUNNEL_RELEASE_PROVENANCE.sigstore.json
artifacts/SHA256SUMS.txt
fail_on_unmatched_files: true
update-readme:
needs: release
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/')
permissions:
contents: write
steps:
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
with:
ref: main
- name: Update README download links
env:
TAG: ${{ github.ref_name }}
run: |
VERSION="${TAG#tunnel-v}"
BASE="https://github.com/fawney19/Aether/releases/download/${TAG}"
if [ -d apps/aether-tunnel ]; then
TUNNEL_DIR="apps/aether-tunnel"
else
TUNNEL_DIR="aether-tunnel"
fi
cd "$TUNNEL_DIR"
TABLE="| Platform | Download |\n|----------|----------|\n"
TABLE+="| Linux x86_64 (GNU) | [aether-tunnel-linux-amd64.tar.gz](${BASE}/aether-tunnel-linux-amd64.tar.gz) |\n"
TABLE+="| Linux ARM64 (GNU) | [aether-tunnel-linux-arm64.tar.gz](${BASE}/aether-tunnel-linux-arm64.tar.gz) |\n"
TABLE+="| Linux x86_64 (musl) | [aether-tunnel-linux-musl-amd64.tar.gz](${BASE}/aether-tunnel-linux-musl-amd64.tar.gz) |\n"
TABLE+="| Linux ARM64 (musl) | [aether-tunnel-linux-musl-arm64.tar.gz](${BASE}/aether-tunnel-linux-musl-arm64.tar.gz) |\n"
TABLE+="| macOS x86_64 | [aether-tunnel-macos-amd64.tar.gz](${BASE}/aether-tunnel-macos-amd64.tar.gz) |\n"
TABLE+="| macOS ARM64 | [aether-tunnel-macos-arm64.tar.gz](${BASE}/aether-tunnel-macos-arm64.tar.gz) |\n"
TABLE+="| Windows x86_64 | [aether-tunnel-windows-amd64.zip](${BASE}/aether-tunnel-windows-amd64.zip) |"
# Replace content between markers
if grep -q '<!-- DOWNLOAD_TABLE_START -->' README.md; then
awk -v table="$TABLE" '
/<!-- DOWNLOAD_TABLE_START -->/ { print; printf "%s\n", table; skip=1; next }
/<!-- DOWNLOAD_TABLE_END -->/ { skip=0 }
!skip { print }
' README.md > README.tmp && mv README.tmp README.md
fi
- name: Commit and push
run: |
if [ -d apps/aether-tunnel ]; then
TUNNEL_DIR="apps/aether-tunnel"
else
TUNNEL_DIR="aether-tunnel"
fi
cd "$TUNNEL_DIR"
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add README.md
git diff --cached --quiet && exit 0
TAG="${GITHUB_REF#refs/tags/}"
git commit -m "chore(tunnel): update download links for ${TAG}"
git push
+45 -8
View File
@@ -7,25 +7,59 @@ on:
permissions:
contents: read
pages: write
id-token: write
concurrency:
group: pages
cancel-in-progress: false
jobs:
preflight:
runs-on: ubuntu-latest
outputs:
deploy_pages: ${{ steps.classify.outputs.deploy_pages }}
steps:
- name: Ensure stable Pages release tag
id: classify
shell: bash
run: |
set -euo pipefail
echo "deploy_pages=false" >> "${GITHUB_OUTPUT}"
if [[ "${GITHUB_REF_TYPE}" != "tag" ]]; then
echo "Manual Pages deployment."
echo "deploy_pages=true" >> "${GITHUB_OUTPUT}"
exit 0
fi
tag="${GITHUB_REF_NAME}"
if [[ ! "${tag}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "Skipping Pages deploy for non-stable release tag: ${tag}"
exit 0
fi
echo "deploy_pages=true" >> "${GITHUB_OUTPUT}"
build:
needs: preflight
if: needs.preflight.outputs.deploy_pages == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
- name: Setup Node.js
uses: actions/setup-node@v5
uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: frontend/package-lock.json
cache-dependency-path: |
frontend/package-lock.json
aether-vscodex/web/package-lock.json
- name: Build aether-vscodex web
working-directory: aether-vscodex/web
run: |
npm ci
npm run build
- name: Install dependencies
working-directory: frontend
@@ -41,10 +75,10 @@ jobs:
run: cp frontend/dist/index.html frontend/dist/404.html
- name: Setup Pages
uses: actions/configure-pages@v5
uses: actions/configure-pages@983d7736d9b0ae728b81ab479565c72886d7745b # v5
- name: Upload artifact
uses: actions/upload-pages-artifact@v3
uses: actions/upload-pages-artifact@56afc609e74202658d3ffba0e8f6dda462b719fa # v3
with:
path: frontend/dist
@@ -54,7 +88,10 @@ jobs:
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
needs: build
permissions:
id-token: write
pages: write
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4
+620
View File
@@ -0,0 +1,620 @@
name: Nightly Release
on:
# 02:17 Asia/Shanghai (18:17 UTC) every day.
schedule:
- cron: '17 18 * * *'
workflow_dispatch:
# Checks and builds only need read access. Publishing jobs opt into write access
# below so a failed build cannot modify the existing nightly release.
permissions:
actions: read
contents: read
# A rolling tag and image are shared by scheduled and manually retried runs.
# Keep GitHub Release immutability disabled for this repository: the tag and
# assets intentionally move after each successful daily build.
concurrency:
group: nightly-main
cancel-in-progress: false
env:
CARGO_INCREMENTAL: '0'
CARGO_PROFILE_DEV_DEBUG: '0'
CARGO_PROFILE_TEST_DEBUG: '0'
CARGO_TERM_COLOR: always
RUST_BACKTRACE: '1'
jobs:
source:
name: Resolve main snapshot
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
sha: ${{ steps.snapshot.outputs.sha }}
short_sha: ${{ steps.snapshot.outputs.short_sha }}
date: ${{ steps.snapshot.outputs.date }}
ghcr_image: ${{ steps.snapshot.outputs.ghcr_image }}
steps:
- name: Require main branch
id: snapshot
shell: bash
run: |
set -euo pipefail
if [[ "${GITHUB_REF}" != "refs/heads/main" ]]; then
echo "Nightly releases must run from refs/heads/main (got ${GITHUB_REF})." >&2
exit 1
fi
sha="${GITHUB_SHA}"
# Docker 镜像仓库名必须全小写;GitHub owner 可能保留大写,先统一规范化。
repository_owner="${GITHUB_REPOSITORY%%/*}"
repository_owner="${repository_owner,,}"
echo "sha=${sha}" >> "${GITHUB_OUTPUT}"
echo "short_sha=${sha:0:7}" >> "${GITHUB_OUTPUT}"
echo "date=$(date -u +'%Y-%m-%d')" >> "${GITHUB_OUTPUT}"
echo "ghcr_image=ghcr.io/${repository_owner}/aether" >> "${GITHUB_OUTPUT}"
echo "Building main at ${sha}."
# Keep the scheduled backend coverage in one place so it cannot drift from PR CI.
rust_ci:
name: Rust CI
needs: source
uses: ./.github/workflows/rust-ci.yml
rust_extended:
name: Rust extended checks
needs: source
runs-on: ubuntu-latest
timeout-minutes: 90
steps:
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
with:
ref: ${{ needs.source.outputs.sha }}
- name: Install pinned Rust toolchain
uses: dtolnay/rust-toolchain@4360b52568e2003a75bf9bc1d59f33a8e3fc893c # stable
with:
toolchain: 1.95.0
- name: Show Rust toolchain
run: rustc -Vv
- name: Rust cache
uses: Swatinem/rust-cache@49a0bdc70d2e1b713ca9e2869b211fcce03d3c1c # v2
with:
shared-key: nightly-rust-1.95-${{ runner.os }}
workspaces: . -> target
- name: Setup sccache
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9
- name: Check all workspace targets
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: 'true'
run: cargo check --workspace --all-targets --all-features --locked
- name: Run workspace doctests
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: 'true'
run: cargo test --workspace --all-features --doc --locked
- name: Show sccache stats
if: always()
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: 'true'
run: sccache --show-stats
frontend:
name: Frontend checks and build
needs: source
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
with:
ref: ${{ needs.source.outputs.sha }}
- name: Setup Node.js
uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5
with:
node-version: '22'
cache: npm
cache-dependency-path: |
frontend/package-lock.json
aether-vscodex/web/package-lock.json
# The frontend prebuild synchronizes the embedded VSCodex UI by running
# its build from a separate package. Install that package explicitly so
# vue-tsc can resolve vite/client, vitest/globals, and node types in a
# clean runner.
- name: Install VSCodex web dependencies
working-directory: aether-vscodex/web
run: npm ci
- name: Install dependencies
working-directory: frontend
run: npm ci
- name: Lint
working-directory: frontend
run: npx --no-install eslint .
- name: Type-check
working-directory: frontend
run: npm run type-check
- name: Run unit tests
working-directory: frontend
run: npm run test:run
- name: Build nightly frontend
working-directory: frontend
env:
AETHER_BUILD_VERSION: nightly-${{ needs.source.outputs.short_sha }}
AETHER_VERSION: nightly
run: npm run build
- name: Upload frontend artifact
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5
with:
name: nightly-frontend-dist
path: frontend/dist/
if-no-files-found: error
overwrite: true
retention-days: 7
repository_health:
name: Repository health checks
needs: source
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
with:
ref: ${{ needs.source.outputs.sha }}
- name: Setup Node.js
uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5
with:
node-version: '22'
- name: Check generated format coverage matrix
run: python3 docs/api/generate_format_field_coverage.py --check
- name: Test pressure report checker
run: node --test tools/pressure/check_gateway_stage_report.test.js
checks:
name: Nightly check gate
runs-on: ubuntu-latest
if: ${{ always() }}
needs:
- source
- rust_ci
- rust_extended
- frontend
- repository_health
steps:
- name: Verify check jobs
shell: bash
run: |
set -euo pipefail
failed=0
echo "source=${{ needs.source.result }}"
echo "rust_ci=${{ needs.rust_ci.result }}"
echo "rust_extended=${{ needs.rust_extended.result }}"
echo "frontend=${{ needs.frontend.result }}"
echo "repository_health=${{ needs.repository_health.result }}"
for result in \
"${{ needs.source.result }}" \
"${{ needs.rust_ci.result }}" \
"${{ needs.rust_extended.result }}" \
"${{ needs.frontend.result }}" \
"${{ needs.repository_health.result }}"; do
if [[ "${result}" != "success" ]]; then
failed=1
fi
done
if [[ "${failed}" -ne 0 ]]; then
echo 'One or more nightly checks failed or were cancelled.' >&2
exit 1
fi
build:
name: Build ${{ matrix.name }}
needs: [source, checks]
if: ${{ needs.checks.result == 'success' }}
runs-on: ${{ matrix.os }}
timeout-minutes: 120
strategy:
fail-fast: false
matrix:
include:
- name: linux-amd64
target: x86_64-unknown-linux-musl
platform: linux
arch: amd64
os: ubuntu-latest
use_cross: true
- name: linux-arm64
target: aarch64-unknown-linux-musl
platform: linux
arch: arm64
os: ubuntu-latest
use_cross: true
- name: macos-amd64
target: x86_64-apple-darwin
platform: macos
arch: amd64
os: macos-15-intel
use_cross: false
- name: macos-arm64
target: aarch64-apple-darwin
platform: macos
arch: arm64
os: macos-15
use_cross: false
steps:
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
with:
ref: ${{ needs.source.outputs.sha }}
- name: Install pinned Rust toolchain
uses: dtolnay/rust-toolchain@4360b52568e2003a75bf9bc1d59f33a8e3fc893c # stable
with:
toolchain: 1.95.0
targets: ${{ matrix.target }}
- name: Rust cache
uses: Swatinem/rust-cache@49a0bdc70d2e1b713ca9e2869b211fcce03d3c1c # v2
with:
shared-key: nightly-release-${{ matrix.target }}
workspaces: . -> target
- name: Install cross
if: matrix.use_cross
uses: taiki-e/install-action@1ae7257be536a92d9218a6b343dc6e6ba650f7e1 # cross
- name: Build release binary
env:
AETHER_BUILD_VERSION: nightly-${{ needs.source.outputs.short_sha }}
AETHER_VERSION: nightly
AETHER_BUILD_TYPE: release
CARGO_TERM_COLOR: always
shell: bash
run: |
if [[ "${{ matrix.use_cross }}" == "true" ]]; then
cross build --release --locked -p aether-gateway --target "${{ matrix.target }}"
else
cargo build --release --locked -p aether-gateway --target "${{ matrix.target }}"
fi
- name: Upload binary artifact
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5
with:
name: nightly-gateway-${{ matrix.platform }}-${{ matrix.arch }}
path: target/${{ matrix.target }}/release/aether-gateway
if-no-files-found: error
overwrite: true
retention-days: 7
docker:
name: Publish nightly GHCR image
needs: [source, checks, build]
if: ${{ needs.checks.result == 'success' && needs.build.result == 'success' }}
runs-on: ubuntu-latest
env:
GHCR_IMAGE: ${{ needs.source.outputs.ghcr_image }}
permissions:
actions: read
contents: read
packages: write
steps:
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
with:
ref: ${{ needs.source.outputs.sha }}
- name: Download Linux binaries and frontend
uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5
with:
pattern: nightly-*
path: artifacts
merge-multiple: false
- name: Prepare Docker build context
shell: bash
run: |
set -euo pipefail
mkdir -p dist/frontend
cp artifacts/nightly-gateway-linux-amd64/aether-gateway dist/aether-gateway-amd64
cp artifacts/nightly-gateway-linux-arm64/aether-gateway dist/aether-gateway-arm64
chmod 0755 dist/aether-gateway-amd64 dist/aether-gateway-arm64
cp -R artifacts/nightly-frontend-dist/. dist/frontend/
- name: Set up QEMU
uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6 # v3
- name: Log in to GHCR
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push nightly image
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6
with:
context: .
file: ./Dockerfile.app
push: true
platforms: linux/amd64,linux/arm64
tags: |
${{ env.GHCR_IMAGE }}:nightly
${{ env.GHCR_IMAGE }}:nightly-${{ needs.source.outputs.sha }}
labels: |
org.opencontainers.image.title=Aether
org.opencontainers.image.version=nightly
org.opencontainers.image.revision=${{ needs.source.outputs.sha }}
org.opencontainers.image.source=https://github.com/${{ github.repository }}
package:
name: Package nightly archives
needs: [source, checks, build]
if: ${{ needs.checks.result == 'success' && needs.build.result == 'success' }}
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
steps:
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
with:
ref: ${{ needs.source.outputs.sha }}
- name: Download nightly artifacts
uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5
with:
pattern: nightly-*
path: artifacts
merge-multiple: false
- name: Build nightly release packages
shell: bash
env:
SOURCE_REF: ${{ needs.source.outputs.sha }}
run: |
set -euo pipefail
VERSION="nightly"
mkdir -p package release-assets
for platform in linux macos; do
for arch in amd64 arm64; do
bundle="aether-${VERSION}-${platform}-${arch}"
root="package/${bundle}"
mkdir -p "${root}/bin" "${root}/frontend"
install -m 0755 \
"artifacts/nightly-gateway-${platform}-${arch}/aether-gateway" \
"${root}/bin/aether-gateway"
cp -R artifacts/nightly-frontend-dist/. "${root}/frontend/"
sed \
-e "s/^SOURCE_REF=\"\${AETHER_SOURCE_REF:-main}\"/SOURCE_REF=\"\${AETHER_SOURCE_REF:-${SOURCE_REF}}\"/" \
-e "s/^VERSION=\"\${AETHER_VERSION:-}\"/VERSION=\"\${AETHER_VERSION:-${VERSION}}\"/" \
install.sh > "${root}/install.sh"
chmod 0755 "${root}/install.sh"
install -m 0755 update.sh "${root}/update.sh"
install -m 0644 docker-compose.yml "${root}/docker-compose.yml"
install -m 0644 docker-compose.single-node.yml "${root}/docker-compose.single-node.yml"
install -m 0644 .env.example "${root}/.env.example"
install -m 0755 generate_keys.sh "${root}/generate_keys.sh"
install -m 0644 README.md "${root}/README.md"
install -m 0644 LICENSE "${root}/LICENSE"
tar -C package -czf "release-assets/${bundle}.tar.gz" "${bundle}"
done
done
sed \
-e "s/^SOURCE_REF=\"\${AETHER_SOURCE_REF:-main}\"/SOURCE_REF=\"\${AETHER_SOURCE_REF:-${SOURCE_REF}}\"/" \
-e "s/^VERSION=\"\${AETHER_VERSION:-}\"/VERSION=\"\${AETHER_VERSION:-${VERSION}}\"/" \
install.sh > release-assets/install.sh
chmod 0755 release-assets/install.sh
(cd release-assets && sha256sum *.tar.gz > SHA256SUMS)
test "$(find release-assets -maxdepth 1 -name '*.tar.gz' | wc -l)" -eq 4
test "$(wc -l < release-assets/SHA256SUMS)" -eq 4
(cd release-assets && sha256sum -c SHA256SUMS)
for archive in release-assets/*.tar.gz; do
tar -tzf "${archive}" >/dev/null
done
- name: Upload nightly package artifact
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5
with:
name: nightly-release-assets
path: release-assets/*
if-no-files-found: error
overwrite: true
retention-days: 7
github_release:
name: Publish nightly GitHub Release
needs: [source, checks, docker, package]
if: ${{ needs.checks.result == 'success' && needs.docker.result == 'success' && needs.package.result == 'success' }}
runs-on: ubuntu-latest
permissions:
actions: read
contents: write
steps:
- name: Download nightly package artifact
uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5
with:
name: nightly-release-assets
path: release-assets
- name: Update rolling nightly release
shell: bash
env:
GH_TOKEN: ${{ github.token }}
REPOSITORY: ${{ github.repository }}
RELEASE_TAG: nightly
SOURCE_SHA: ${{ needs.source.outputs.sha }}
SOURCE_SHORT_SHA: ${{ needs.source.outputs.short_sha }}
RELEASE_DATE: ${{ needs.source.outputs.date }}
GHCR_IMAGE: ${{ needs.source.outputs.ghcr_image }}
run: |
set -euo pipefail
release_title="Aether Nightly ${RELEASE_DATE} (${SOURCE_SHORT_SHA})"
notes_file="${RUNNER_TEMP}/nightly-release-notes.md"
cat > "${notes_file}" <<EOF
## Aether nightly
This rolling prerelease was built from [main commit ${SOURCE_SHORT_SHA}](https://github.com/${REPOSITORY}/commit/${SOURCE_SHA}).
- Source branch: main
- Source commit: ${SOURCE_SHA}
- Build date (UTC): ${RELEASE_DATE}
- Container image: ${GHCR_IMAGE}:nightly
- Commit image: ${GHCR_IMAGE}:nightly-${SOURCE_SHA}
The nightly tag and assets are replaced by the next successful daily build.
EOF
# Create a draft on the first run. Later runs repair the same rolling
# release on retry if any upload or metadata update is interrupted.
if ! gh release view "${RELEASE_TAG}" --repo "${REPOSITORY}" >/dev/null 2>&1; then
gh release create "${RELEASE_TAG}" \
--repo "${REPOSITORY}" \
--draft \
--prerelease \
--latest=false \
--target "${SOURCE_SHA}" \
--title "${release_title}" \
--notes-file "${notes_file}"
fi
# Upload archives first, then the checksum/installer metadata. This
# keeps a failed upload from leaving a checksum that describes files
# which have not reached the Release yet.
gh release upload "${RELEASE_TAG}" release-assets/*.tar.gz \
--repo "${REPOSITORY}" \
--clobber
gh release upload "${RELEASE_TAG}" \
release-assets/SHA256SUMS \
release-assets/install.sh \
--repo "${REPOSITORY}" \
--clobber
# target_commitish does not move an existing git tag. Move the ref
# only after the complete asset set is available.
if gh api "repos/${REPOSITORY}/git/ref/tags/${RELEASE_TAG}" >/dev/null 2>&1; then
gh api -X PATCH "repos/${REPOSITORY}/git/refs/tags/${RELEASE_TAG}" \
-f "sha=${SOURCE_SHA}" \
-F 'force=true' >/dev/null
else
gh api -X POST "repos/${REPOSITORY}/git/refs" \
-f "ref=refs/tags/${RELEASE_TAG}" \
-f "sha=${SOURCE_SHA}" >/dev/null
fi
gh release edit "${RELEASE_TAG}" \
--repo "${REPOSITORY}" \
--draft=false \
--prerelease \
--latest=false \
--target "${SOURCE_SHA}" \
--title "${release_title}" \
--notes-file "${notes_file}"
expected_assets=(
aether-nightly-linux-amd64.tar.gz
aether-nightly-linux-arm64.tar.gz
aether-nightly-macos-amd64.tar.gz
aether-nightly-macos-arm64.tar.gz
SHA256SUMS
install.sh
)
asset_names="$(gh release view "${RELEASE_TAG}" --repo "${REPOSITORY}" --json assets --jq '.assets[].name')"
for expected_asset in "${expected_assets[@]}"; do
if ! grep -Fxq "${expected_asset}" <<<"${asset_names}"; then
echo "Published release is missing asset ${expected_asset}." >&2
exit 1
fi
done
resolved_sha=""
for attempt in {1..10}; do
resolved_sha="$(gh api "repos/${REPOSITORY}/commits/${RELEASE_TAG}" --jq '.sha' 2>/dev/null || true)"
if [[ "${resolved_sha}" == "${SOURCE_SHA}" ]]; then
break
fi
sleep 2
done
if [[ "${resolved_sha}" != "${SOURCE_SHA}" ]]; then
echo "nightly tag resolved to ${resolved_sha}, expected ${SOURCE_SHA}." >&2
exit 1
fi
release_state="$(gh release view "${RELEASE_TAG}" --repo "${REPOSITORY}" --json isDraft,isPrerelease --jq '[.isDraft, .isPrerelease] | @tsv')"
if [[ "${release_state}" != $'false\ttrue' ]]; then
echo "nightly release has unexpected state: ${release_state}" >&2
exit 1
fi
echo "Published ${RELEASE_TAG} for ${SOURCE_SHA}."
summary:
name: Nightly summary
runs-on: ubuntu-latest
if: ${{ always() }}
needs:
- source
- rust_ci
- rust_extended
- frontend
- repository_health
- checks
- build
- docker
- package
- github_release
steps:
- name: Verify nightly pipeline
shell: bash
run: |
set -euo pipefail
failed=0
for entry in \
"source=${{ needs.source.result }}" \
"rust_ci=${{ needs.rust_ci.result }}" \
"rust_extended=${{ needs.rust_extended.result }}" \
"frontend=${{ needs.frontend.result }}" \
"repository_health=${{ needs.repository_health.result }}" \
"checks=${{ needs.checks.result }}" \
"build=${{ needs.build.result }}" \
"docker=${{ needs.docker.result }}" \
"package=${{ needs.package.result }}" \
"github_release=${{ needs.github_release.result }}"; do
echo "${entry}"
if [[ "${entry#*=}" != "success" ]]; then
failed=1
fi
done
if [[ "${failed}" -ne 0 ]]; then
echo 'Nightly pipeline did not publish a new release.' >&2
exit 1
fi
+298 -76
View File
@@ -6,8 +6,12 @@ on:
workflow_dispatch:
permissions:
contents: write
packages: write
actions: read
contents: read
concurrency:
group: release-aether-${{ github.ref }}
cancel-in-progress: false
env:
REGISTRY: ghcr.io
@@ -15,18 +19,73 @@ env:
DOCKERHUB_IMAGE: fawney19/aether
jobs:
preflight:
name: Release preflight
runs-on: ubuntu-latest
outputs:
publish: ${{ steps.classify.outputs.publish }}
version_tag: ${{ steps.classify.outputs.version_tag }}
prerelease: ${{ steps.classify.outputs.prerelease }}
make_latest: ${{ steps.classify.outputs.make_latest }}
steps:
- name: Classify release tag
id: classify
shell: bash
run: |
set -euo pipefail
echo "publish=false" >> "${GITHUB_OUTPUT}"
echo "version_tag=" >> "${GITHUB_OUTPUT}"
echo "prerelease=false" >> "${GITHUB_OUTPUT}"
echo "make_latest=false" >> "${GITHUB_OUTPUT}"
if [[ "${GITHUB_REF_TYPE}" != "tag" ]]; then
echo "Manual release build; publish jobs will be skipped."
exit 0
fi
tag="${GITHUB_REF_NAME}"
if [[ ! "${tag}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(-(beta|rc)\.[0-9]+)?$ ]]; then
echo "Unsupported release tag: ${tag}" >&2
echo "Expected vX.Y.Z, vX.Y.Z-beta.N, or vX.Y.Z-rc.N." >&2
exit 1
fi
echo "version_tag=${tag}" >> "${GITHUB_OUTPUT}"
if [[ "${tag}" == *-* ]]; then
echo "prerelease=true" >> "${GITHUB_OUTPUT}"
else
echo "make_latest=true" >> "${GITHUB_OUTPUT}"
fi
if [[ "${GITHUB_EVENT_NAME}" == "push" ]]; then
echo "publish=true" >> "${GITHUB_OUTPUT}"
else
echo "Manual release build for ${tag}; publish jobs will be skipped."
fi
frontend:
name: Build frontend
needs: preflight
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
- name: Setup Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: 22
cache: npm
cache-dependency-path: frontend/package-lock.json
cache-dependency-path: |
frontend/package-lock.json
aether-vscodex/web/package-lock.json
- name: Build aether-vscodex web
working-directory: aether-vscodex/web
run: |
npm ci
npm run build
- name: Install & build
working-directory: frontend
@@ -35,114 +94,215 @@ jobs:
npm run build
- name: Upload frontend artifact
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5
with:
name: frontend-dist
path: frontend/dist/
if-no-files-found: error
retention-days: 1
vscodex:
name: Build VS Code Codex extension
needs: preflight
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
- name: Setup Node.js
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: 22
cache: npm
cache-dependency-path: |
aether-vscodex/package-lock.json
aether-vscodex/web/package-lock.json
aether-vscodex/vscode-extension/package-lock.json
- name: Install module test dependencies
working-directory: aether-vscodex
run: npm ci
- name: Build the embedded Web UI
working-directory: aether-vscodex/web
run: |
npm ci
npm run build
- name: Install extension dependencies
working-directory: aether-vscodex/vscode-extension
run: npm ci
- name: Check and compile the extension
working-directory: aether-vscodex/vscode-extension
run: |
npm run check
npm run build
- name: Run module tests
working-directory: aether-vscodex
run: npm test
- name: Run Web UI tests
working-directory: aether-vscodex/web
run: npm test
- name: Package VSIX
working-directory: aether-vscodex/vscode-extension
shell: bash
run: |
set -euo pipefail
version="$(node -p "require('./package.json').version")"
npx --yes @vscode/vsce package --no-update-package-json --allow-missing-repository
source_vsix="codex-remote-collab-${version}.vsix"
test -f "${source_vsix}"
mv "${source_vsix}" "aether-vscodex-${version}.vsix"
unzip -l "aether-vscodex-${version}.vsix" | grep 'extension/node_modules/ws/index.js' >/dev/null
- name: Upload VSIX artifact
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5
with:
name: aether-vscodex-vsix
path: aether-vscodex/vscode-extension/aether-vscodex-*.vsix
if-no-files-found: error
retention-days: 7
build:
name: Build ${{ matrix.name }}
runs-on: ubuntu-latest
needs: preflight
runs-on: ${{ matrix.os }}
strategy:
fail-fast: true
matrix:
include:
- name: linux-amd64
target: x86_64-unknown-linux-musl
platform: linux
arch: amd64
os: ubuntu-latest
use_cross: true
- name: linux-arm64
target: aarch64-unknown-linux-musl
platform: linux
arch: arm64
os: ubuntu-latest
use_cross: true
- name: macos-amd64
target: x86_64-apple-darwin
platform: macos
arch: amd64
os: macos-15-intel
use_cross: false
- name: macos-arm64
target: aarch64-apple-darwin
platform: macos
arch: arm64
os: macos-15
use_cross: false
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
uses: dtolnay/rust-toolchain@4360b52568e2003a75bf9bc1d59f33a8e3fc893c # stable
with:
targets: ${{ matrix.target }}
- name: Rust cache
uses: Swatinem/rust-cache@v2
uses: Swatinem/rust-cache@49a0bdc70d2e1b713ca9e2869b211fcce03d3c1c # v2
with:
shared-key: release-${{ matrix.target }}
workspaces: . -> target
- name: Install cross
uses: taiki-e/install-action@cross
if: matrix.use_cross
uses: taiki-e/install-action@1ae7257be536a92d9218a6b343dc6e6ba650f7e1 # cross
- name: Build
env:
AETHER_VERSION: ${{ startsWith(github.ref, 'refs/tags/v') && github.ref_name || '' }}
AETHER_VERSION: ${{ needs.preflight.outputs.version_tag }}
AETHER_BUILD_TYPE: release
CARGO_TERM_COLOR: always
run: cross build --release --locked -p aether-gateway --target ${{ matrix.target }}
shell: bash
run: |
if [[ "${{ matrix.use_cross }}" == "true" ]]; then
cross build --release --locked -p aether-gateway --target ${{ matrix.target }}
else
cargo build --release --locked -p aether-gateway --target ${{ matrix.target }}
fi
- name: Upload binary artifact
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5
with:
name: aether-gateway-${{ matrix.arch }}
name: aether-gateway-${{ matrix.platform }}-${{ matrix.arch }}
path: target/${{ matrix.target }}/release/aether-gateway
if-no-files-found: error
retention-days: 1
docker:
name: Docker multi-arch
needs: [frontend, build]
needs: [preflight, frontend, build]
if: needs.preflight.outputs.publish == 'true'
runs-on: ubuntu-latest
permissions:
actions: read
attestations: write
contents: read
id-token: write
packages: write
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
- name: Download all artifacts
uses: actions/download-artifact@v5
uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5
with:
path: artifacts
- name: Prepare dist layout
run: |
mkdir -p dist
cp artifacts/aether-gateway-amd64/aether-gateway dist/aether-gateway-amd64
cp artifacts/aether-gateway-arm64/aether-gateway dist/aether-gateway-arm64
cp artifacts/aether-gateway-linux-amd64/aether-gateway dist/aether-gateway-amd64
cp artifacts/aether-gateway-linux-arm64/aether-gateway dist/aether-gateway-arm64
chmod +x dist/aether-gateway-amd64 dist/aether-gateway-arm64
cp -r artifacts/frontend-dist dist/frontend
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3
- name: Log in to GHCR
uses: docker/login-action@v3
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Log in to Docker Hub
uses: docker/login-action@v3
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
uses: docker/metadata-action@c299e40c65443455700f0fdfc63efafe5b349051 # v5
with:
images: |
${{ env.REGISTRY }}/${{ env.GHCR_IMAGE }}
docker.io/${{ env.DOCKERHUB_IMAGE }}
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=raw,value=pre,enable=${{ contains(github.ref, '-') }}
type=raw,value=fix,enable=${{ contains(github.ref, '-fix') }}
type=semver,pattern={{major}}.{{minor}},enable=${{ needs.preflight.outputs.make_latest == 'true' }}
type=raw,value=latest,enable=${{ needs.preflight.outputs.make_latest == 'true' }}
type=raw,value=beta,enable=${{ contains(github.ref_name, '-beta.') }}
type=raw,value=rc,enable=${{ contains(github.ref_name, '-rc.') }}
type=sha,prefix=
flavor: |
latest=auto
latest=false
- name: Build and push
uses: docker/build-push-action@v6
id: push
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6
with:
context: .
file: ./Dockerfile.app
@@ -151,15 +311,36 @@ jobs:
labels: ${{ steps.meta.outputs.labels }}
platforms: linux/amd64,linux/arm64
- name: Attest GHCR image provenance
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-name: ${{ env.REGISTRY }}/${{ env.GHCR_IMAGE }}
subject-digest: ${{ steps.push.outputs.digest }}
push-to-registry: true
create-storage-record: false
- name: Attest Docker Hub image provenance
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-name: docker.io/${{ env.DOCKERHUB_IMAGE }}
subject-digest: ${{ steps.push.outputs.digest }}
push-to-registry: true
create-storage-record: false
package:
name: Release tarballs
needs: [frontend, build]
needs: [preflight, frontend, build]
runs-on: ubuntu-latest
permissions:
actions: read
attestations: write
contents: read
id-token: write
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
- name: Download all artifacts
uses: actions/download-artifact@v5
uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5
with:
path: artifacts
@@ -168,37 +349,65 @@ jobs:
set -euo pipefail
if [[ "${GITHUB_REF_TYPE}" == "tag" ]]; then
VERSION="${GITHUB_REF_NAME}"
SOURCE_REF="${GITHUB_REF_NAME}"
else
VERSION="snapshot-${GITHUB_SHA::7}"
SOURCE_REF="${GITHUB_SHA}"
fi
mkdir -p package release-assets
for arch in amd64 arm64; do
bundle="aether-${VERSION}-linux-${arch}"
root="package/${bundle}"
mkdir -p \
"${root}/bin" \
"${root}/frontend"
for platform in linux macos; do
for arch in amd64 arm64; do
bundle="aether-${VERSION}-${platform}-${arch}"
root="package/${bundle}"
mkdir -p \
"${root}/bin" \
"${root}/frontend"
install -m 0755 "artifacts/aether-gateway-${arch}/aether-gateway" "${root}/bin/aether-gateway"
cp -R artifacts/frontend-dist/. "${root}/frontend/"
sed "s/^VERSION=\"\${AETHER_VERSION:-}\"/VERSION=\"\${AETHER_VERSION:-${VERSION}}\"/" install.sh > "${root}/install.sh"
chmod 0755 "${root}/install.sh"
install -m 0644 docker-compose.yml "${root}/docker-compose.yml"
install -m 0644 .env.example "${root}/.env.example"
install -m 0755 generate_keys.sh "${root}/generate_keys.sh"
install -m 0644 README.md "${root}/README.md"
install -m 0644 LICENSE "${root}/LICENSE"
install -m 0755 "artifacts/aether-gateway-${platform}-${arch}/aether-gateway" "${root}/bin/aether-gateway"
cp -R artifacts/frontend-dist/. "${root}/frontend/"
sed \
-e "s/^SOURCE_REF=\"\${AETHER_SOURCE_REF:-main}\"/SOURCE_REF=\"\${AETHER_SOURCE_REF:-${SOURCE_REF}}\"/" \
-e "s/^VERSION=\"\${AETHER_VERSION:-}\"/VERSION=\"\${AETHER_VERSION:-${VERSION}}\"/" \
install.sh > "${root}/install.sh"
chmod 0755 "${root}/install.sh"
install -m 0755 update.sh "${root}/update.sh"
install -m 0644 docker-compose.yml "${root}/docker-compose.yml"
install -m 0644 docker-compose.single-node.yml "${root}/docker-compose.single-node.yml"
install -m 0644 .env.example "${root}/.env.example"
install -m 0755 generate_keys.sh "${root}/generate_keys.sh"
install -m 0644 README.md "${root}/README.md"
install -m 0644 LICENSE "${root}/LICENSE"
tar -C package -czf "release-assets/${bundle}.tar.gz" "${bundle}"
tar -C package -czf "release-assets/${bundle}.tar.gz" "${bundle}"
done
done
sed "s/^VERSION=\"\${AETHER_VERSION:-}\"/VERSION=\"\${AETHER_VERSION:-${VERSION}}\"/" install.sh > release-assets/install.sh
sed \
-e "s/^SOURCE_REF=\"\${AETHER_SOURCE_REF:-main}\"/SOURCE_REF=\"\${AETHER_SOURCE_REF:-${SOURCE_REF}}\"/" \
-e "s/^VERSION=\"\${AETHER_VERSION:-}\"/VERSION=\"\${AETHER_VERSION:-${VERSION}}\"/" \
install.sh > release-assets/install.sh
chmod +x release-assets/install.sh
(cd release-assets && sha256sum *.tar.gz > SHA256SUMS)
- name: Attest release package provenance
id: attest-release
if: needs.preflight.outputs.publish == 'true'
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-path: |
release-assets/*.tar.gz
release-assets/install.sh
release-assets/SHA256SUMS
- name: Bundle release package provenance
if: needs.preflight.outputs.publish == 'true'
env:
ATTESTATION_BUNDLE: ${{ steps.attest-release.outputs.bundle-path }}
run: install -m 0644 "${ATTESTATION_BUNDLE}" release-assets/AETHER_RELEASE_PROVENANCE.sigstore.json
- name: Upload release package artifact
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5
with:
name: release-assets
path: release-assets/*
@@ -207,42 +416,55 @@ jobs:
github-release:
name: GitHub Release assets
needs: [docker, package]
if: startsWith(github.ref, 'refs/tags/')
needs: [preflight, docker, package, vscodex]
if: needs.preflight.outputs.publish == 'true'
runs-on: ubuntu-latest
permissions:
actions: read
contents: write
steps:
- name: Classify release tag
id: release_tag
shell: bash
run: |
set -euo pipefail
tag="${GITHUB_REF_NAME}"
if [[ ! "${tag}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z][0-9A-Za-z.-]*)?$ ]]; then
echo "Unsupported release tag: ${tag}" >&2
exit 1
fi
if [[ "${tag}" == *-* ]]; then
echo "prerelease=true" >> "${GITHUB_OUTPUT}"
echo "make_latest=false" >> "${GITHUB_OUTPUT}"
else
echo "prerelease=false" >> "${GITHUB_OUTPUT}"
echo "make_latest=true" >> "${GITHUB_OUTPUT}"
fi
- name: Download release package artifact
uses: actions/download-artifact@v5
uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5
with:
name: release-assets
path: release-assets
- name: Download VSIX artifact
uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5
with:
name: aether-vscodex-vsix
path: release-assets
- name: Delete stale draft releases for tag
env:
GH_TOKEN: ${{ github.token }}
RELEASE_TAG: ${{ github.ref_name }}
REPOSITORY: ${{ github.repository }}
shell: bash
run: |
set -euo pipefail
draft_ids="$(gh api "repos/${REPOSITORY}/releases" --paginate --jq '.[] | select(.tag_name == env.RELEASE_TAG and .draft == true) | .id')"
if [[ -z "${draft_ids}" ]]; then
echo "No stale draft releases for ${RELEASE_TAG}"
exit 0
fi
while IFS= read -r release_id; do
[[ -z "${release_id}" ]] && continue
echo "Deleting stale draft release ${release_id} for ${RELEASE_TAG}"
gh api -X DELETE "repos/${REPOSITORY}/releases/${release_id}"
done <<< "${draft_ids}"
- name: Publish GitHub Release assets
uses: softprops/action-gh-release@v2
uses: softprops/action-gh-release@3bb12739c298aeb8a4eeaf626c5b8d85266b0e65 # v2
with:
generate_release_notes: true
prerelease: ${{ steps.release_tag.outputs.prerelease }}
make_latest: ${{ steps.release_tag.outputs.make_latest }}
prerelease: ${{ needs.preflight.outputs.prerelease }}
make_latest: ${{ needs.preflight.outputs.make_latest }}
files: |
release-assets/*.tar.gz
release-assets/AETHER_RELEASE_PROVENANCE.sigstore.json
release-assets/SHA256SUMS
release-assets/install.sh
release-assets/*.vsix
+293 -211
View File
@@ -1,47 +1,113 @@
name: Rust CI
on:
workflow_call:
push:
branches:
- master
- aether-rust-pioneer
- main
paths:
- "Cargo.toml"
- "Cargo.lock"
- "crates/**"
- "apps/**"
- "install.sh"
- "deploy.sh"
- "update.sh"
- "generate_keys.sh"
- ".env.example"
- "README.md"
- "Dockerfile.app"
- "docker-compose.yml"
- "docker-compose.single-node.yml"
- "docker-compose.local.yml"
- "docker-compose.release-local.yml"
- "tests/compose_database_config_test.py"
- "tests/install_*_test.sh"
- "tests/deploy_*_test.sh"
- "tests/update_*_test.sh"
- "tests/release_supply_chain_test.sh"
- "tests/tunnel_installer_config_security_test.sh"
- ".github/workflows/build-tunnel.yml"
- ".github/workflows/deploy-pages.yml"
- ".github/workflows/release.yml"
- ".github/workflows/rust-ci.yml"
- ".github/workflows/nightly.yml"
pull_request:
paths:
- "Cargo.toml"
- "Cargo.lock"
- "crates/**"
- "apps/**"
- "install.sh"
- "deploy.sh"
- "update.sh"
- "generate_keys.sh"
- ".env.example"
- "README.md"
- "Dockerfile.app"
- "docker-compose.yml"
- "docker-compose.single-node.yml"
- "docker-compose.local.yml"
- "docker-compose.release-local.yml"
- "tests/compose_database_config_test.py"
- "tests/install_*_test.sh"
- "tests/deploy_*_test.sh"
- "tests/update_*_test.sh"
- "tests/release_supply_chain_test.sh"
- "tests/tunnel_installer_config_security_test.sh"
- ".github/workflows/build-tunnel.yml"
- ".github/workflows/deploy-pages.yml"
- ".github/workflows/release.yml"
- ".github/workflows/rust-ci.yml"
- ".github/workflows/nightly.yml"
concurrency:
group: rust-ci-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
group: rust-ci-${{ github.event_name }}-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
CARGO_INCREMENTAL: 0
CARGO_PROFILE_DEV_DEBUG: 0
CARGO_PROFILE_TEST_DEBUG: 0
CARGO_TERM_COLOR: always
jobs:
shell_security:
name: Shell security fixtures
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
- name: Run installer and supply-chain fixtures
shell: bash
run: |
python3 tests/compose_database_config_test.py
bash tests/deploy_state_safety_test.sh
bash tests/install_archive_safety_test.sh
bash tests/install_container_runtime_security_test.sh
bash tests/install_current_release_link_test.sh
bash tests/install_local_bundle_safety_test.sh
bash tests/install_privileged_write_safety_test.sh
bash tests/install_source_trust_test.sh
bash tests/release_supply_chain_test.sh
bash tests/update_compose_safety_test.sh
bash tests/tunnel_installer_config_security_test.sh
fmt:
name: Format
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
- name: Show Rust toolchain
run: rustup show active-toolchain
- name: Install rustfmt component
run: rustup component add rustfmt
uses: dtolnay/rust-toolchain@4360b52568e2003a75bf9bc1d59f33a8e3fc893c # stable
with:
toolchain: 1.95.0
components: rustfmt
- name: Format
run: cargo fmt --all --check
@@ -50,31 +116,28 @@ jobs:
name: Clippy (Gateway)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
- name: Show Rust toolchain
run: rustup show active-toolchain
- name: Install clippy component
run: rustup component add clippy
uses: dtolnay/rust-toolchain@4360b52568e2003a75bf9bc1d59f33a8e3fc893c # stable
with:
toolchain: 1.95.0
components: clippy
- name: Rust cache
uses: Swatinem/rust-cache@v2
uses: Swatinem/rust-cache@49a0bdc70d2e1b713ca9e2869b211fcce03d3c1c # v2
with:
shared-key: rust-ci-${{ runner.os }}
workspaces: . -> target
- name: Setup sccache
uses: mozilla-actions/[email protected]
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9
- name: Clippy
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
run: cargo clippy -p aether-gateway --all-targets -- -D warnings
run: cargo clippy -p aether-gateway --lib --bins --examples -- -D warnings
- name: Show sccache stats
if: always()
@@ -87,25 +150,22 @@ jobs:
name: Clippy (Data)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
- name: Show Rust toolchain
run: rustup show active-toolchain
- name: Install clippy component
run: rustup component add clippy
uses: dtolnay/rust-toolchain@4360b52568e2003a75bf9bc1d59f33a8e3fc893c # stable
with:
toolchain: 1.95.0
components: clippy
- name: Rust cache
uses: Swatinem/rust-cache@v2
uses: Swatinem/rust-cache@49a0bdc70d2e1b713ca9e2869b211fcce03d3c1c # v2
with:
shared-key: rust-ci-${{ runner.os }}
workspaces: . -> target
- name: Setup sccache
uses: mozilla-actions/[email protected]
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9
- name: Clippy
env:
@@ -124,31 +184,28 @@ jobs:
name: Clippy (Workspace Rest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
- name: Show Rust toolchain
run: rustup show active-toolchain
- name: Install clippy component
run: rustup component add clippy
uses: dtolnay/rust-toolchain@4360b52568e2003a75bf9bc1d59f33a8e3fc893c # stable
with:
toolchain: 1.95.0
components: clippy
- name: Rust cache
uses: Swatinem/rust-cache@v2
uses: Swatinem/rust-cache@49a0bdc70d2e1b713ca9e2869b211fcce03d3c1c # v2
with:
shared-key: rust-ci-${{ runner.os }}
workspaces: . -> target
- name: Setup sccache
uses: mozilla-actions/[email protected]
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9
- name: Clippy
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
run: cargo clippy --workspace --exclude aether-gateway --exclude aether-data --all-targets -- -D warnings
run: cargo clippy --workspace --exclude aether-gateway --exclude aether-data --exclude aether-integration-tests --all-targets -- -D warnings
- name: Show sccache stats
if: always()
@@ -179,28 +236,47 @@ jobs:
name: Test (Gateway)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
uses: dtolnay/rust-toolchain@4360b52568e2003a75bf9bc1d59f33a8e3fc893c # stable
- name: Show Rust toolchain
run: rustup show active-toolchain
- name: Rust cache
uses: Swatinem/rust-cache@v2
uses: Swatinem/rust-cache@49a0bdc70d2e1b713ca9e2869b211fcce03d3c1c # v2
with:
shared-key: rust-ci-${{ runner.os }}
workspaces: . -> target
- name: Setup sccache
uses: mozilla-actions/[email protected]
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9
- name: Test
- name: Setup mold
uses: rui314/setup-mold@7e4f20ad28a2e8ca6fd0892ccf72e2abb706b9c3 # v1
- name: Install nextest
uses: taiki-e/install-action@d5f9268ff7620505a81ada10ddf18cdd72240185 # nextest
- name: Expose PostgreSQL test binaries
run: pg_config --bindir >> "$GITHUB_PATH"
- name: Test lib
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
run: cargo test -p aether-gateway
RUST_MIN_STACK: "16777216"
RUSTFLAGS: "-C link-arg=-fuse-ld=mold"
run: cargo nextest run -p aether-gateway --lib
- name: Test bins
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
RUST_MIN_STACK: "16777216"
RUSTFLAGS: "-C link-arg=-fuse-ld=mold"
run: cargo nextest run -p aether-gateway --bins
- name: Show sccache stats
if: always()
@@ -213,28 +289,72 @@ jobs:
name: Test (Data)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
uses: dtolnay/rust-toolchain@4360b52568e2003a75bf9bc1d59f33a8e3fc893c # stable
- name: Show Rust toolchain
run: rustup show active-toolchain
- name: Rust cache
uses: Swatinem/rust-cache@v2
uses: Swatinem/rust-cache@49a0bdc70d2e1b713ca9e2869b211fcce03d3c1c # v2
with:
shared-key: rust-ci-${{ runner.os }}
workspaces: . -> target
- name: Setup sccache
uses: mozilla-actions/[email protected]
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9
- name: Install nextest
uses: taiki-e/install-action@d5f9268ff7620505a81ada10ddf18cdd72240185 # nextest
- name: Expose PostgreSQL test binaries
run: pg_config --bindir >> "$GITHUB_PATH"
- name: Test
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
run: cargo test -p aether-data
AETHER_REQUIRE_LOCAL_POSTGRES_TESTS: "true"
run: cargo nextest run -p aether-data
- name: Show sccache stats
if: always()
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
run: sccache --show-stats
check_data_features:
name: Check (Data Feature - ${{ matrix.feature }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
feature:
- postgres
- all-drivers
steps:
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@4360b52568e2003a75bf9bc1d59f33a8e3fc893c # stable
- name: Rust cache
uses: Swatinem/rust-cache@49a0bdc70d2e1b713ca9e2869b211fcce03d3c1c # v2
with:
shared-key: rust-ci-${{ runner.os }}
workspaces: . -> target
- name: Setup sccache
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9
- name: Check selected data driver
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
run: cargo check -p aether-data --no-default-features --features ${{ matrix.feature }}
- name: Show sccache stats
if: always()
@@ -247,28 +367,104 @@ jobs:
name: Test (Workspace Rest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
uses: dtolnay/rust-toolchain@4360b52568e2003a75bf9bc1d59f33a8e3fc893c # stable
- name: Show Rust toolchain
run: rustup show active-toolchain
- name: Rust cache
uses: Swatinem/rust-cache@v2
uses: Swatinem/rust-cache@49a0bdc70d2e1b713ca9e2869b211fcce03d3c1c # v2
with:
shared-key: rust-ci-${{ runner.os }}
workspaces: . -> target
- name: Setup sccache
uses: mozilla-actions/[email protected]
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9
- name: Install nextest
uses: taiki-e/install-action@d5f9268ff7620505a81ada10ddf18cdd72240185 # nextest
- name: Test
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
run: cargo test --workspace --exclude aether-gateway --exclude aether-data
run: cargo nextest run --workspace --exclude aether-gateway --exclude aether-data --exclude aether-integration-tests
- name: Show sccache stats
if: always()
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
run: sccache --show-stats
test_data_adapters:
name: Test (Data Adapter - ${{ matrix.package }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
package:
- aether-data-postgres
steps:
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@4360b52568e2003a75bf9bc1d59f33a8e3fc893c # stable
- name: Rust cache
uses: Swatinem/rust-cache@49a0bdc70d2e1b713ca9e2869b211fcce03d3c1c # v2
with:
shared-key: rust-ci-${{ runner.os }}
workspaces: . -> target
- name: Setup sccache
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9
- name: Install nextest
uses: taiki-e/install-action@d5f9268ff7620505a81ada10ddf18cdd72240185 # nextest
- name: Test adapter
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
run: cargo nextest run -p ${{ matrix.package }}
- name: Show sccache stats
if: always()
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
run: sccache --show-stats
check_integration_scenarios:
name: Test (Integration Scenarios)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@4360b52568e2003a75bf9bc1d59f33a8e3fc893c # stable
- name: Rust cache
uses: Swatinem/rust-cache@49a0bdc70d2e1b713ca9e2869b211fcce03d3c1c # v2
with:
shared-key: rust-ci-${{ runner.os }}
workspaces: . -> target
- name: Setup sccache
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9
- name: Expose PostgreSQL test binaries
run: pg_config --bindir >> "$GITHUB_PATH"
- name: Test scenario binaries and end-to-end suites
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
run: cargo test -p aether-integration-tests --bins --tests
- name: Show sccache stats
if: always()
@@ -283,52 +479,24 @@ jobs:
needs:
- test_gateway
- test_data
- check_data_features
- test_rest
- test_data_adapters
- check_integration_scenarios
if: ${{ always() }}
steps:
- name: Verify test jobs
run: |
if [ "${{ needs.test_gateway.result }}" != "success" ] || \
[ "${{ needs.test_data.result }}" != "success" ] || \
[ "${{ needs.test_rest.result }}" != "success" ]; then
[ "${{ needs.check_data_features.result }}" != "success" ] || \
[ "${{ needs.test_rest.result }}" != "success" ] || \
[ "${{ needs.test_data_adapters.result }}" != "success" ] || \
[ "${{ needs.check_integration_scenarios.result }}" != "success" ]; then
echo "Tests failed"
exit 1
fi
data_db_smoke_sqlite:
name: Data DB Smoke (SQLite)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
- name: Show Rust toolchain
run: rustup show active-toolchain
- name: Rust cache
uses: Swatinem/rust-cache@v2
with:
shared-key: rust-ci-${{ runner.os }}
workspaces: . -> target
- name: Setup sccache
uses: mozilla-actions/[email protected]
- name: Run SQLite data smoke tests
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
run: cargo test -p aether-data sqlite --lib
- name: Show sccache stats
if: always()
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
run: sccache --show-stats
data_db_smoke_postgres:
name: Data DB Smoke (Postgres)
runs-on: ubuntu-latest
@@ -347,142 +515,58 @@ jobs:
--health-timeout=5s
--health-retries=20
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
uses: dtolnay/rust-toolchain@4360b52568e2003a75bf9bc1d59f33a8e3fc893c # stable
- name: Show Rust toolchain
run: rustup show active-toolchain
- name: Rust cache
uses: Swatinem/rust-cache@v2
uses: Swatinem/rust-cache@49a0bdc70d2e1b713ca9e2869b211fcce03d3c1c # v2
with:
shared-key: rust-ci-${{ runner.os }}
workspaces: . -> target
- name: Setup sccache
uses: mozilla-actions/[email protected]
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9
- name: Add PostgreSQL server binaries to PATH
run: echo "$(pg_config --bindir)" >> "$GITHUB_PATH"
- name: Run Postgres migration smoke test
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
AETHER_TEST_POSTGRES_URL: postgres://aether:[email protected]:5432/aether_test
run: cargo test -p aether-data postgres_migrations_create_core_config_tables_when_url_is_set --lib -- --nocapture
run: cargo test -p aether-data --all-features postgres_migrations_create_core_config_tables_when_url_is_set --lib -- --nocapture
- name: Run Postgres provider metadata migration smoke test
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
AETHER_TEST_POSTGRES_URL: postgres://aether:[email protected]:5432/aether_test
run: cargo test -p aether-data --all-features postgres_provider_upstream_metadata_migration_preserves_json_when_url_is_set --lib -- --nocapture
- name: Run Postgres API key lifecycle tests
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
AETHER_REQUIRE_LOCAL_POSTGRES_TESTS: "true"
run: |
cargo test -p aether-data --all-features lifecycle::migrate::tests::postgres_request_candidates_preserve_deleted_api_key_identity --lib -- --exact --nocapture
cargo test -p aether-data --all-features lifecycle::migrate::tests::postgres_request_candidate_migration_decouples_legacy_api_key_foreign_key --lib -- --exact --nocapture
cargo test -p aether-data --all-features lifecycle::migrate::tests::postgres_stats_daily_api_key_migration_decouples_legacy_foreign_key --lib -- --exact --nocapture
cargo test -p aether-data --all-features lifecycle::migrate::tests::postgres_expired_api_key_cleanup_preserves_historical_identity --lib -- --exact --nocapture
cargo test -p aether-data --all-features lifecycle::migrate::tests::postgres_api_key_leaderboard_user_filter_preserves_aggregate_history --lib -- --exact --nocapture
- name: Run Postgres core export smoke test
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
AETHER_TEST_POSTGRES_URL: postgres://aether:[email protected]:5432/aether_test
run: cargo test -p aether-data postgres_core_export_reads_migrated_database_rows_when_url_is_set --lib -- --nocapture
- name: Run SQLite-to-Postgres import smoke test
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
AETHER_TEST_POSTGRES_URL: postgres://aether:[email protected]:5432/aether_test
run: cargo test -p aether-data sqlite_core_export_reads_migrated_database_rows --lib -- --nocapture
- name: Show sccache stats
if: always()
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
run: sccache --show-stats
data_db_smoke_mysql:
name: Data DB Smoke (MySQL)
runs-on: ubuntu-latest
services:
mysql:
image: mysql:8.0
env:
MYSQL_DATABASE: aether_test
MYSQL_USER: aether
MYSQL_PASSWORD: aether
MYSQL_ROOT_PASSWORD: aether_root
ports:
- 3306:3306
options: >-
--health-cmd="mysqladmin ping -h 127.0.0.1 -uaether -paether --silent"
--health-interval=5s
--health-timeout=5s
--health-retries=20
steps:
- uses: actions/checkout@v5
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
- name: Show Rust toolchain
run: rustup show active-toolchain
- name: Rust cache
uses: Swatinem/rust-cache@v2
with:
shared-key: rust-ci-${{ runner.os }}
workspaces: . -> target
- name: Setup sccache
uses: mozilla-actions/[email protected]
- name: Run MySQL migration smoke test
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
AETHER_TEST_MYSQL_URL: mysql://aether:[email protected]:3306/aether_test
run: cargo test -p aether-data mysql_migrations_create_core_config_tables_when_url_is_set --lib -- --nocapture
- name: Run MySQL usage write smoke test
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
AETHER_TEST_MYSQL_URL: mysql://aether:[email protected]:3306/aether_test
run: cargo test -p aether-data mysql_usage_write_repository_upserts_when_url_is_set --lib -- --nocapture
- name: Run MySQL usage read smoke test
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
AETHER_TEST_MYSQL_URL: mysql://aether:[email protected]:3306/aether_test
run: cargo test -p aether-data mysql_usage_read_repository_reads_usage_contract_views_when_url_is_set --lib -- --nocapture
- name: Run MySQL provider catalog smoke test
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
AETHER_TEST_MYSQL_URL: mysql://aether:[email protected]:3306/aether_test
run: cargo test -p aether-data mysql_provider_catalog_repository_round_trips_when_url_is_set --lib -- --nocapture
- name: Run MySQL core export smoke test
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
AETHER_TEST_MYSQL_URL: mysql://aether:[email protected]:3306/aether_test
run: cargo test -p aether-data mysql_core_export_reads_migrated_database_rows_when_url_is_set --lib -- --nocapture
- name: Run MySQL wallet read smoke test
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
AETHER_TEST_MYSQL_URL: mysql://aether:[email protected]:3306/aether_test
run: cargo test -p aether-data mysql_wallet_read_repository_reads_wallet_contract_views --lib -- --nocapture
- name: Run MySQL wallet daily usage aggregation smoke test
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
AETHER_TEST_MYSQL_URL: mysql://aether:[email protected]:3306/aether_test
run: cargo test -p aether-data mysql_wallet_daily_usage_aggregation_uses_settlement_wallets_when_url_is_set --lib -- --nocapture
- name: Run MySQL stats aggregation smoke test
env:
RUSTC_WRAPPER: sccache
SCCACHE_GHA_ENABLED: "true"
AETHER_TEST_MYSQL_URL: mysql://aether:[email protected]:3306/aether_test
run: cargo test -p aether-data mysql_stats_aggregation_runs_after_mysql_migrations_when_url_is_set --lib -- --nocapture
run: cargo test -p aether-data --all-features postgres_core_export_reads_migrated_database_rows_when_url_is_set --lib -- --nocapture
- name: Show sccache stats
if: always()
@@ -495,16 +579,12 @@ jobs:
name: Data DB Smoke
runs-on: ubuntu-latest
needs:
- data_db_smoke_sqlite
- data_db_smoke_postgres
- data_db_smoke_mysql
if: ${{ always() }}
steps:
- name: Verify database smoke jobs
run: |
if [ "${{ needs.data_db_smoke_sqlite.result }}" != "success" ] || \
[ "${{ needs.data_db_smoke_postgres.result }}" != "success" ] || \
[ "${{ needs.data_db_smoke_mysql.result }}" != "success" ]; then
if [ "${{ needs.data_db_smoke_postgres.result }}" != "success" ]; then
echo "Data DB smoke failed"
exit 1
fi
@@ -517,6 +597,7 @@ jobs:
- clippy
- test
- data_db_smoke
- shell_security
if: ${{ always() }}
steps:
- name: Verify required jobs
@@ -524,7 +605,8 @@ jobs:
if [ "${{ needs.fmt.result }}" != "success" ] || \
[ "${{ needs.clippy.result }}" != "success" ] || \
[ "${{ needs.test.result }}" != "success" ] || \
[ "${{ needs.data_db_smoke.result }}" != "success" ]; then
[ "${{ needs.data_db_smoke.result }}" != "success" ] || \
[ "${{ needs.shell_security.result }}" != "success" ]; then
echo "Rust CI failed"
exit 1
fi
+12 -1
View File
@@ -1,12 +1,21 @@
# Created by https://www.toptal.com/developers/gitignore/api/python
# Edit at https://www.toptal.com/developers/gitignore?templates=python
*.rsa
*_rsa
# AI Assistant Configuration
.codex/
.claude/
.deepseek/
.serena/
.gemini*/
.plans
.playwright-mcp/
docs/architecture
!docs/architecture/architecture-dark.svg
!docs/architecture/architecture-light.svg
### Python ###
*.db
@@ -242,4 +251,6 @@ src/_version.py
# Analysis folder (third-party code for reference)
analysis/
new-api/
apps/aether-proxy/aether-proxy.toml
apps/aether-tunnel/aether-tunnel.toml
# Generated by frontend/scripts/sync-vscodex.mjs.
frontend/public/aether-vscodex/
+2
View File
@@ -0,0 +1,2 @@
[tools]
rust = "latest"
Generated
+1525 -323
View File
File diff suppressed because it is too large Load Diff
+78 -22
View File
@@ -1,27 +1,50 @@
[workspace]
members = [
"apps/aether-proxy",
"crates/aether-ai-formats",
"apps/aether-tunnel",
"crates/aether-ai/formats",
"crates/aether-admin",
"crates/aether-ai-serving",
"crates/aether-data-contracts",
"crates/aether-data-schema",
"crates/aether-admission-core",
"crates/aether-ai/serving",
"crates/aether-pool-core",
"crates/aether-provider/core",
"crates/aether-provider/pool",
"crates/aether-routing-core",
"crates/aether-data/contracts",
"crates/aether-data/adapters/postgres",
"crates/aether-data/query",
"crates/aether-data/schema",
"crates/aether-dispatch-core",
"crates/aether-cache",
"crates/aether-billing",
"crates/aether-wallet",
"crates/aether-crypto",
"crates/aether-contracts",
"crates/aether-data",
"crates/aether-data/runtime",
"crates/aether-model-fetch",
"crates/aether-oauth",
"crates/aether-provider-transport",
"crates/aether-provider/transport",
"crates/aether-scheduler-core",
"crates/aether-usage-runtime",
"crates/aether-runtime/state",
"crates/aether-task/runtime",
"crates/aether-task/core",
"crates/aether-gateway/frontdoor",
"crates/aether-gateway/control",
"crates/aether-gateway/execution",
"crates/aether-gateway/workers",
"crates/aether-gateway/tunnel",
"crates/aether-testing/loadtools",
"crates/aether-testing/integration",
"crates/aether-usage/core",
"crates/aether-testing/support",
"crates/aether-usage/runtime",
"crates/aether-video-tasks-core",
"apps/aether-gateway",
"crates/aether-http",
"crates/aether-runtime",
"crates/aether-testkit",
"crates/aether-runtime/base",
"crates/aether-testing/testkit",
]
default-members = [
"apps/aether-gateway",
]
resolver = "2"
@@ -32,58 +55,91 @@ repository = "https://github.com/fawney19/Aether.git"
[workspace.dependencies]
aether-admin = { path = "crates/aether-admin" }
aether-ai-formats = { path = "crates/aether-ai-formats" }
aether-ai-serving = { path = "crates/aether-ai-serving" }
aether-data-contracts = { path = "crates/aether-data-contracts" }
aether-data-schema = { path = "crates/aether-data-schema" }
aether-admission-core = { path = "crates/aether-admission-core" }
aether-ai-formats = { path = "crates/aether-ai/formats" }
aether-ai-serving = { path = "crates/aether-ai/serving" }
aether-pool-core = { path = "crates/aether-pool-core" }
aether-provider-core = { path = "crates/aether-provider/core" }
aether-provider-pool = { path = "crates/aether-provider/pool" }
aether-routing-core = { path = "crates/aether-routing-core" }
aether-data-contracts = { path = "crates/aether-data/contracts" }
aether-data-postgres = { path = "crates/aether-data/adapters/postgres" }
aether-data-query = { path = "crates/aether-data/query" }
aether-data-schema = { path = "crates/aether-data/schema" }
aether-dispatch-core = { path = "crates/aether-dispatch-core" }
aether-cache = { path = "crates/aether-cache" }
aether-billing = { path = "crates/aether-billing" }
aether-wallet = { path = "crates/aether-wallet" }
aether-crypto = { path = "crates/aether-crypto" }
aether-contracts = { path = "crates/aether-contracts" }
aether-data = { path = "crates/aether-data" }
aether-data = { path = "crates/aether-data/runtime" }
aether-model-fetch = { path = "crates/aether-model-fetch" }
aether-oauth = { path = "crates/aether-oauth" }
aether-provider-transport = { path = "crates/aether-provider-transport" }
aether-provider-transport = { path = "crates/aether-provider/transport" }
aether-scheduler-core = { path = "crates/aether-scheduler-core" }
aether-usage-runtime = { path = "crates/aether-usage-runtime" }
aether-runtime-state = { path = "crates/aether-runtime/state" }
aether-task-runtime = { path = "crates/aether-task/runtime" }
aether-task-core = { path = "crates/aether-task/core" }
aether-gateway-frontdoor = { path = "crates/aether-gateway/frontdoor" }
aether-gateway-control = { path = "crates/aether-gateway/control" }
aether-gateway-execution = { path = "crates/aether-gateway/execution" }
aether-gateway-workers = { path = "crates/aether-gateway/workers" }
aether-gateway-tunnel = { path = "crates/aether-gateway/tunnel" }
aether-loadtools = { path = "crates/aether-testing/loadtools" }
aether-integration-tests = { path = "crates/aether-testing/integration" }
aether-test-support = { path = "crates/aether-testing/support" }
aether-usage-core = { path = "crates/aether-usage/core" }
aether-usage-runtime = { path = "crates/aether-usage/runtime" }
aether-video-tasks-core = { path = "crates/aether-video-tasks-core" }
aether-gateway = { path = "apps/aether-gateway" }
aether-http = { path = "crates/aether-http" }
aether-runtime = { path = "crates/aether-runtime" }
aether-testkit = { path = "crates/aether-testkit" }
aether-runtime = { path = "crates/aether-runtime/base" }
aether-testkit = { path = "crates/aether-testing/testkit" }
aes = "0.8"
aes-gcm = "0.10"
aws-lc-rs = { version = "1.16.2", default-features = false, features = ["alloc", "aws-lc-sys"] }
async-stream = "0.3"
async-trait = "0.1"
axum = "0.8"
base64 = "0.22"
bcrypt = "0.16"
brotli = "8"
bytes = "1"
cbc = "0.1"
chrono = { version = "0.4", features = ["serde"] }
chrono-tz = "0.10"
crypto_box = { version = "0.9", features = ["seal"] }
ed25519-dalek = { version = "2.2", features = ["pkcs8"] }
flate2 = "1"
futures-util = "0.3"
hmac = "0.12"
http = "1"
object_store = { version = "0.14.1", default-features = false, features = ["aws"] }
pbkdf2 = { version = "0.12", default-features = false, features = ["hmac"] }
percent-encoding = "2"
reqwest = { version = "0.12", default-features = false, features = ["json", "stream", "rustls-tls", "http2", "socks"] }
redis = { version = "0.28", default-features = false, features = ["tokio-comp", "script", "streams"] }
redis = { version = "0.28", default-features = false, features = ["tokio-comp", "script", "streams", "connection-manager"] }
regex = "1"
rustls = { version = "0.23", features = ["ring"] }
semver = "1"
serde = { version = "1", features = ["derive"] }
serde_json = { version = "1", features = ["preserve_order"] }
serde_path_to_error = "0.1"
sha2 = "0.10"
sqlx = { version = "0.8", default-features = false, features = ["postgres", "mysql", "sqlite", "runtime-tokio-rustls", "chrono"] }
socket2 = "0.6"
tar = "0.4"
sqlx = { version = "0.8", default-features = false, features = ["runtime-tokio-rustls", "chrono"] }
thiserror = "2"
tokio = { version = "1", features = ["macros", "net", "rt-multi-thread", "signal", "sync", "time"] }
tokio-util = { version = "0.7", features = ["codec", "io-util"] }
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] }
uuid = { version = "1", features = ["serde", "v4", "v5"] }
uuid = { version = "1", features = ["serde", "v4", "v5", "v7"] }
webpki-roots = "0.26"
wreq = { version = "6.0.0-rc.28", default-features = false, features = ["json", "stream", "socks", "webpki-roots", "ws"] }
wreq-util = "3.0.0-rc.10"
url = "2"
zstd = "0.13"
[profile.dev]
# Keep file/line information for backtraces while avoiding full debug info
+33 -16
View File
@@ -1,31 +1,48 @@
# syntax=docker/dockerfile:1
# Aether Gateway 运行时镜像(交叉编译方案)
# 二进制和前端产物均由 CI 预先构建,此 Dockerfile 仅做打包
# 用法: docker buildx build --platform linux/amd64,linux/arm64 -f Dockerfile.app .
# Aether Gateway runtime image (cross-compilation)
# Binary and frontend assets are pre-built by CI; this Dockerfile only packages them.
# Usage: docker buildx build --platform linux/amd64,linux/arm64 -f Dockerfile.app .
#
# 构建上下文中须包含:
# dist/aether-gateway-amd64 (x86_64-unknown-linux-musl 交叉编译产物)
# dist/aether-gateway-arm64 (aarch64-unknown-linux-musl 交叉编译产物)
# dist/frontend/ (npm run build 产物)
# Build context must contain:
# dist/aether-gateway-amd64 (x86_64-unknown-linux-musl cross-compiled binary)
# dist/aether-gateway-arm64 (aarch64-unknown-linux-musl cross-compiled binary)
# dist/frontend/ (npm run build output)
FROM gcr.io/distroless/static-debian12
# --- layout stage: create /opt/aether directory structure with symlink ---
# distroless has no shell, so we use busybox to set up the symlink.
FROM busybox:1.37.0-musl@sha256:fc6dddc4c44b1bfe37f41cae8e67d1693828e8f42a91862816d7953e2c9d3f23 AS layout
# TARGETARCH 由 buildx 自动注入: amd64 或 arm64
ARG TARGETARCH
COPY dist/aether-gateway-${TARGETARCH} /usr/local/bin/aether-gateway
COPY dist/frontend/ /srv/frontend
RUN mkdir -p /opt/aether/releases/image/bin /opt/aether/releases/image/frontend /opt/aether/logs
WORKDIR /app
COPY dist/aether-gateway-${TARGETARCH} /opt/aether/releases/image/bin/aether-gateway
COPY dist/frontend/ /opt/aether/releases/image/frontend/
# Keep the immutable release root-owned while guaranteeing that the runtime
# identity can traverse and read every packaged asset.
RUN chmod -R u=rwX,go=rX /opt/aether/releases/image \
&& chmod 0755 /opt/aether/releases/image/bin/aether-gateway
RUN ln -s /opt/aether/releases/image /opt/aether/current
# --- final stage: distroless runtime ---
FROM gcr.io/distroless/static-debian12@sha256:6447365a6337c3732f412d1b74357b30a633831955b2bc45552b0086be907687
COPY --from=layout /opt/aether /opt/aether
WORKDIR /opt/aether
ENV RUST_LOG=aether_gateway=info \
APP_PORT=8084 \
AETHER_GATEWAY_STATIC_DIR=/srv/frontend
HOME=/tmp/aether-home \
AETHER_UPDATE_STRATEGY=docker \
AETHER_GATEWAY_STATIC_DIR=/opt/aether/current/frontend
EXPOSE 8084
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD ["/usr/local/bin/aether-gateway", "--healthcheck"]
CMD ["/opt/aether/current/bin/aether-gateway", "--healthcheck"]
USER root
ENTRYPOINT ["/usr/local/bin/aether-gateway"]
USER 0:0
ENTRYPOINT ["/opt/aether/current/bin/aether-gateway"]
+39 -9
View File
@@ -1,23 +1,42 @@
# syntax=docker/dockerfile:1
# syntax=docker.m.daocloud.io/docker/dockerfile:1
# Aether 运行镜像:Rust gateway 直接服务 API + 前端静态文件(国内镜像源版本)
# 构建命令: docker build -f Dockerfile.app.local -t aether-app:latest .
# 构建命令: docker build --build-arg AETHER_BUILD_VERSION=v0.7.2 -f Dockerfile.app.local -t aether-app:latest .
ARG RUST_VERSION=1.95.0
ARG NODE_BASE_IMAGE=docker.m.daocloud.io/library/node:22-slim
ARG RUST_BASE_IMAGE=docker.m.daocloud.io/library/rust:${RUST_VERSION}-slim
# ==================== 前端构建 ====================
FROM node:22-slim AS frontend-builder
FROM ${NODE_BASE_IMAGE} AS frontend-builder
ARG AETHER_BUILD_VERSION
ENV AETHER_BUILD_VERSION=${AETHER_BUILD_VERSION} \
AETHER_VERSION=${AETHER_BUILD_VERSION}
WORKDIR /app/aether-vscodex/web
COPY aether-vscodex/web/package*.json ./
RUN --mount=type=cache,id=aether-vscodex-npm-cache,target=/root/.npm,sharing=locked \
npm config set registry https://registry.npmmirror.com && \
npm ci --no-audit --no-fund
COPY aether-vscodex/public /app/aether-vscodex/public
COPY aether-vscodex/web/ ./
RUN npm run build
WORKDIR /app/frontend
COPY frontend/package*.json ./
RUN npm config set registry https://registry.npmmirror.com && npm ci
RUN --mount=type=cache,id=aether-npm-cache,target=/root/.npm,sharing=locked \
npm config set registry https://registry.npmmirror.com && \
npm ci --no-audit --no-fund
COPY frontend/ ./
RUN npm run build
# ==================== Rust gateway 构建 ====================
FROM rust:1.94.1-slim AS gateway-base
FROM ${RUST_BASE_IMAGE} AS gateway-base
WORKDIR /build
# 本地镜像优先缩短构建时间,保留 release 语义,但改用更快的 thin LTO。
# 生产级 release 构建:保留 thin LTO,同时用 lld 缩短最终链接阶段。
ENV CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse \
CARGO_PROFILE_RELEASE_LTO=thin \
CARGO_PROFILE_RELEASE_CODEGEN_UNITS=16
CARGO_PROFILE_RELEASE_CODEGEN_UNITS=16 \
RUSTFLAGS="-C linker=clang -C link-arg=-fuse-ld=lld"
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked \
@@ -25,7 +44,12 @@ RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
apt-get update && apt-get install -y --no-install-recommends \
build-essential \
ca-certificates \
clang \
cmake \
git \
libclang-dev \
libssl-dev \
lld \
pkg-config \
perl
@@ -40,11 +64,14 @@ COPY crates/ ./crates/
RUN cargo chef prepare --recipe-path recipe.json
FROM gateway-base AS gateway-builder
ARG AETHER_BUILD_VERSION
ENV AETHER_BUILD_VERSION=${AETHER_BUILD_VERSION} \
AETHER_VERSION=${AETHER_BUILD_VERSION}
COPY --from=gateway-planner /build/recipe.json ./recipe.json
RUN --mount=type=cache,id=aether-cargo-registry,target=/usr/local/cargo/registry,sharing=locked \
--mount=type=cache,id=aether-cargo-git,target=/usr/local/cargo/git,sharing=locked \
--mount=type=cache,id=aether-cargo-target-local,target=/build/target,sharing=locked \
cargo chef cook --release --locked --package aether-gateway --bin aether-gateway --recipe-path recipe.json
cargo chef cook --release --locked --package aether-gateway --bin aether-gateway --features jemalloc --recipe-path recipe.json
COPY Cargo.toml Cargo.lock ./
COPY apps/ ./apps/
@@ -52,7 +79,8 @@ COPY crates/ ./crates/
RUN --mount=type=cache,id=aether-cargo-registry,target=/usr/local/cargo/registry,sharing=locked \
--mount=type=cache,id=aether-cargo-git,target=/usr/local/cargo/git,sharing=locked \
--mount=type=cache,id=aether-cargo-target-local,target=/build/target,sharing=locked \
cargo build --release --locked -p aether-gateway && \
set -eux; \
cargo build --release --locked -p aether-gateway --bin aether-gateway --features jemalloc; \
cp target/release/aether-gateway /tmp/aether-gateway
# ==================== 最小运行时打包 ====================
@@ -121,6 +149,7 @@ ENV LANG=C.UTF-8 \
LC_ALL=C.UTF-8 \
RUST_LOG=aether_gateway=info \
APP_PORT=8084 \
AETHER_UPDATE_STRATEGY=manual \
AETHER_GATEWAY_STATIC_DIR=/srv/frontend
EXPOSE 8084
@@ -128,4 +157,5 @@ EXPOSE 8084
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD ["/usr/local/bin/aether-gateway", "--healthcheck"]
USER 0:0
ENTRYPOINT ["/usr/local/bin/aether-gateway"]
+160
View File
@@ -0,0 +1,160 @@
# syntax=docker.m.daocloud.io/docker/dockerfile:1
# Aether 本地发布版联调镜像
# 作用:用当前源码构建一个 release-layout 容器,专门测试管理后台在线更新流程。
ARG RUST_VERSION=1.95.0
ARG NODE_BASE_IMAGE=docker.m.daocloud.io/library/node:22-slim
ARG RUST_BASE_IMAGE=docker.m.daocloud.io/library/rust:${RUST_VERSION}-slim
# ==================== 前端构建 ====================
FROM ${NODE_BASE_IMAGE} AS frontend-builder
ARG AETHER_BUILD_VERSION
ENV AETHER_BUILD_VERSION=${AETHER_BUILD_VERSION} \
AETHER_VERSION=${AETHER_BUILD_VERSION}
WORKDIR /app/aether-vscodex/web
COPY aether-vscodex/web/package*.json ./
RUN --mount=type=cache,id=aether-vscodex-npm-cache,target=/root/.npm,sharing=locked \
npm config set registry https://registry.npmmirror.com && \
npm ci --no-audit --no-fund
COPY aether-vscodex/public /app/aether-vscodex/public
COPY aether-vscodex/web/ ./
RUN npm run build
WORKDIR /app/frontend
COPY frontend/package*.json ./
RUN --mount=type=cache,id=aether-npm-cache,target=/root/.npm,sharing=locked \
npm config set registry https://registry.npmmirror.com && \
npm ci --no-audit --no-fund
COPY frontend/ ./
RUN npm run build
# ==================== Rust gateway 构建 ====================
FROM ${RUST_BASE_IMAGE} AS gateway-base
WORKDIR /build
ENV CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse \
CARGO_PROFILE_RELEASE_LTO=thin \
CARGO_PROFILE_RELEASE_CODEGEN_UNITS=16
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked \
sed -i 's/deb.debian.org/mirrors.tuna.tsinghua.edu.cn/g' /etc/apt/sources.list.d/debian.sources && \
apt-get update && apt-get install -y --no-install-recommends \
build-essential \
ca-certificates \
cmake \
git \
libclang-dev \
libssl-dev \
pkg-config \
perl
RUN --mount=type=cache,id=aether-cargo-registry,target=/usr/local/cargo/registry,sharing=locked \
--mount=type=cache,id=aether-cargo-git,target=/usr/local/cargo/git,sharing=locked \
cargo install cargo-chef --locked
FROM gateway-base AS gateway-planner
COPY Cargo.toml Cargo.lock ./
COPY apps/ ./apps/
COPY crates/ ./crates/
RUN cargo chef prepare --recipe-path recipe.json
FROM gateway-base AS gateway-builder
ARG AETHER_BUILD_VERSION
ARG AETHER_BUILD_TYPE=release
ENV AETHER_BUILD_VERSION=${AETHER_BUILD_VERSION} \
AETHER_VERSION=${AETHER_BUILD_VERSION} \
AETHER_BUILD_TYPE=${AETHER_BUILD_TYPE}
COPY --from=gateway-planner /build/recipe.json ./recipe.json
RUN --mount=type=cache,id=aether-cargo-registry,target=/usr/local/cargo/registry,sharing=locked \
--mount=type=cache,id=aether-cargo-git,target=/usr/local/cargo/git,sharing=locked \
--mount=type=cache,id=aether-cargo-target-release-local,target=/build/target,sharing=locked \
cargo chef cook --release --locked --package aether-gateway --bin aether-gateway --features jemalloc --recipe-path recipe.json
COPY Cargo.toml Cargo.lock ./
COPY apps/ ./apps/
COPY crates/ ./crates/
RUN --mount=type=cache,id=aether-cargo-registry,target=/usr/local/cargo/registry,sharing=locked \
--mount=type=cache,id=aether-cargo-git,target=/usr/local/cargo/git,sharing=locked \
--mount=type=cache,id=aether-cargo-target-release-local,target=/build/target,sharing=locked \
cargo build --release --locked -p aether-gateway --features jemalloc && \
cp target/release/aether-gateway /tmp/aether-gateway
# ==================== 最小运行时打包 ====================
FROM gateway-builder AS runtime-prep
RUN set -eux; \
mkdir -p \
/runtime-root/app/data \
/runtime-root/etc \
/runtime-root/etc/ssl \
/runtime-root/lib \
/runtime-root/lib64 \
/runtime-root/usr/lib \
/runtime-root/opt/aether/logs \
/runtime-root/opt/aether/releases/image/bin \
/runtime-root/opt/aether/releases/image/frontend; \
cp /tmp/aether-gateway /runtime-root/opt/aether/releases/image/bin/aether-gateway; \
ln -s /opt/aether/releases/image /runtime-root/opt/aether/current; \
: > /tmp/runtime-libs.txt; \
: > /tmp/runtime-scan-queue.txt; \
printf '%s\n' /tmp/aether-gateway >> /tmp/runtime-scan-queue.txt; \
while [ -s /tmp/runtime-scan-queue.txt ]; do \
current="$(head -n1 /tmp/runtime-scan-queue.txt)"; \
sed -i '1d' /tmp/runtime-scan-queue.txt; \
ldd "$current" | awk '/=>/ { print $3 } $1 ~ /^\// { print $1 }' | while read -r lib; do \
[ -n "$lib" ]; \
if ! grep -Fxq "$lib" /tmp/runtime-libs.txt; then \
printf '%s\n' "$lib" >> /tmp/runtime-libs.txt; \
printf '%s\n' "$lib" >> /tmp/runtime-scan-queue.txt; \
fi; \
done; \
done; \
sort -u /tmp/runtime-libs.txt -o /tmp/runtime-libs.txt; \
while read -r lib; do \
[ -n "$lib" ]; \
dest="/runtime-root$(dirname "$lib")"; \
mkdir -p "$dest"; \
cp -L "$lib" "$dest/"; \
done < /tmp/runtime-libs.txt; \
for lib in \
/lib/x86_64-linux-gnu/libnss_dns.so.2 \
/lib/x86_64-linux-gnu/libnss_files.so.2 \
/lib/x86_64-linux-gnu/libresolv.so.2; do \
if [ -f "$lib" ]; then \
dest="/runtime-root$(dirname "$lib")"; \
mkdir -p "$dest"; \
cp -L "$lib" "$dest/"; \
fi; \
done; \
cp -a /usr/lib/ssl /runtime-root/usr/lib/; \
cp -a /etc/ssl/certs /runtime-root/etc/ssl/; \
if [ -f /etc/ssl/openssl.cnf ]; then \
cp /etc/ssl/openssl.cnf /runtime-root/etc/ssl/openssl.cnf; \
fi; \
if [ -f /etc/nsswitch.conf ]; then \
cp /etc/nsswitch.conf /runtime-root/etc/nsswitch.conf; \
fi
COPY --from=frontend-builder /app/frontend/dist /runtime-root/opt/aether/releases/image/frontend
# ==================== 运行时镜像 ====================
FROM scratch
COPY --from=runtime-prep /runtime-root/ /
WORKDIR /app
ENV LANG=C.UTF-8 \
LC_ALL=C.UTF-8 \
RUST_LOG=aether_gateway=info \
APP_PORT=8084 \
AETHER_BASE_DIR=/opt/aether \
AETHER_UPDATE_STRATEGY=self \
AETHER_GATEWAY_STATIC_DIR=/opt/aether/current/frontend
EXPOSE 8084
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD ["/opt/aether/current/bin/aether-gateway", "--healthcheck"]
USER 0:0
ENTRYPOINT ["/opt/aether/current/bin/aether-gateway"]
+582
View File
@@ -0,0 +1,582 @@
SHELL := /bin/bash
DEV_RUST_LOG := info,executor::candidate_loop=debug,stream::execution=debug
ifeq ($(origin RUST_LOG), command line)
DEV_RUST_LOG := $(RUST_LOG)
endif
export DEV_RUST_LOG
.PHONY: dev dev-backend dev-frontend db-status db-prepare migration backfill
define DEV_BACKEND_SCRIPT
set -euo pipefail
if [ ! -f .env ]; then
echo "=> 未找到 .env,请先执行: cp .env.example .env"
exit 1
fi
set -a
source .env
set +a
if [[ -n "$${ADMIN_EMAIL:-}" || -n "$${ADMIN_USERNAME:-}" || -n "$${ADMIN_PASSWORD:-}" ]]; then
if [[ -z "$${ADMIN_USERNAME:-}" || -z "$${ADMIN_PASSWORD:-}" ]]; then
echo "=> 管理员自举配置不完整,请在 .env 中设置 ADMIN_USERNAME 和 ADMIN_PASSWORD"
exit 1
fi
fi
dotenv_has_key() {
local key="$$1"
grep -Eq "^[[:space:]]*$${key}=" .env
}
lowercase() {
printf '%s' "$$1" | tr '[:upper:]' '[:lower:]'
}
dev_uses_postgres_database() {
local driver
local url
driver="$$(lowercase "$${AETHER_DATABASE_DRIVER:-}")"
url="$${AETHER_DATABASE_URL:-$${DATABASE_URL:-}}"
if [[ -z "$${driver}" && -z "$${url}" ]]; then
return 0
fi
[[ "$${driver}" == "postgres" || "$${driver}" == "postgresql" || "$${url}" == postgres:* || "$${url}" == postgresql:* ]]
}
dev_uses_redis_runtime() {
local backend
backend="$$(lowercase "$${AETHER_RUNTIME_BACKEND:-}")"
if [[ "$${backend}" == "memory" ]]; then
return 1
fi
if [[ "$${backend}" == "redis" ]]; then
return 0
fi
return 0
}
print_dev_infra_hint() {
echo "=> 本地开发依赖未就绪。"
echo "=> 可手动启动 Postgres / Redis:"
echo "=> docker compose up -d postgres redis"
}
check_postgres_ready() {
local host="$$1"
local port="$$2"
if command -v pg_isready >/dev/null 2>&1; then
pg_isready -h "$${host}" -p "$${port}" >/dev/null 2>&1
return $$?
fi
if command -v nc >/dev/null 2>&1; then
nc -z "$${host}" "$${port}" >/dev/null 2>&1
return $$?
fi
return 0
}
check_redis_ready() {
local host="$$1"
local port="$$2"
local password="$$3"
if command -v redis-cli >/dev/null 2>&1; then
REDISCLI_AUTH="$${password}" redis-cli -h "$${host}" -p "$${port}" ping >/dev/null 2>&1
return $$?
fi
if command -v nc >/dev/null 2>&1; then
nc -z "$${host}" "$${port}" >/dev/null 2>&1
return $$?
fi
return 0
}
is_local_host() {
case "$$1" in
localhost|127.0.0.1|::1)
return 0
;;
esac
return 1
}
ensure_dev_infra() {
local postgres_host="$${DB_HOST:-localhost}"
local postgres_port="$${DB_PORT:-5432}"
local redis_host="$${REDIS_HOST:-localhost}"
local redis_port="$${REDIS_PORT:-6379}"
local redis_password="$${REDIS_PASSWORD:-}"
local need_postgres=false
local need_redis=false
local services=()
if dev_uses_postgres_database; then
if ! check_postgres_ready "$${postgres_host}" "$${postgres_port}"; then
if is_local_host "$${postgres_host}"; then
need_postgres=true
services+=(postgres)
else
echo "=> PostgreSQL 不可用: $${postgres_host}:$${postgres_port}"
print_dev_infra_hint
return 1
fi
fi
fi
if dev_uses_redis_runtime; then
if ! check_redis_ready "$${redis_host}" "$${redis_port}" "$${redis_password}"; then
if is_local_host "$${redis_host}"; then
need_redis=true
services+=(redis)
else
echo "=> Redis 不可用: $${redis_host}:$${redis_port}"
print_dev_infra_hint
return 1
fi
fi
fi
if [ "$${#services[@]}" -eq 0 ]; then
return 0
fi
if ! command -v docker >/dev/null 2>&1; then
echo "=> 未找到 docker,无法自动启动本地开发依赖。"
print_dev_infra_hint
return 1
fi
echo "=> 本地开发依赖未就绪,正在启动: docker compose up -d $${services[*]}"
if ! docker compose up -d "$${services[@]}"; then
echo "=> docker compose 启动本地开发依赖失败。"
print_dev_infra_hint
return 1
fi
for _ in {1..100}; do
local ready=true
if [ "$${need_postgres}" = "true" ] && ! check_postgres_ready "$${postgres_host}" "$${postgres_port}"; then
ready=false
fi
if [ "$${need_redis}" = "true" ] && ! check_redis_ready "$${redis_host}" "$${redis_port}" "$${redis_password}"; then
ready=false
fi
if [ "$${ready}" = "true" ]; then
return 0
fi
sleep 0.2
done
if [ "$${need_postgres}" = "true" ] && ! check_postgres_ready "$${postgres_host}" "$${postgres_port}"; then
echo "=> PostgreSQL 不可用: $${postgres_host}:$${postgres_port}"
fi
if [ "$${need_redis}" = "true" ] && ! check_redis_ready "$${redis_host}" "$${redis_port}" "$${redis_password}"; then
echo "=> Redis 不可用: $${redis_host}:$${redis_port}"
fi
print_dev_infra_hint
return 1
}
print_startup_failure_hint() {
local log_file="$$1"
if [ -n "$${log_file}" ] && [ -f "$${log_file}" ]; then
if grep -Eq "database schema is behind" "$${log_file}"; then
echo "=> 检测到数据库尚未准备完成,请执行: make db-prepare"
return
fi
if grep -Eq "database backfills are behind" "$${log_file}"; then
echo "=> 检测到数据库尚未准备完成,请执行: make db-prepare"
return
fi
if grep -Eq "bootstrap admin env is partially configured.*ADMIN_PASSWORD" "$${log_file}"; then
echo "=> 首次启动需要管理员密码,请在 .env 中设置 ADMIN_PASSWORD"
return
fi
fi
echo "=> 未识别到明确的修复动作,请根据上面的日志继续排查。"
}
wait_for_startup() {
local pid="$$1"
local timeout_seconds="$$2"
local service_name="$$3"
shift 3
STARTUP_WAIT_EARLY_EXIT=false
local attempts=$$((timeout_seconds * 10))
if [ "$${attempts}" -lt 1 ]; then
attempts=1
fi
for ((i = 0; i < attempts; i++)); do
if "$$@" >/dev/null 2>&1; then
return 0
fi
if ! kill -0 "$${pid}" >/dev/null 2>&1; then
STARTUP_WAIT_EARLY_EXIT=true
echo "=> $${service_name} 启动进程已提前退出,请检查上面的日志。"
print_startup_failure_hint "$${GATEWAY_LOG_FILE}"
return 1
fi
sleep 0.1
done
if "$$@" >/dev/null 2>&1; then
return 0
fi
if ! kill -0 "$${pid}" >/dev/null 2>&1; then
STARTUP_WAIT_EARLY_EXIT=true
echo "=> $${service_name} 启动进程已提前退出,请检查上面的日志。"
print_startup_failure_hint "$${GATEWAY_LOG_FILE}"
return 1
fi
echo "=> $${service_name} 在 $${timeout_seconds}s 内未通过启动检查。"
echo "=> 如果这是冷编译或存在并发 cargo 构建,可调大启动超时后重试。"
return 1
}
create_gateway_log_file() {
local tmp_root="$${TMPDIR:-/tmp}"
tmp_root="$${tmp_root%/}"
GATEWAY_LOG_DIR="$$(mktemp -d "$${tmp_root}/aether-dev-startup.XXXXXX")"
GATEWAY_LOG_FILE="$${GATEWAY_LOG_DIR}/gateway.log"
: > "$${GATEWAY_LOG_FILE}"
}
cleanup() {
local status="$${1:-0}"
trap - INT TERM EXIT
if [ -n "$${GATEWAY_PID:-}" ]; then
echo ""
echo "=> 停止 aether-gateway..."
kill "$${GATEWAY_PID}" >/dev/null 2>&1 || true
wait "$${GATEWAY_PID}" >/dev/null 2>&1 || true
fi
if [ -n "$${GATEWAY_LOG_FILE:-}" ] && [ -f "$${GATEWAY_LOG_FILE}" ]; then
rm -f "$${GATEWAY_LOG_FILE}"
fi
if [ -n "$${GATEWAY_LOG_DIR:-}" ] && [ -d "$${GATEWAY_LOG_DIR}" ]; then
rmdir "$${GATEWAY_LOG_DIR}" >/dev/null 2>&1 || true
fi
exit "$${status}"
}
trap 'cleanup 130' INT
trap 'cleanup 143' TERM
trap 'cleanup $$?' EXIT
export APP_PORT="$${APP_PORT:-8084}"
export RUST_LOG="$${DEV_RUST_LOG}"
RUST_SERVICE_STARTUP_TIMEOUT_SECONDS="$${RUST_SERVICE_STARTUP_TIMEOUT_SECONDS:-180}"
GATEWAY_STARTUP_TIMEOUT_SECONDS="$${GATEWAY_STARTUP_TIMEOUT_SECONDS:-$${RUST_SERVICE_STARTUP_TIMEOUT_SECONDS}}"
export AETHER_GATEWAY_VIDEO_TASK_TRUTH_SOURCE_MODE="$${AETHER_GATEWAY_VIDEO_TASK_TRUTH_SOURCE_MODE:-rust-authoritative}"
if dev_uses_postgres_database; then
export DATABASE_URL="postgresql://$${DB_USER:-postgres}:$${DB_PASSWORD:-}@$${DB_HOST:-localhost}:$${DB_PORT:-5432}/$${DB_NAME:-aether}"
if ! dotenv_has_key "AETHER_GATEWAY_DATA_POSTGRES_URL"; then
export AETHER_GATEWAY_DATA_POSTGRES_URL="$${DATABASE_URL}"
fi
fi
if dev_uses_redis_runtime; then
export REDIS_URL="redis://:$${REDIS_PASSWORD:-}@$${REDIS_HOST:-localhost}:$${REDIS_PORT:-6379}/0"
if ! dotenv_has_key "AETHER_GATEWAY_DATA_REDIS_URL"; then
export AETHER_GATEWAY_DATA_REDIS_URL="$${REDIS_URL}"
fi
else
unset REDIS_URL
unset AETHER_GATEWAY_DATA_REDIS_URL
fi
if ! dotenv_has_key "AETHER_GATEWAY_DATA_ENCRYPTION_KEY"; then
export AETHER_GATEWAY_DATA_ENCRYPTION_KEY="$${ENCRYPTION_KEY:-}"
fi
export DB_POOL_SIZE="$${DB_POOL_SIZE:-5}"
export DB_MAX_OVERFLOW="$${DB_MAX_OVERFLOW:-5}"
export HTTP_MAX_CONNECTIONS="$${HTTP_MAX_CONNECTIONS:-20}"
export HTTP_KEEPALIVE_CONNECTIONS="$${HTTP_KEEPALIVE_CONNECTIONS:-5}"
if ! command -v cargo >/dev/null 2>&1; then
echo "=> 未找到 cargo,无法启动 aether-gateway。请先安装 Rust toolchain。"
exit 1
fi
if ! command -v curl >/dev/null 2>&1; then
echo "=> 未找到 curl,无法检查 aether-gateway 健康状态。请先安装 curl。"
exit 1
fi
if [ -z "$${RUSTC_WRAPPER:-}" ] && command -v sccache >/dev/null 2>&1; then
export RUSTC_WRAPPER="$$(command -v sccache)"
echo "=> 启用 Rust 编译缓存: $${RUSTC_WRAPPER}"
fi
if ! ensure_dev_infra; then
exit 1
fi
echo "=> 编译 aether-gateway..."
cargo build -p aether-gateway --bin aether-gateway
GATEWAY_PID=""
GATEWAY_LOG_DIR=""
GATEWAY_LOG_FILE=""
STARTUP_WAIT_EARLY_EXIT=false
create_gateway_log_file
echo "=> 启动 aether-gateway (Rust frontdoor: 0.0.0.0:$${APP_PORT})..."
echo "=> 日志过滤: $${RUST_LOG}"
echo "=> 执行命令: target/debug/aether-gateway --app-port $${APP_PORT}"
target/debug/aether-gateway --app-port "$${APP_PORT}" > >(
tee -a "$${GATEWAY_LOG_FILE}"
) 2>&1 &
GATEWAY_PID=$$!
if ! wait_for_startup "$${GATEWAY_PID}" "$${GATEWAY_STARTUP_TIMEOUT_SECONDS}" "aether-gateway" curl -sf "http://127.0.0.1:$${APP_PORT}/_gateway/health"; then
if [ "$${STARTUP_WAIT_EARLY_EXIT}" = "true" ]; then
GATEWAY_PID=""
fi
exit 1
fi
if wait "$${GATEWAY_PID}"; then
gateway_exit_code=0
else
gateway_exit_code=$$?
fi
GATEWAY_PID=""
if [ "$${gateway_exit_code}" -ne 130 ] && [ "$${gateway_exit_code}" -ne 143 ]; then
echo "=> aether-gateway 运行失败并已退出,请检查上面的日志。"
print_startup_failure_hint "$${GATEWAY_LOG_FILE}"
fi
exit "$${gateway_exit_code}"
endef
export DEV_BACKEND_SCRIPT
define DEV_SCRIPT
set -euo pipefail
backend_pid=""
frontend_pid=""
cleanup() {
local status="$${1:-0}"
trap - INT TERM EXIT
if [ -n "$${backend_pid}" ] || [ -n "$${frontend_pid}" ]; then
echo ""
echo "=> 停止本地开发服务..."
if [ -n "$${backend_pid}" ]; then
kill "$${backend_pid}" >/dev/null 2>&1 || true
wait "$${backend_pid}" >/dev/null 2>&1 || true
fi
if [ -n "$${frontend_pid}" ]; then
kill "$${frontend_pid}" >/dev/null 2>&1 || true
wait "$${frontend_pid}" >/dev/null 2>&1 || true
fi
fi
exit "$${status}"
}
wait_for_backend_ready() {
while :; do
if curl -sf "http://127.0.0.1:$${APP_PORT}/_gateway/health" >/dev/null 2>&1; then
return 0
fi
if ! kill -0 "$${backend_pid}" >/dev/null 2>&1; then
if wait "$${backend_pid}"; then
status=0
else
status=$$?
fi
if [ "$${status}" -ne 0 ]; then
echo "=> 后端进程已退出 (status $${status})"
else
echo "=> 后端进程已退出"
fi
backend_pid=""
cleanup "$${status}"
fi
sleep 0.2
done
}
trap 'cleanup 130' INT
trap 'cleanup 143' TERM
trap 'cleanup $$?' EXIT
if [ -f .env ]; then
set -a
source .env
set +a
fi
export APP_PORT="$${APP_PORT:-8084}"
echo "=> 启动后端: 先编译 aether-gateway,再运行 target/debug/aether-gateway --app-port $${APP_PORT:-8084}"
/bin/bash -euo pipefail -c "$$DEV_BACKEND_SCRIPT" &
backend_pid=$$!
echo "=> 等待后端健康检查: http://127.0.0.1:$${APP_PORT}/_gateway/health"
wait_for_backend_ready
echo "=> 启动前端: cd frontend && npm run dev"
( cd frontend && exec npm run dev ) &
frontend_pid=$$!
while :; do
if ! kill -0 "$${backend_pid}" >/dev/null 2>&1; then
if wait "$${backend_pid}"; then
status=0
else
status=$$?
fi
if [ "$${status}" -ne 0 ]; then
echo "=> 后端进程已退出 (status $${status})"
else
echo "=> 后端进程已退出"
fi
backend_pid=""
cleanup "$${status}"
fi
if ! kill -0 "$${frontend_pid}" >/dev/null 2>&1; then
if wait "$${frontend_pid}"; then
status=0
else
status=$$?
fi
if [ "$${status}" -ne 0 ]; then
echo "=> 前端进程已退出 (status $${status})"
else
echo "=> 前端进程已退出"
fi
frontend_pid=""
cleanup "$${status}"
fi
sleep 1
done
endef
export DEV_SCRIPT
define DB_TASK_SCRIPT
set -euo pipefail
if [ -z "$${DB_TASK_COMMAND:-}" ] || [ -z "$${DB_TASK_LABEL:-}" ]; then
echo "=> 内部错误: DB_TASK_COMMAND / DB_TASK_LABEL 未设置"
exit 1
fi
read -r -a db_task_args <<< "$${DB_TASK_COMMAND}"
if [ "$${#db_task_args[@]}" -eq 0 ]; then
echo "=> 内部错误: DB_TASK_COMMAND 为空"
exit 1
fi
if [ ! -f .env ]; then
echo "=> 未找到 .env,请先执行: cp .env.example .env"
exit 1
fi
set -a
source .env
set +a
dotenv_has_key() {
local key="$$1"
grep -Eq "^[[:space:]]*$${key}=" .env
}
lowercase() {
printf '%s' "$$1" | tr '[:upper:]' '[:lower:]'
}
uses_postgres_database() {
local driver
local url
driver="$$(lowercase "$${AETHER_DATABASE_DRIVER:-}")"
url="$${AETHER_DATABASE_URL:-$${DATABASE_URL:-}}"
if [[ -z "$${driver}" && -z "$${url}" ]]; then
return 0
fi
[[ "$${driver}" == "postgres" || "$${driver}" == "postgresql" || "$${url}" == postgres:* || "$${url}" == postgresql:* ]]
}
if uses_postgres_database; then
export DATABASE_URL="postgresql://$${DB_USER:-postgres}:$${DB_PASSWORD:-}@$${DB_HOST:-localhost}:$${DB_PORT:-5432}/$${DB_NAME:-aether}"
if ! dotenv_has_key "AETHER_GATEWAY_DATA_POSTGRES_URL"; then
export AETHER_GATEWAY_DATA_POSTGRES_URL="$${DATABASE_URL}"
fi
fi
if ! dotenv_has_key "AETHER_GATEWAY_DATA_ENCRYPTION_KEY"; then
export AETHER_GATEWAY_DATA_ENCRYPTION_KEY="$${ENCRYPTION_KEY:-}"
fi
if ! command -v cargo >/dev/null 2>&1; then
echo "=> 未找到 cargo,无法执行 $${DB_TASK_LABEL}。请先安装 Rust toolchain。"
exit 1
fi
echo "=> 执行 $${DB_TASK_LABEL}: cargo run -p aether-gateway --bin aether-gateway -- $${db_task_args[*]}"
exec cargo run -p aether-gateway --bin aether-gateway -- "$${db_task_args[@]}"
endef
export DB_TASK_SCRIPT
dev:
@$(SHELL) -euo pipefail -c "$$DEV_SCRIPT"
dev-backend:
@$(SHELL) -euo pipefail -c "$$DEV_BACKEND_SCRIPT"
dev-frontend:
@cd frontend && npm run dev
db-status:
@DB_TASK_COMMAND="db status" DB_TASK_LABEL="数据库状态检查" $(SHELL) -euo pipefail -c "$$DB_TASK_SCRIPT"
db-prepare:
@DB_TASK_COMMAND="db prepare" DB_TASK_LABEL="数据库准备" $(SHELL) -euo pipefail -c "$$DB_TASK_SCRIPT"
migration:
@DB_TASK_COMMAND="--migrate" DB_TASK_LABEL="数据库迁移" $(SHELL) -euo pipefail -c "$$DB_TASK_SCRIPT"
backfill:
@DB_TASK_COMMAND="--apply-backfills" DB_TASK_LABEL="数据库 backfill" $(SHELL) -euo pipefail -c "$$DB_TASK_SCRIPT"
+63 -143
View File
@@ -44,201 +44,121 @@ cd Aether
# 2. 配置环境变量
cp .env.example .env
./generate_keys.sh # 生成 JWT_SECRET_KEY / ENCRYPTION_KEY, 并填入 .env
# .env 包含数据库、JWT 和数据加密密钥,先限制为仅当前用户可读写
chmod 600 .env
# 生成 JWT / 加密 / Postgres / Redis 独立随机密钥,并填入 .env
./generate_keys.sh
# 编辑 .env 设置 ADMIN_PASSWORD
# 3. 首次部署 / 更新
# 3. Docker 部署 / 更新(PostgreSQL + Redis)
docker compose pull && docker compose up -d
# 4. 默认会在 app 启动前自动执行挂起的 migration / backfill
# 如需手工控制,可在 .env 中设 AETHER_GATEWAY_AUTO_PREPARE_DATABASE=false
# 然后按需执行:
docker compose run --rm app --migrate
docker compose run --rm app --apply-backfills
# 5. 升级前备份 (可选)
docker compose exec postgres pg_dump -U postgres aether | gzip > backup_$(date +%Y%m%d_%H%M%S).sql.gz
```
### Docker Compose(本地构建镜像)
### 一键安装(PostgreSQL + Redis)
```bash
# 1. 克隆代码
git clone https://github.com/fawney19/Aether.git
cd Aether
# 2. 配置环境变量
cp .env.example .env
./generate_keys.sh # 生成 JWT_SECRET_KEY / ENCRYPTION_KEY, 并填入 .env
# 编辑 .env 设置 ADMIN_PASSWORD
# 3. 部署 / 更新(自动构建并启动)
git pull
./deploy.sh
curl -fsSL https://raw.githubusercontent.com/fawney19/Aether/main/install.sh | sudo bash -s -- --mode compose
```
### 一键安装(可选部署方式)
原生 Linux systemd / macOS launchd 安装需先准备 PostgreSQL,将连接串通过 `DATABASE_URL` 传给安装进程,并选择 `--mode single-node`;不再自动创建本地数据库文件。
安装脚本先从 `aether-rust-pioneer` 分支下载,不依赖 GitHub Release 的 `latest` 脚本地址。运行后会先选择版本,再选择部署方式。
### Nightly(每日 main 构建)
Nightly workflow 每天从 `main` 的固定 commit 构建并发布滚动的 GitHub Release `nightly`,同时推送多架构 GHCR 镜像 `ghcr.io/fawney19/aether:nightly`。Nightly 是预发布版本,适合验证最新代码,不保证与正式版相同的稳定性。滚动 Release 需要仓库保持关闭 GitHub Release immutability。
安装最新 nightly(PostgreSQL + Redis):
```bash
curl -fsSL https://raw.githubusercontent.com/fawney19/Aether/aether-rust-pioneer/install.sh | sudo bash
curl -fsSL https://raw.githubusercontent.com/fawney19/Aether/main/install.sh | sudo bash -s -- --mode compose --channel nightly
```
运行后按提示输入版本和部署方式。固定安装某个 tag 时,版本选择选 `2`,再输入类似 `v0.7.0-rc23` 的 tag。默认会安装最新预发布版本;Docker Compose 模式默认使用 `pre` 镜像通道。
如果安装目录里已经有配置,脚本会优先复用:Docker Compose 保留已有 `.env`,systemd 保留已有 `/etc/aether/aether-gateway.env`。只有首次生成新配置时才会提示输入管理员密码。
Docker Compose 用户可在部署目录的 `.env` 中设置 `APP_IMAGE=ghcr.io/fawney19/aether:nightly`,然后运行 `./update.sh` 获取下一次 nightly。二进制部署请沿用已有 PostgreSQL 环境配置,并使用 `--mode single-node --channel nightly` 重新运行安装脚本升级;当前管理后台的在线更新列表只跟踪正式版/RC/Beta,不会自动提示下一次 nightly。
```text
Choose Aether version:
1) Latest pre release
2) Exact tag, for example v0.7.0-rc23
## 本地开发
Enter choice [1]:
Choose Aether deployment mode:
1) Docker Compose: app + Postgres + Redis
2) Single-node service: systemd + SQLite + in-process runtime
3) Cluster node service: systemd + shared database + Redis
Enter choice [2]:
```
安装后的常用命令:
依赖 Docker、Rust toolchain、Node.js 和 make。
首次启动前需要在 `.env` 中设置 `ADMIN_PASSWORD`,用于创建本地管理员。
```bash
sudo systemctl status aether-gateway --no-pager
sudo journalctl -u aether-gateway -f
sudo systemctl restart aether-gateway
make dev
```
默认单机数据和日志都在安装目录内:
```text
/opt/aether/data/aether.db
/opt/aether/logs
```
多节点不能使用 SQLite 或 `AETHER_RUNTIME_BACKEND=memory`。如果先只生成了多节点模板,编辑 `/etc/aether/aether-gateway.env` 后重跑安装脚本即可:
```env
AETHER_GATEWAY_DEPLOYMENT_TOPOLOGY=multi-node
AETHER_GATEWAY_NODE_ROLE=frontdoor
DATABASE_URL=postgresql://...
REDIS_URL=redis://...
```
### 本地开发
`make dev` 会同时启动后端 `aether-gateway` 和前端 `frontend` 的 Vite dev server。需要单独启动时可使用 `make dev-backend` 或 `make dev-frontend`。
Postgres / Redis 本地依赖未就绪时,`make dev` 会自动执行 `docker compose up -d postgres redis`。
`make dev` 会先完成后端编译,再开始计算服务健康检查超时。数据库 schema 和必要的派生数据准备也会在启动时自动完成;通常不需要手动区分 migration 与 backfill。升级不会主动重写或清除已有业务历史记录,新写入会直接遵循当前的数据持久化策略。排查或部署前预执行时可使用:
```bash
# 启动依赖
docker compose -f docker-compose.build.yml up -d postgres redis
# 数据库迁移(仅在已有数据库引入新 migration 时需要)
./dev.sh --migrate
# 数据回填
./dev.sh --apply-backfills
# 后端
./dev.sh
# 前端
cd frontend && npm install && npm run dev
make db-status
make db-prepare
```
`./dev.sh` 现在只保留一种本地模式:
## Codex 远程协同
| 角色 | 本地地址 | 说明 |
|------|----------|------|
| Rust frontdoor | 默认 `http://localhost:8084` | `aether-gateway`,本地唯一公开入口;实际端口由 `APP_PORT` 控制 |
`aether-vscodex/` 是独立的 VS Code Codex 协同模块:同步模式跟随 VS Code 官方 Codex 面板当前会话且不另起进程;异步模式使用独立 app-server,让浏览器自行列出、恢复、新建和切换会话。两种模式都能从本机 URL 或 Aether 云端查看输出、发送消息和处理授权,模块内的 Vue 前端提供中英文界面。
本地默认链路是:
安装、云端配对和安全边界请参阅 [`aether-vscodex/README.md`](aether-vscodex/README.md)。
```text
client -> rust frontdoor (aether-gateway) -> execution_runtime/provider transport
```
## Aether Tunnel (可选)
其中:
- `aether-gateway` 负责公开入口、健康检查、格式转换、本地执行 runtime,以及当前已迁到 Rust 的 frontdoor/control/background 路径。
- `./dev.sh` 不再启动 Python 宿主;未下沉到 Rust 的 legacy 路由会直接失败。
- `./dev.sh --migrate` 会复用 `.env` 里的数据库配置,显式执行一次数据库迁移后退出。
- `./dev.sh` 默认把 `AETHER_GATEWAY_VIDEO_TASK_TRUTH_SOURCE_MODE` 设为 `rust-authoritative`,避免本地还依赖 Python sync report 语义。
- 空库首次启动会自动初始化到当前 baseline。
- `aether-gateway` 默认启动不会自动应用后续 schema migration;如果数据库版本落后,服务会拒绝启动,并提示先执行 `aether-gateway --migrate`。
- 仓库自带的 `docker-compose.yml` 和 `docker-compose.build.yml` 都已把 `AETHER_GATEWAY_AUTO_PREPARE_DATABASE` 设为默认开启,因此无论是预构建镜像部署还是 `./deploy.sh` / 本地构建 compose,常规启动都会在监听端口前自动执行挂起的 migration 和 backfill。
## Aether Proxy (可选)
Aether Proxy 是配套的正向代理节点,部署在海外 VPS 上,为墙内的 Aether 实例中转 API 流量。或者部署在其他服务器为指定的提供商、账号、Key使用不同的节点访问。支持 TUI 向导一键配置、systemd 服务管理、TLS 加密、DNS 缓存及连接池调优。
Aether Tunnel 是配套的正向代理节点,部署在海外 VPS 上,为墙内的 Aether 实例中转 API 流量。
- Docker Compose 部署或下载预编译二进制直接运行
- 通过 `aether-proxy setup` 完成交互式配置,自动注册为系统服务
- 详细文档见 [apps/aether-proxy/README.md](apps/aether-proxy/README.md)
- 提供 macOS/Linux 与 Windows 一键脚本,自动下载最新 `tunnel-v*` 制品并向现有 `aether-tunnel.toml` 追加 `[[servers]]`
- 通过 `aether-tunnel setup` 完成交互式配置,自动注册为系统服务
- 详细文档见 [apps/aether-tunnel/README.md](apps/aether-tunnel/README.md)
## API 文档
- Embeddings: [OpenAI compatible `POST /v1/embeddings`](docs/api/embeddings.md)
- Rerank: [OpenAI/Jina compatible `POST /v1/rerank`](docs/api/rerank.md)
- Responses WebSocket mode: [protocol and Aether behavior](docs/WebSocket-Mode.md)
- WebSocket probes: [Codex](docs/operations/codex-responses-websocket-probe.md) · [OpenAI Responses](docs/operations/openai-responses-websocket-probe.md)
## 环境变量
部署建议:
- Docker Compose:根目录 [`.env.example`](.env.example)
- systemd 二进制部署:使用根目录 `install.sh` 生成 `/etc/aether/aether-gateway.env`
当前主链路真正要关注的是这组变量:
- `APP_PORT`:`aether-gateway` 唯一监听端口,固定绑定 `0.0.0.0:${APP_PORT}`
- `AETHER_DATABASE_DRIVER` / `AETHER_DATABASE_URL`:二进制单机部署可用 `sqlite`,例如 `sqlite:///opt/aether/data/aether.db`
- `DATABASE_URL` / `REDIS_URL`:`aether-gateway` 直接读取的共享后端连接串;多节点必须配置共享数据库和 Redis
- `AETHER_RUNTIME_BACKEND=memory|redis`:单机 SQLite 默认用 `memory`;多节点必须用 Redis
- `AETHER_GATEWAY_AUTO_PREPARE_DATABASE`:常规启动前自动执行挂起的 schema migration 和 backfill;仓库自带的 `docker-compose.yml` 和 `docker-compose.build.yml` 默认开启
- `DATABASE_URL`:PostgreSQL 连接串,例如 `postgresql://USER:PASSWORD@HOST:5432/aether`
- `AETHER_GATEWAY_DATA_POSTGRES_MIN_CONNECTIONS` / `AETHER_GATEWAY_DATA_POSTGRES_MAX_CONNECTIONS`:数据库连接池手动覆盖值;未配置时 PostgreSQL 按每核 `4` 条自动推导,总池范围为 `32-100`。该预算按进程计算,多实例部署应按数据库连接上限显式分配
- `AETHER_GATEWAY_MAX_IN_FLIGHT_REQUESTS`:单实例请求并发上限;未配置时按 CPU 自动推导(基础范围 `512-65536`),低文件描述符预算时会进一步下调
- `AETHER_GATEWAY_REQUEST_BODY_BUFFER_BUDGET_MB`:单实例同时读取和解压请求体的加权内存预算,默认 `256MB`
- `AETHER_GATEWAY_REQUEST_BODY_READ_TIMEOUT_MS`:可选的请求体完整读取超时;默认或显式设为 `0` 时关闭,非零值限制在 `1000-600000ms`
- `AETHER_MAX_REQUEST_BODY_MB`:单请求解压后请求体上限,默认 `256MB`;显式设为 `0` 表示不再收紧默认值,但仍受 `256MB` 安全硬上限约束
- `AETHER_MAX_INTERNAL_BUFFERED_BODY_MB`:heartbeat、管理探测等内部整包响应体上限,默认 `64MB`;显式设为 `0` 表示不再收紧默认值,但仍受 `256MB` 安全硬上限约束
- `AETHER_TUNNEL_NODE_STATUS_QUEUE_CAPACITY`:隧道节点状态上报队列容量,默认 `1024`;满载时拒绝新事件,避免控制面故障导致无界内存增长
- `AETHER_TUNNEL_RELAY_ALLOW_PRIVATE_TARGETS`:跨网关 owner relay 解析到私有/保留地址时的显式运维开关,默认关闭;仅当多网关 relay URL 是受控的内网 HTTPS 地址时设置为 `true`。它不改变普通 provider 请求的 DNS/代理策略,也不允许明文 HTTP 非 loopback relay
- `AETHER_TUNNEL_RELAY_PRIVATE_HOST_ALLOWLIST`:更窄的 owner relay 私网例外,填写逗号分隔的精确主机名(例如 `gateway-a.internal,gateway-b.internal`,忽略大小写和末尾点);仅这些主机解析出的私有地址会被允许,并且请求仍使用解析后地址 pin。不要填写通配符或 `.internal` 这类后缀
- `AETHER_INTERNAL_GATEWAY_AUTH_SECRET`:旧版 `/api/internal/gateway/*` 高权限控制面的独立 HMAC 密钥,至少 `32` 字节;未配置时该控制面返回 `404`。不要复用 JWT、数据加密或 tunnel relay 密钥,多节点必须使用同一值及共享 Redis 防重放
- `AETHER_GATEWAY_SECURITY_CACHE_TTL_MS`:IP 黑白名单本地缓存时间,默认 `1000ms`,写操作会主动失效相关缓存
- `AETHER_MAX_REDACTED_SYNC_RESPONSE_BODY_MB`:PII 恢复同步响应缓冲上限,默认 `64MB`;显式设为 `0` 表示不再收紧默认值,但仍受 `256MB` 安全硬上限约束
- `REDIS_URL`:Redis 连接串;仅 Postgres + Redis 的 Docker Compose 部署需要配置
- `AETHER_RUNTIME_BACKEND=memory|redis`:运行时缓存/协调后端。配置 Redis 时使用 `redis`,否则使用 `memory`;多节点部署和需要跨 gateway 重启恢复 OpenAI Responses continuation history 的部署必须使用共享 Redis
- `AETHER_GATEWAY_DATABASE_MODE=auto|verify-only`:数据库启动策略,默认 `auto`,自动完成挂起的 schema migration 和 backfill;`verify-only` 仅检查并在数据库落后时拒绝启动
- `AETHER_GATEWAY_AUTO_PREPARE_DATABASE`:旧版兼容开关;新配置请使用 `AETHER_GATEWAY_DATABASE_MODE`
- `JWT_SECRET_KEY` / `ENCRYPTION_KEY`:认证和敏感数据加密所需密钥
- `AETHER_BACKUP_ENCRYPTION_KEY`:推荐的 S3 备份独立加密密钥;缺省回退到 `ENCRYPTION_KEY`。新备份使用带 key ID 的 AES-256-GCM v2 envelope,轮换前必须保留旧密钥
- `API_KEY_PREFIX`:用户和管理员新建 API Key 时使用的前缀,默认 `sk`
- `ADMIN_USERNAME` / `ADMIN_PASSWORD` / `ADMIN_EMAIL`:首次启动时自举首个本地管理员;`install.sh` 会提示输入管理员密码
- `CORS_ORIGINS` / `CORS_ALLOW_CREDENTIALS`:前端跨域来源控制;如果要跨域带登录 Cookie,`CORS_ORIGINS` 不能写 `*`
- `AETHER_GATEWAY_DEPLOYMENT_TOPOLOGY=single-node|multi-node`
- `AETHER_GATEWAY_NODE_ROLE=all|frontdoor|background`
- `RUST_LOG`:Rust 日志过滤,例如 `aether_gateway=info`、`aether_gateway=debug,sqlx=warn`
- Docker Compose 的 `DB_PASSWORD` / `REDIS_PASSWORD` 默认使用 `aether`
- `DB_PASSWORD` / `REDIS_PASSWORD`:Docker Compose 后端密码,首次安装时分别随机生成;手工部署必须替换示例占位值,不要互相复用
systemd 的 `.env` 必须保持简单 `KEY=VALUE` 形式,不要写 `export`、`${VAR}` 或命令替换。
### S3 备份离线恢复
## Q&A
### Q: 如何开启/关闭请求体记录?
管理员在 **系统设置** 中配置日志记录的详细程度:
| 级别 | 记录内容 |
|------|----------|
| Base | 基本请求信息 |
| Headers | Base + 请求头 |
| Full | Headers + 请求体 |
### Q: 更新出问题如何回滚?
**有备份的情况(推荐):**
先从 S3 下载完整的 `.json.zst.aes256gcm` 对象,再使用原始的完整 S3 object key 做认证解密。恢复工具只验证并输出本地 JSON,不会直接写数据库;数据库导入仍应在维护窗口通过管理端完成。
```bash
# Docker Compose:
# 1. 切回旧镜像 tag / digest
# 2. 恢复 Postgres 备份
# 3. 再启动 app
# systemd:
# 1. 把 /opt/aether/current 切回旧 release
# 2. systemctl restart aether-gateway
# 3. 如果升级包含数据库结构变更,再恢复 Postgres 备份
AETHER_BACKUP_ENCRYPTION_KEY='原备份密钥' \
cargo run -p aether-gateway --bin aether-backup-restore -- \
--input ./backup.json.zst.aes256gcm \
--object-key 'aether/backups/aether-data-backup-20260822-010000.json.zst.aes256gcm' \
--output ./restored-backup.json
```
> 可以在升级前通过 `docker inspect ghcr.io/fawney19/aether:latest --format '{{index .RepoDigests 0}}'` 记录当前镜像 digest,方便回滚时使用。
工具默认拒绝覆盖,输出采用原子写并在 Unix 上设置为 `0600`;Unix 可用 `--overwrite` 原子替换,Windows 为避免非原子删除窗口会要求选择新输出路径。密钥不能作为命令行参数。可使用 `AETHER_BACKUP_ENCRYPTION_KEY`、兼容用 `AETHER_GATEWAY_DATA_ENCRYPTION_KEY` / `ENCRYPTION_KEY`、受保护的 `--key-file`,或 `AETHER_BACKUP_KEYRING_FILE`。Keyring JSON 格式为 `{"version":1,"keys":["当前或历史 v2 secret"],"legacy_v1":["旧 v1 secret"]}`;条目也可写成 `{"secret":"..."}`(兼容字段名 `key`)。也可由 `AETHER_BACKUP_HISTORICAL_KEYS_JSON` 提供同一结构。密钥文件必须是非符号链接的普通文件,Unix 下权限需为 `0600` 或更严格。
**没有备份的情况:**
当前不应该再依赖旧的 `alembic downgrade` 路线。空库首次启动会自动初始化;如果开启 `AETHER_GATEWAY_AUTO_PREPARE_DATABASE=true`(仓库自带的两份 compose 默认都如此),常规服务启动也会自动应用挂起的 migration/backfill。无论是否自动执行,只要本次发布带来了不可逆的数据结构变化,没有备份就不能保证安全回滚。因此升级前强烈建议先备份 `Postgres`。
默认限制密文为 `512MiB`、解压后 JSON 为 `1GiB`,可通过受限的 `--max-encrypted-mib` / `--max-json-mib` 调整。网关最多扫描同一备份前缀下 10,000 个对象,并且不会自动删除 S3 对象:`backup_s3_retention_count` 只用于报告超出保留数量的清理候选。旧明文备份在创建并验证加密副本后仍会保留,必须通过 bucket lifecycle 或支持版本条件的外部清理工具移除;启用 Versioning 时还需清理 noncurrent versions,Object Lock/retention 可能阻止物理删除。
---
@@ -256,4 +176,4 @@ systemd 的 `.env` 必须保持简单 `KEY=VALUE` 形式,不要写 `export`、
## Star History
[![Star History Chart](https://api.star-history.com/svg?repos=fawney19/Aether&type=Date)](https://star-history.com/#fawney19/Aether&Date)
[![Star History Chart](https://api.star-history.com/svg?repos=fawney19/Aether&type=date&legend=top-left)](https://www.star-history.com/?repos=fawney19%2FAether&type=date&legend=top-left)
-14
View File
@@ -1,14 +0,0 @@
"""AI Proxy
A proxy server that enables AI models to work with multiple API providers.
"""
# 注意: dotenv 加载已统一移至 src/config/settings.py
# 不要在此处重复加载
try:
from src._version import __version__
except ImportError:
__version__ = "0.0.0.dev0"
__author__ = "AI Proxy"
-24
View File
@@ -1,24 +0,0 @@
# file generated by vcs-versioning
# don't change, don't track in version control
from __future__ import annotations
__all__ = [
"__version__",
"__version_tuple__",
"version",
"version_tuple",
"__commit_id__",
"commit_id",
]
version: str
__version__: str
__version_tuple__: tuple[int | str, ...]
version_tuple: tuple[int | str, ...]
commit_id: str | None
__commit_id__: str | None
__version__ = version = '0.6.4.dev6+gddf18fed9.d20260331'
__version_tuple__ = version_tuple = (0, 6, 4, 'dev6', 'gddf18fed9.d20260331')
__commit_id__ = commit_id = None
-51
View File
@@ -1,51 +0,0 @@
# Alembic 配置文件
# 用于数据库版本化迁移
[alembic]
# 迁移脚本存放目录
script_location = alembic
# 模板文件
file_template = %%(year)d%%(month).2d%%(day).2d_%%(hour).2d%%(minute).2d_%%(rev)s_%%(slug)s
# 时区(用于生成迁移文件的时间戳)
timezone = UTC
# 数据库连接 URL(会被 env.py 从环境变量覆盖)
# Docker 环境中会从 DATABASE_URL 环境变量读取
sqlalchemy.url = postgresql://postgres:${DB_PASSWORD}@localhost:5432/aether
# 日志配置
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARN
handlers = console
qualname =
[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S
-121
View File
@@ -1,121 +0,0 @@
"""
Alembic 环境配置
用于数据库迁移的运行时环境设置
"""
import os
import sys
from logging.config import fileConfig
from pathlib import Path
from sqlalchemy import engine_from_config, pool, text
from alembic import context
# 添加项目根目录到 Python 路径
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
# 加载 .env 文件(本地开发时需要)
try:
from dotenv import load_dotenv
env_file = Path(__file__).parent.parent / ".env"
if env_file.exists():
load_dotenv(env_file)
except ImportError:
pass
# 导入所有数据库模型(确保 Alembic 能检测到所有表)
from src.models.database import Base
# Alembic Config 对象
config = context.config
# 从环境变量获取数据库 URL
# 优先使用 DATABASE_URL,否则从 DB_PASSWORD 自动构建(与 docker compose 保持一致)
database_url = os.getenv("DATABASE_URL")
if not database_url:
db_password = os.getenv("DB_PASSWORD", "")
db_host = os.getenv("DB_HOST", "localhost")
db_port = os.getenv("DB_PORT", "5432")
db_name = os.getenv("DB_NAME", "aether")
db_user = os.getenv("DB_USER", "postgres")
database_url = f"postgresql://{db_user}:{db_password}@{db_host}:{db_port}/{db_name}"
config.set_main_option("sqlalchemy.url", database_url)
# 配置日志
if config.config_file_name is not None:
fileConfig(config.config_file_name)
# 目标元数据(包含所有表定义)
target_metadata = Base.metadata
# PostgreSQL 全局迁移锁,避免多进程并发执行 Alembic 导致竞态(重复加列/索引等)
# 使用会话级 advisory lock(pg_advisory_lock),在迁移完成后手动释放。
# ID 由 crc32("aether-alembic-migration") 拼接生成,仅需全局唯一即可。
MIGRATION_ADVISORY_LOCK_ID = 582694137405821
def run_migrations_offline() -> None:
"""
离线模式运行迁移
在离线模式下,不需要连接数据库,
只生成 SQL 脚本
"""
url = config.get_main_option("sqlalchemy.url")
context.configure(
url=url,
target_metadata=target_metadata,
literal_binds=True,
dialect_opts={"paramstyle": "named"},
compare_type=True, # 比较列类型变更
compare_server_default=True, # 比较默认值变更
)
with context.begin_transaction():
context.run_migrations()
def run_migrations_online() -> None:
"""
在线模式运行迁移
在线模式下,直接连接数据库执行迁移
"""
connectable = engine_from_config(
config.get_section(config.config_ini_section, {}),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
with connectable.connect() as connection:
try:
# 使用会话级 advisory lock(非事务级),避免干扰 Alembic 的事务管理。
# pg_advisory_lock 在会话结束时自动释放,不受 COMMIT/ROLLBACK 影响。
if connection.dialect.name == "postgresql":
connection.execute(
text("SELECT pg_advisory_lock(:lock_id)"),
{"lock_id": MIGRATION_ADVISORY_LOCK_ID},
)
connection.commit()
context.configure(
connection=connection,
target_metadata=target_metadata,
compare_type=True,
compare_server_default=True,
transaction_per_migration=True, # 每个迁移文件独立事务,完成即提交
)
with context.begin_transaction():
context.run_migrations()
except Exception:
raise
# 根据模式选择运行方式
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()
-26
View File
@@ -1,26 +0,0 @@
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision | comma,n}
Create Date: ${create_date}
"""
from alembic import op
import sqlalchemy as sa
${imports if imports else ""}
# revision identifiers, used by Alembic.
revision = ${repr(up_revision)}
down_revision = ${repr(down_revision)}
branch_labels = ${repr(branch_labels)}
depends_on = ${repr(depends_on)}
def upgrade() -> None:
"""应用迁移:升级到新版本"""
${upgrades if upgrades else "pass"}
def downgrade() -> None:
"""回滚迁移:降级到旧版本"""
${downgrades if downgrades else "pass"}
@@ -1,775 +0,0 @@
"""Baseline migration - all tables consolidated
Revision ID: 20251210_baseline
Revises:
Create Date: 2024-12-10
This is the consolidated baseline migration that creates all tables from scratch.
Includes all schema changes up to circuit breaker v2.
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
# revision identifiers
revision = "20251210_baseline"
down_revision = None
branch_labels = None
depends_on = None
def upgrade() -> None:
# Create ENUM types (with IF NOT EXISTS for idempotency)
op.execute("DO $$ BEGIN CREATE TYPE userrole AS ENUM ('admin', 'user'); EXCEPTION WHEN duplicate_object THEN NULL; END $$")
op.execute(
"DO $$ BEGIN CREATE TYPE providerbillingtype AS ENUM ('monthly_quota', 'pay_as_you_go', 'free_tier'); EXCEPTION WHEN duplicate_object THEN NULL; END $$"
)
# ==================== users ====================
op.create_table(
"users",
sa.Column("id", sa.String(36), primary_key=True, index=True),
sa.Column("email", sa.String(255), unique=True, index=True, nullable=False),
sa.Column("username", sa.String(100), unique=True, index=True, nullable=False),
sa.Column("password_hash", sa.String(255), nullable=False),
sa.Column(
"role",
postgresql.ENUM("admin", "user", name="userrole", create_type=False),
nullable=False,
server_default="user",
),
sa.Column("allowed_providers", sa.JSON, nullable=True),
sa.Column("allowed_endpoints", sa.JSON, nullable=True),
sa.Column("allowed_models", sa.JSON, nullable=True),
sa.Column("model_capability_settings", sa.JSON, nullable=True),
sa.Column("quota_usd", sa.Float, nullable=True),
sa.Column("used_usd", sa.Float, server_default="0.0"),
sa.Column("total_usd", sa.Float, server_default="0.0"),
sa.Column("is_active", sa.Boolean, server_default="true", nullable=False),
sa.Column("is_deleted", sa.Boolean, server_default="false", nullable=False),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column("last_login_at", sa.DateTime(timezone=True), nullable=True),
)
# ==================== providers ====================
op.create_table(
"providers",
sa.Column("id", sa.String(36), primary_key=True, index=True),
sa.Column("name", sa.String(100), unique=True, index=True, nullable=False),
sa.Column("display_name", sa.String(100), nullable=False),
sa.Column("description", sa.Text, nullable=True),
sa.Column("website", sa.String(500), nullable=True),
sa.Column(
"billing_type",
postgresql.ENUM(
"monthly_quota", "pay_as_you_go", "free_tier", name="providerbillingtype", create_type=False
),
nullable=False,
server_default="pay_as_you_go",
),
sa.Column("monthly_quota_usd", sa.Float, nullable=True),
sa.Column("monthly_used_usd", sa.Float, server_default="0.0"),
sa.Column("quota_reset_day", sa.Integer, server_default="30"),
sa.Column("quota_last_reset_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("quota_expires_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("rpm_limit", sa.Integer, nullable=True),
sa.Column("rpm_used", sa.Integer, server_default="0"),
sa.Column("rpm_reset_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("provider_priority", sa.Integer, server_default="100"),
sa.Column("is_active", sa.Boolean, server_default="true", nullable=False),
sa.Column("rate_limit", sa.Integer, nullable=True),
sa.Column("concurrent_limit", sa.Integer, nullable=True),
sa.Column("config", sa.JSON, nullable=True),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
)
# ==================== global_models ====================
op.create_table(
"global_models",
sa.Column("id", sa.String(36), primary_key=True, index=True),
sa.Column("name", sa.String(100), unique=True, index=True, nullable=False),
sa.Column("display_name", sa.String(100), nullable=False),
sa.Column("description", sa.Text, nullable=True),
sa.Column("icon_url", sa.String(500), nullable=True),
sa.Column("official_url", sa.String(500), nullable=True),
sa.Column("default_price_per_request", sa.Float, nullable=True),
sa.Column("default_tiered_pricing", sa.JSON, nullable=False),
sa.Column("default_supports_vision", sa.Boolean, server_default="false", nullable=True),
sa.Column("default_supports_function_calling", sa.Boolean, server_default="false", nullable=True),
sa.Column("default_supports_streaming", sa.Boolean, server_default="true", nullable=True),
sa.Column("default_supports_extended_thinking", sa.Boolean, server_default="false", nullable=True),
sa.Column("default_supports_image_generation", sa.Boolean, server_default="false", nullable=True),
sa.Column("supported_capabilities", sa.JSON, nullable=True),
sa.Column("is_active", sa.Boolean, server_default="true", nullable=False),
sa.Column("usage_count", sa.Integer, server_default="0", nullable=False, index=True),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
)
# ==================== api_keys ====================
op.create_table(
"api_keys",
sa.Column("id", sa.String(36), primary_key=True, index=True),
sa.Column(
"user_id", sa.String(36), sa.ForeignKey("users.id", ondelete="CASCADE"), nullable=False
),
sa.Column("key_hash", sa.String(64), unique=True, index=True, nullable=False),
sa.Column("key_encrypted", sa.Text, nullable=True),
sa.Column("name", sa.String(100), nullable=True),
sa.Column("total_requests", sa.Integer, server_default="0"),
sa.Column("total_cost_usd", sa.Float, server_default="0.0"),
sa.Column("balance_used_usd", sa.Float, server_default="0.0"),
sa.Column("current_balance_usd", sa.Float, nullable=True),
sa.Column("is_standalone", sa.Boolean, server_default="false", nullable=False),
sa.Column("allowed_providers", sa.JSON, nullable=True),
sa.Column("allowed_endpoints", sa.JSON, nullable=True),
sa.Column("allowed_api_formats", sa.JSON, nullable=True),
sa.Column("allowed_models", sa.JSON, nullable=True),
sa.Column("rate_limit", sa.Integer, server_default="100"),
sa.Column("concurrent_limit", sa.Integer, server_default="5", nullable=True),
sa.Column("force_capabilities", sa.JSON, nullable=True),
sa.Column("is_active", sa.Boolean, server_default="true", nullable=False),
sa.Column("last_used_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("auto_delete_on_expiry", sa.Boolean, server_default="false", nullable=False),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
)
# ==================== provider_endpoints ====================
op.create_table(
"provider_endpoints",
sa.Column("id", sa.String(36), primary_key=True, index=True),
sa.Column(
"provider_id",
sa.String(36),
sa.ForeignKey("providers.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column("api_format", sa.String(50), nullable=False),
sa.Column("base_url", sa.String(500), nullable=False),
sa.Column("headers", sa.JSON, nullable=True),
sa.Column("timeout", sa.Integer, server_default="300"),
sa.Column("max_retries", sa.Integer, server_default="3"),
sa.Column("max_concurrent", sa.Integer, nullable=True),
sa.Column("rate_limit", sa.Integer, nullable=True),
sa.Column("is_active", sa.Boolean, server_default="true", nullable=False),
sa.Column("custom_path", sa.String(200), nullable=True),
sa.Column("config", sa.JSON, nullable=True),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.UniqueConstraint("provider_id", "api_format", name="uq_provider_api_format"),
)
op.create_index(
"idx_endpoint_format_active", "provider_endpoints", ["api_format", "is_active"]
)
# ==================== models ====================
op.create_table(
"models",
sa.Column("id", sa.String(36), primary_key=True, index=True),
sa.Column(
"provider_id", sa.String(36), sa.ForeignKey("providers.id"), nullable=False
),
sa.Column(
"global_model_id",
sa.String(36),
sa.ForeignKey("global_models.id"),
nullable=False,
index=True,
),
sa.Column("provider_model_name", sa.String(200), nullable=False),
sa.Column("price_per_request", sa.Float, nullable=True),
sa.Column("tiered_pricing", sa.JSON, nullable=True),
sa.Column("supports_vision", sa.Boolean, nullable=True),
sa.Column("supports_function_calling", sa.Boolean, nullable=True),
sa.Column("supports_streaming", sa.Boolean, nullable=True),
sa.Column("supports_extended_thinking", sa.Boolean, nullable=True),
sa.Column("supports_image_generation", sa.Boolean, nullable=True),
sa.Column("is_active", sa.Boolean, server_default="true", nullable=False),
sa.Column("is_available", sa.Boolean, server_default="true"),
sa.Column("config", sa.JSON, nullable=True),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.UniqueConstraint("provider_id", "provider_model_name", name="uq_provider_model"),
)
# ==================== model_mappings ====================
op.create_table(
"model_mappings",
sa.Column("id", sa.String(36), primary_key=True, index=True),
sa.Column("source_model", sa.String(200), nullable=False, index=True),
sa.Column(
"target_global_model_id",
sa.String(36),
sa.ForeignKey("global_models.id", ondelete="CASCADE"),
nullable=False,
index=True,
),
sa.Column(
"provider_id", sa.String(36), sa.ForeignKey("providers.id"), nullable=True, index=True
),
sa.Column("mapping_type", sa.String(20), nullable=False, server_default="alias", index=True),
sa.Column("is_active", sa.Boolean, server_default="true", nullable=False),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.UniqueConstraint("source_model", "provider_id", name="uq_model_mapping_source_provider"),
)
# ==================== provider_api_keys ====================
op.create_table(
"provider_api_keys",
sa.Column("id", sa.String(36), primary_key=True, index=True),
sa.Column(
"endpoint_id",
sa.String(36),
sa.ForeignKey("provider_endpoints.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column("api_key", sa.String(500), nullable=False),
sa.Column("name", sa.String(100), nullable=False),
sa.Column("note", sa.String(500), nullable=True),
sa.Column("rate_multiplier", sa.Float, server_default="1.0", nullable=False),
sa.Column("internal_priority", sa.Integer, server_default="50"),
sa.Column("global_priority", sa.Integer, nullable=True),
sa.Column("max_concurrent", sa.Integer, nullable=True),
sa.Column("rate_limit", sa.Integer, nullable=True),
sa.Column("daily_limit", sa.Integer, nullable=True),
sa.Column("monthly_limit", sa.Integer, nullable=True),
sa.Column("allowed_models", sa.JSON, nullable=True),
sa.Column("capabilities", sa.JSON, nullable=True),
sa.Column("learned_max_concurrent", sa.Integer, nullable=True),
sa.Column("concurrent_429_count", sa.Integer, server_default="0", nullable=False),
sa.Column("rpm_429_count", sa.Integer, server_default="0", nullable=False),
sa.Column("last_429_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("last_429_type", sa.String(50), nullable=True),
sa.Column("last_concurrent_peak", sa.Integer, nullable=True),
sa.Column("adjustment_history", sa.JSON, nullable=True),
# Sliding window fields (replaces high_utilization_start)
sa.Column("utilization_samples", sa.JSON, nullable=True),
sa.Column("last_probe_increase_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("health_score", sa.Float, server_default="1.0"),
sa.Column("consecutive_failures", sa.Integer, server_default="0"),
sa.Column("last_failure_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("cache_ttl_minutes", sa.Integer, server_default="5", nullable=False),
sa.Column("max_probe_interval_minutes", sa.Integer, server_default="32", nullable=False),
sa.Column("circuit_breaker_open", sa.Boolean, server_default="false", nullable=False),
sa.Column("circuit_breaker_open_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("next_probe_at", sa.DateTime(timezone=True), nullable=True),
# Circuit breaker v2 fields
sa.Column("request_results_window", sa.JSON, nullable=True),
sa.Column("half_open_until", sa.DateTime(timezone=True), nullable=True),
sa.Column("half_open_successes", sa.Integer, server_default="0", nullable=True),
sa.Column("half_open_failures", sa.Integer, server_default="0", nullable=True),
sa.Column("request_count", sa.Integer, server_default="0"),
sa.Column("success_count", sa.Integer, server_default="0"),
sa.Column("error_count", sa.Integer, server_default="0"),
sa.Column("total_response_time_ms", sa.Integer, server_default="0"),
sa.Column("last_used_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("last_error_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("last_error_msg", sa.Text, nullable=True),
sa.Column("is_active", sa.Boolean, server_default="true", nullable=False),
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=True),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
)
# ==================== usage ====================
op.create_table(
"usage",
sa.Column("id", sa.String(36), primary_key=True, index=True),
sa.Column(
"user_id",
sa.String(36),
sa.ForeignKey("users.id", ondelete="SET NULL"),
nullable=True,
),
sa.Column(
"api_key_id",
sa.String(36),
sa.ForeignKey("api_keys.id", ondelete="SET NULL"),
nullable=True,
),
sa.Column("request_id", sa.String(100), unique=True, index=True, nullable=False),
sa.Column("provider", sa.String(100), nullable=False),
sa.Column("model", sa.String(100), nullable=False),
sa.Column("target_model", sa.String(100), nullable=True),
sa.Column(
"provider_id",
sa.String(36),
sa.ForeignKey("providers.id", ondelete="SET NULL"),
nullable=True,
),
sa.Column(
"provider_endpoint_id",
sa.String(36),
sa.ForeignKey("provider_endpoints.id", ondelete="SET NULL"),
nullable=True,
),
sa.Column(
"provider_api_key_id",
sa.String(36),
sa.ForeignKey("provider_api_keys.id", ondelete="SET NULL"),
nullable=True,
),
sa.Column("input_tokens", sa.Integer, server_default="0"),
sa.Column("output_tokens", sa.Integer, server_default="0"),
sa.Column("total_tokens", sa.Integer, server_default="0"),
sa.Column("cache_creation_input_tokens", sa.Integer, server_default="0"),
sa.Column("cache_read_input_tokens", sa.Integer, server_default="0"),
sa.Column("input_cost_usd", sa.Float, server_default="0.0"),
sa.Column("output_cost_usd", sa.Float, server_default="0.0"),
sa.Column("cache_cost_usd", sa.Float, server_default="0.0"),
sa.Column("cache_creation_cost_usd", sa.Float, server_default="0.0"),
sa.Column("cache_read_cost_usd", sa.Float, server_default="0.0"),
sa.Column("request_cost_usd", sa.Float, server_default="0.0"),
sa.Column("total_cost_usd", sa.Float, server_default="0.0"),
sa.Column("actual_input_cost_usd", sa.Float, server_default="0.0"),
sa.Column("actual_output_cost_usd", sa.Float, server_default="0.0"),
sa.Column("actual_cache_creation_cost_usd", sa.Float, server_default="0.0"),
sa.Column("actual_cache_read_cost_usd", sa.Float, server_default="0.0"),
sa.Column("actual_request_cost_usd", sa.Float, server_default="0.0"),
sa.Column("actual_total_cost_usd", sa.Float, server_default="0.0"),
sa.Column("rate_multiplier", sa.Float, server_default="1.0"),
sa.Column("input_price_per_1m", sa.Float, nullable=True),
sa.Column("output_price_per_1m", sa.Float, nullable=True),
sa.Column("cache_creation_price_per_1m", sa.Float, nullable=True),
sa.Column("cache_read_price_per_1m", sa.Float, nullable=True),
sa.Column("price_per_request", sa.Float, nullable=True),
sa.Column("request_type", sa.String(50), nullable=True),
sa.Column("api_format", sa.String(50), nullable=True),
sa.Column("is_stream", sa.Boolean, server_default="false"),
sa.Column("status_code", sa.Integer, nullable=True),
sa.Column("error_message", sa.Text, nullable=True),
sa.Column("response_time_ms", sa.Integer, nullable=True),
sa.Column("status", sa.String(20), server_default="completed", nullable=False, index=True),
sa.Column("request_headers", sa.JSON, nullable=True),
sa.Column("request_body", sa.JSON, nullable=True),
sa.Column("provider_request_headers", sa.JSON, nullable=True),
sa.Column("response_headers", sa.JSON, nullable=True),
sa.Column("response_body", sa.JSON, nullable=True),
sa.Column("request_body_compressed", sa.LargeBinary, nullable=True),
sa.Column("response_body_compressed", sa.LargeBinary, nullable=True),
sa.Column("request_metadata", sa.JSON, nullable=True),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.func.now(),
nullable=False,
index=True,
),
)
# usage 表复合索引(优化常见查询)
op.create_index("idx_usage_user_created", "usage", ["user_id", "created_at"])
op.create_index("idx_usage_apikey_created", "usage", ["api_key_id", "created_at"])
op.create_index("idx_usage_provider_model_created", "usage", ["provider", "model", "created_at"])
# ==================== user_quotas ====================
op.create_table(
"user_quotas",
sa.Column("id", sa.String(36), primary_key=True, index=True),
sa.Column(
"user_id", sa.String(36), sa.ForeignKey("users.id", ondelete="CASCADE"), nullable=False
),
sa.Column("quota_type", sa.String(50), nullable=False),
sa.Column("quota_usd", sa.Float, nullable=False),
sa.Column("period_start", sa.DateTime(timezone=True), nullable=False),
sa.Column("period_end", sa.DateTime(timezone=True), nullable=False),
sa.Column("used_usd", sa.Float, server_default="0.0"),
sa.Column("is_active", sa.Boolean, server_default="true"),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
)
# ==================== system_configs ====================
op.create_table(
"system_configs",
sa.Column("id", sa.String(36), primary_key=True, index=True),
sa.Column("key", sa.String(100), unique=True, nullable=False),
sa.Column("value", sa.JSON, nullable=False),
sa.Column("description", sa.Text, nullable=True),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
)
# ==================== user_preferences ====================
op.create_table(
"user_preferences",
sa.Column("id", sa.String(36), primary_key=True, index=True),
sa.Column(
"user_id",
sa.String(36),
sa.ForeignKey("users.id", ondelete="CASCADE"),
unique=True,
nullable=False,
),
sa.Column("avatar_url", sa.String(500), nullable=True),
sa.Column("bio", sa.Text, nullable=True),
sa.Column(
"default_provider_id", sa.String(36), sa.ForeignKey("providers.id"), nullable=True
),
sa.Column("theme", sa.String(20), server_default="light"),
sa.Column("language", sa.String(10), server_default="zh-CN"),
sa.Column("timezone", sa.String(50), server_default="Asia/Shanghai"),
sa.Column("email_notifications", sa.Boolean, server_default="true"),
sa.Column("usage_alerts", sa.Boolean, server_default="true"),
sa.Column("announcement_notifications", sa.Boolean, server_default="true"),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
)
# ==================== announcements ====================
op.create_table(
"announcements",
sa.Column("id", sa.String(36), primary_key=True, index=True),
sa.Column("title", sa.String(200), nullable=False),
sa.Column("content", sa.Text, nullable=False),
sa.Column("type", sa.String(20), server_default="info"),
sa.Column("priority", sa.Integer, server_default="0"),
sa.Column(
"author_id",
sa.String(36),
sa.ForeignKey("users.id", ondelete="SET NULL"),
nullable=True,
),
sa.Column("is_active", sa.Boolean, server_default="true", index=True),
sa.Column("is_pinned", sa.Boolean, server_default="false"),
sa.Column("start_time", sa.DateTime(timezone=True), nullable=True),
sa.Column("end_time", sa.DateTime(timezone=True), nullable=True),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.func.now(),
nullable=False,
index=True,
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
)
# ==================== announcement_reads ====================
op.create_table(
"announcement_reads",
sa.Column("id", sa.String(36), primary_key=True, index=True),
sa.Column(
"user_id", sa.String(36), sa.ForeignKey("users.id", ondelete="CASCADE"), nullable=False
),
sa.Column(
"announcement_id", sa.String(36), sa.ForeignKey("announcements.id"), nullable=False
),
sa.Column(
"read_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.UniqueConstraint("user_id", "announcement_id", name="uq_user_announcement"),
)
# ==================== audit_logs ====================
op.create_table(
"audit_logs",
sa.Column("id", sa.String(36), primary_key=True, index=True),
sa.Column("event_type", sa.String(50), nullable=False, index=True),
sa.Column(
"user_id",
sa.String(36),
sa.ForeignKey("users.id", ondelete="SET NULL"),
nullable=True,
index=True,
),
sa.Column("api_key_id", sa.String(36), nullable=True),
sa.Column("description", sa.Text, nullable=False),
sa.Column("ip_address", sa.String(45), nullable=True),
sa.Column("user_agent", sa.String(500), nullable=True),
sa.Column("request_id", sa.String(100), nullable=True, index=True),
sa.Column("event_metadata", sa.JSON, nullable=True),
sa.Column("status_code", sa.Integer, nullable=True),
sa.Column("error_message", sa.Text, nullable=True),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.func.now(),
nullable=False,
index=True,
),
)
# ==================== request_candidates ====================
op.create_table(
"request_candidates",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("request_id", sa.String(100), nullable=False, index=True),
sa.Column(
"user_id", sa.String(36), sa.ForeignKey("users.id", ondelete="CASCADE"), nullable=True
),
sa.Column(
"api_key_id",
sa.String(36),
sa.ForeignKey("api_keys.id", ondelete="CASCADE"),
nullable=True,
),
sa.Column("candidate_index", sa.Integer, nullable=False),
sa.Column("retry_index", sa.Integer, nullable=False, server_default="0"),
sa.Column(
"provider_id",
sa.String(36),
sa.ForeignKey("providers.id", ondelete="CASCADE"),
nullable=True,
),
sa.Column(
"endpoint_id",
sa.String(36),
sa.ForeignKey("provider_endpoints.id", ondelete="CASCADE"),
nullable=True,
),
sa.Column(
"key_id",
sa.String(36),
sa.ForeignKey("provider_api_keys.id", ondelete="CASCADE"),
nullable=True,
),
sa.Column("status", sa.String(20), nullable=False),
sa.Column("skip_reason", sa.Text, nullable=True),
sa.Column("is_cached", sa.Boolean, server_default="false"),
sa.Column("status_code", sa.Integer, nullable=True),
sa.Column("error_type", sa.String(50), nullable=True),
sa.Column("error_message", sa.Text, nullable=True),
sa.Column("latency_ms", sa.Integer, nullable=True),
sa.Column("concurrent_requests", sa.Integer, nullable=True),
sa.Column("extra_data", sa.JSON, nullable=True),
sa.Column("required_capabilities", sa.JSON, nullable=True),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column("started_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("finished_at", sa.DateTime(timezone=True), nullable=True),
sa.UniqueConstraint(
"request_id", "candidate_index", "retry_index", name="uq_request_candidate_with_retry"
),
)
op.create_index("idx_request_candidates_request_id", "request_candidates", ["request_id"])
op.create_index("idx_request_candidates_status", "request_candidates", ["status"])
op.create_index("idx_request_candidates_provider_id", "request_candidates", ["provider_id"])
# ==================== stats_daily ====================
op.create_table(
"stats_daily",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("date", sa.DateTime(timezone=True), nullable=False, unique=True, index=True),
sa.Column("total_requests", sa.Integer, server_default="0", nullable=False),
sa.Column("success_requests", sa.Integer, server_default="0", nullable=False),
sa.Column("error_requests", sa.Integer, server_default="0", nullable=False),
sa.Column("input_tokens", sa.BigInteger, server_default="0", nullable=False),
sa.Column("output_tokens", sa.BigInteger, server_default="0", nullable=False),
sa.Column("cache_creation_tokens", sa.BigInteger, server_default="0", nullable=False),
sa.Column("cache_read_tokens", sa.BigInteger, server_default="0", nullable=False),
sa.Column("total_cost", sa.Float, server_default="0.0", nullable=False),
sa.Column("actual_total_cost", sa.Float, server_default="0.0", nullable=False),
sa.Column("input_cost", sa.Float, server_default="0.0", nullable=False),
sa.Column("output_cost", sa.Float, server_default="0.0", nullable=False),
sa.Column("cache_creation_cost", sa.Float, server_default="0.0", nullable=False),
sa.Column("cache_read_cost", sa.Float, server_default="0.0", nullable=False),
sa.Column("avg_response_time_ms", sa.Float, server_default="0.0", nullable=False),
sa.Column("fallback_count", sa.Integer, server_default="0", nullable=False),
sa.Column("unique_models", sa.Integer, server_default="0", nullable=False),
sa.Column("unique_providers", sa.Integer, server_default="0", nullable=False),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
)
# ==================== stats_summary ====================
op.create_table(
"stats_summary",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("cutoff_date", sa.DateTime(timezone=True), nullable=False),
sa.Column("all_time_requests", sa.Integer, server_default="0", nullable=False),
sa.Column("all_time_success_requests", sa.Integer, server_default="0", nullable=False),
sa.Column("all_time_error_requests", sa.Integer, server_default="0", nullable=False),
sa.Column("all_time_input_tokens", sa.BigInteger, server_default="0", nullable=False),
sa.Column("all_time_output_tokens", sa.BigInteger, server_default="0", nullable=False),
sa.Column(
"all_time_cache_creation_tokens", sa.BigInteger, server_default="0", nullable=False
),
sa.Column("all_time_cache_read_tokens", sa.BigInteger, server_default="0", nullable=False),
sa.Column("all_time_cost", sa.Float, server_default="0.0", nullable=False),
sa.Column("all_time_actual_cost", sa.Float, server_default="0.0", nullable=False),
sa.Column("total_users", sa.Integer, server_default="0", nullable=False),
sa.Column("active_users", sa.Integer, server_default="0", nullable=False),
sa.Column("total_api_keys", sa.Integer, server_default="0", nullable=False),
sa.Column("active_api_keys", sa.Integer, server_default="0", nullable=False),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
)
# ==================== stats_user_daily ====================
op.create_table(
"stats_user_daily",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column(
"user_id", sa.String(36), sa.ForeignKey("users.id", ondelete="CASCADE"), nullable=False
),
sa.Column("date", sa.DateTime(timezone=True), nullable=False, index=True),
sa.Column("total_requests", sa.Integer, server_default="0", nullable=False),
sa.Column("success_requests", sa.Integer, server_default="0", nullable=False),
sa.Column("error_requests", sa.Integer, server_default="0", nullable=False),
sa.Column("input_tokens", sa.BigInteger, server_default="0", nullable=False),
sa.Column("output_tokens", sa.BigInteger, server_default="0", nullable=False),
sa.Column("cache_creation_tokens", sa.BigInteger, server_default="0", nullable=False),
sa.Column("cache_read_tokens", sa.BigInteger, server_default="0", nullable=False),
sa.Column("total_cost", sa.Float, server_default="0.0", nullable=False),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.UniqueConstraint("user_id", "date", name="uq_stats_user_daily"),
)
op.create_index("idx_stats_user_daily_user_date", "stats_user_daily", ["user_id", "date"])
# ==================== api_key_provider_mappings ====================
op.create_table(
"api_key_provider_mappings",
sa.Column("id", sa.String(36), primary_key=True, index=True),
sa.Column(
"api_key_id",
sa.String(36),
sa.ForeignKey("api_keys.id", ondelete="CASCADE"),
nullable=False,
index=True,
),
sa.Column(
"provider_id",
sa.String(36),
sa.ForeignKey("providers.id", ondelete="CASCADE"),
nullable=False,
index=True,
),
sa.Column("priority_adjustment", sa.Integer, server_default="0"),
sa.Column("weight_multiplier", sa.Float, server_default="1.0"),
sa.Column("is_enabled", sa.Boolean, server_default="true", nullable=False),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.UniqueConstraint("api_key_id", "provider_id", name="uq_apikey_provider"),
)
op.create_index(
"idx_apikey_provider_enabled", "api_key_provider_mappings", ["api_key_id", "is_enabled"]
)
# ==================== provider_usage_tracking ====================
op.create_table(
"provider_usage_tracking",
sa.Column("id", sa.String(36), primary_key=True, index=True),
sa.Column(
"provider_id",
sa.String(36),
sa.ForeignKey("providers.id", ondelete="CASCADE"),
nullable=False,
index=True,
),
sa.Column("window_start", sa.DateTime(timezone=True), nullable=False, index=True),
sa.Column("window_end", sa.DateTime(timezone=True), nullable=False),
sa.Column("total_requests", sa.Integer, server_default="0"),
sa.Column("successful_requests", sa.Integer, server_default="0"),
sa.Column("failed_requests", sa.Integer, server_default="0"),
sa.Column("avg_response_time_ms", sa.Float, server_default="0.0"),
sa.Column("total_response_time_ms", sa.Float, server_default="0.0"),
sa.Column("total_cost_usd", sa.Float, server_default="0.0"),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
)
op.create_index(
"idx_provider_window", "provider_usage_tracking", ["provider_id", "window_start"]
)
op.create_index("idx_window_time", "provider_usage_tracking", ["window_start", "window_end"])
def downgrade() -> None:
# Drop tables in reverse order (respecting foreign key dependencies)
op.drop_table("provider_usage_tracking")
op.drop_table("api_key_provider_mappings")
op.drop_table("stats_user_daily")
op.drop_table("stats_summary")
op.drop_table("stats_daily")
op.drop_table("request_candidates")
op.drop_table("audit_logs")
op.drop_table("announcement_reads")
op.drop_table("announcements")
op.drop_table("user_preferences")
op.drop_table("system_configs")
op.drop_table("user_quotas")
op.drop_table("usage")
op.drop_table("provider_api_keys")
op.drop_table("model_mappings")
op.drop_table("models")
op.drop_table("provider_endpoints")
op.drop_table("api_keys")
op.drop_table("global_models")
op.drop_table("providers")
op.drop_table("users")
# Drop ENUM types
op.execute("DROP TYPE IF EXISTS providerbillingtype")
op.execute("DROP TYPE IF EXISTS userrole")
@@ -1,315 +0,0 @@
"""remove_model_mappings_add_aliases
合并迁移:
1. 添加 provider_model_aliases 字段到 models 表
2. 迁移 model_mappings 数据到 provider_model_aliases
3. 删除 model_mappings 表
4. 添加索引优化别名解析性能
Revision ID: e9b3d63f0cbf
Revises: 20251210_baseline
Create Date: 2025-12-14 13:00:22.828183+00:00
"""
import json
from datetime import datetime, timezone
import sqlalchemy as sa
from alembic import op
from sqlalchemy.orm import Session
# revision identifiers, used by Alembic.
revision = 'e9b3d63f0cbf'
down_revision = '20251210_baseline'
branch_labels = None
depends_on = None
def column_exists(bind, table_name: str, column_name: str) -> bool:
"""检查列是否存在"""
result = bind.execute(
sa.text(
"""
SELECT EXISTS (
SELECT 1 FROM information_schema.columns
WHERE table_name = :table_name AND column_name = :column_name
)
"""
),
{"table_name": table_name, "column_name": column_name},
)
return result.scalar()
def table_exists(bind, table_name: str) -> bool:
"""检查表是否存在"""
result = bind.execute(
sa.text(
"""
SELECT EXISTS (
SELECT 1 FROM information_schema.tables
WHERE table_name = :table_name
)
"""
),
{"table_name": table_name},
)
return result.scalar()
def index_exists(bind, index_name: str) -> bool:
"""检查索引是否存在"""
result = bind.execute(
sa.text(
"""
SELECT EXISTS (
SELECT 1 FROM pg_indexes
WHERE indexname = :index_name
)
"""
),
{"index_name": index_name},
)
return result.scalar()
def upgrade() -> None:
"""添加 provider_model_aliases 字段,迁移数据,删除 model_mappings 表"""
bind = op.get_bind()
# 1. 添加 provider_model_aliases 字段(如果不存在)
if not column_exists(bind, "models", "provider_model_aliases"):
op.add_column(
'models',
sa.Column('provider_model_aliases', sa.JSON(), nullable=True)
)
# 2. 迁移 model_mappings 数据(如果表存在)
session = Session(bind=bind)
model_mappings_table = sa.table(
"model_mappings",
sa.column("source_model", sa.String),
sa.column("target_global_model_id", sa.String),
sa.column("provider_id", sa.String),
sa.column("mapping_type", sa.String),
sa.column("is_active", sa.Boolean),
)
models_table = sa.table(
"models",
sa.column("id", sa.String),
sa.column("provider_id", sa.String),
sa.column("global_model_id", sa.String),
sa.column("provider_model_aliases", sa.JSON),
sa.column("updated_at", sa.DateTime(timezone=True)),
)
def normalize_alias_list(value) -> list[dict]:
"""将 DB 返回的 JSON 值规范化为 list[{'name': str, 'priority': int}]"""
if value is None:
return []
if isinstance(value, str):
try:
value = json.loads(value) if value else []
except Exception:
return []
if not isinstance(value, list):
return []
normalized: list[dict] = []
for item in value:
if not isinstance(item, dict):
continue
raw_name = item.get("name")
if not isinstance(raw_name, str):
continue
name = raw_name.strip()
if not name:
continue
raw_priority = item.get("priority", 1)
try:
priority = int(raw_priority)
except Exception:
priority = 1
if priority < 1:
priority = 1
normalized.append({"name": name, "priority": priority})
return normalized
# 查询所有活跃的 provider 级别 alias(只迁移 is_active=True 且 mapping_type='alias' 的)
# 全局别名/映射不迁移(新架构不再支持 source_model -> GlobalModel.name 的解析)
# 仅当 model_mappings 表存在时执行迁移
if table_exists(bind, "model_mappings"):
mappings = session.execute(
sa.select(
model_mappings_table.c.source_model,
model_mappings_table.c.target_global_model_id,
model_mappings_table.c.provider_id,
)
.where(
model_mappings_table.c.is_active.is_(True),
model_mappings_table.c.provider_id.isnot(None),
model_mappings_table.c.mapping_type == "alias",
)
.order_by(model_mappings_table.c.provider_id, model_mappings_table.c.source_model)
).all()
# 按 (provider_id, target_global_model_id) 分组,收集别名
alias_groups: dict = {}
for source_model, target_global_model_id, provider_id in mappings:
if not isinstance(source_model, str):
continue
source_model = source_model.strip()
if not source_model:
continue
if not isinstance(provider_id, str) or not provider_id:
continue
if not isinstance(target_global_model_id, str) or not target_global_model_id:
continue
key = (provider_id, target_global_model_id)
if key not in alias_groups:
alias_groups[key] = []
priority = len(alias_groups[key]) + 1
alias_groups[key].append({"name": source_model, "priority": priority})
# 更新对应的 models 记录
for (provider_id, global_model_id), aliases in alias_groups.items():
model_row = session.execute(
sa.select(models_table.c.id, models_table.c.provider_model_aliases)
.where(
models_table.c.provider_id == provider_id,
models_table.c.global_model_id == global_model_id,
)
.limit(1)
).first()
if model_row:
model_id = model_row[0]
existing_aliases = normalize_alias_list(model_row[1])
existing_names = {a["name"] for a in existing_aliases}
merged_aliases = list(existing_aliases)
for alias in aliases:
name = alias.get("name")
if not isinstance(name, str):
continue
name = name.strip()
if not name or name in existing_names:
continue
merged_aliases.append(
{
"name": name,
"priority": len(merged_aliases) + 1,
}
)
existing_names.add(name)
session.execute(
models_table.update()
.where(models_table.c.id == model_id)
.values(
provider_model_aliases=merged_aliases if merged_aliases else None,
updated_at=datetime.now(timezone.utc),
)
)
session.commit()
# 3. 删除 model_mappings 表
op.drop_table('model_mappings')
# 4. 添加索引优化别名解析性能
# provider_model_name 索引(支持精确匹配,如果不存在)
if not index_exists(bind, "idx_model_provider_model_name"):
op.create_index(
"idx_model_provider_model_name",
"models",
["provider_model_name"],
unique=False,
postgresql_where=sa.text("is_active = true"),
)
# provider_model_aliases GIN 索引(支持 JSONB 查询,仅 PostgreSQL)
if bind.dialect.name == "postgresql":
# 将 json 列转为 jsonb(jsonb 性能更好且支持 GIN 索引)
# 使用 IF NOT EXISTS 风格的检查来避免重复转换
op.execute(
"""
DO $$
BEGIN
IF EXISTS (
SELECT 1 FROM information_schema.columns
WHERE table_name = 'models'
AND column_name = 'provider_model_aliases'
AND data_type = 'json'
) THEN
ALTER TABLE models
ALTER COLUMN provider_model_aliases TYPE jsonb
USING provider_model_aliases::jsonb;
END IF;
END $$;
"""
)
# 创建 GIN 索引
op.execute(
"""
CREATE INDEX IF NOT EXISTS idx_model_provider_model_aliases_gin
ON models USING gin(provider_model_aliases jsonb_path_ops)
WHERE is_active = true
"""
)
def downgrade() -> None:
"""恢复 model_mappings 表,移除 provider_model_aliases 字段和索引"""
bind = op.get_bind()
# 1. 删除索引
op.drop_index("idx_model_provider_model_name", table_name="models")
if bind.dialect.name == "postgresql":
op.execute("DROP INDEX IF EXISTS idx_model_provider_model_aliases_gin")
# 将 jsonb 列还原为 json
op.execute(
"""
ALTER TABLE models
ALTER COLUMN provider_model_aliases TYPE json
USING provider_model_aliases::json
"""
)
# 2. 恢复 model_mappings 表
op.create_table(
'model_mappings',
sa.Column('id', sa.String(36), primary_key=True),
sa.Column('source_model', sa.String(200), nullable=False),
sa.Column(
'target_global_model_id',
sa.String(36),
sa.ForeignKey('global_models.id', ondelete='CASCADE'),
nullable=False,
),
sa.Column('provider_id', sa.String(36), sa.ForeignKey('providers.id'), nullable=True),
sa.Column('mapping_type', sa.String(20), nullable=False, server_default='alias'),
sa.Column('is_active', sa.Boolean(), nullable=False, server_default='true'),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.UniqueConstraint('source_model', 'provider_id', name='uq_model_mapping_source_provider'),
)
op.create_index('ix_model_mappings_source_model', 'model_mappings', ['source_model'])
op.create_index('ix_model_mappings_target_global_model_id', 'model_mappings', ['target_global_model_id'])
op.create_index('ix_model_mappings_provider_id', 'model_mappings', ['provider_id'])
op.create_index('ix_model_mappings_mapping_type', 'model_mappings', ['mapping_type'])
# 3. 移除 provider_model_aliases 字段
op.drop_column('models', 'provider_model_aliases')
@@ -1,47 +0,0 @@
"""add first_byte_time_ms to usage table
Revision ID: 180e63a9c83a
Revises: e9b3d63f0cbf
Create Date: 2025-12-15 17:07:44.631032+00:00
"""
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = '180e63a9c83a'
down_revision = 'e9b3d63f0cbf'
branch_labels = None
depends_on = None
def column_exists(bind, table_name: str, column_name: str) -> bool:
"""检查列是否存在"""
result = bind.execute(
sa.text(
"""
SELECT EXISTS (
SELECT 1 FROM information_schema.columns
WHERE table_name = :table_name AND column_name = :column_name
)
"""
),
{"table_name": table_name, "column_name": column_name},
)
return result.scalar()
def upgrade() -> None:
"""应用迁移:升级到新版本"""
bind = op.get_bind()
# 添加首字时间字段到 usage 表(如果不存在)
if not column_exists(bind, "usage", "first_byte_time_ms"):
op.add_column('usage', sa.Column('first_byte_time_ms', sa.Integer(), nullable=True))
def downgrade() -> None:
"""回滚迁移:降级到旧版本"""
# 删除首字时间字段
op.drop_column('usage', 'first_byte_time_ms')
@@ -1,110 +0,0 @@
"""refactor global_model to use config json field
Revision ID: 1cc6942cf06f
Revises: 180e63a9c83a
Create Date: 2025-12-16 03:11:32.480976+00:00
"""
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
# revision identifiers, used by Alembic.
revision = '1cc6942cf06f'
down_revision = '180e63a9c83a'
branch_labels = None
depends_on = None
def column_exists(bind, table_name: str, column_name: str) -> bool:
"""检查列是否存在"""
result = bind.execute(
sa.text(
"""
SELECT EXISTS (
SELECT 1 FROM information_schema.columns
WHERE table_name = :table_name AND column_name = :column_name
)
"""
),
{"table_name": table_name, "column_name": column_name},
)
return result.scalar()
def upgrade() -> None:
"""应用迁移:升级到新版本
1. 添加 config 列
2. 把旧数据迁移到 config
3. 删除旧列
"""
bind = op.get_bind()
# 检查是否已经迁移过(config 列存在且旧列不存在)
has_config = column_exists(bind, "global_models", "config")
has_old_columns = column_exists(bind, "global_models", "default_supports_streaming")
if has_config and not has_old_columns:
# 已完成迁移,跳过
return
# 1. 添加 config 列(使用 JSONB 类型,支持索引和更高效的查询)
if not has_config:
op.add_column('global_models', sa.Column('config', postgresql.JSONB(), nullable=True))
# 2. 迁移数据:把旧字段合并到 config JSON(仅当旧列存在时)
if has_old_columns:
op.execute("""
UPDATE global_models
SET config = jsonb_strip_nulls(jsonb_build_object(
'streaming', COALESCE(default_supports_streaming, true),
'vision', CASE WHEN COALESCE(default_supports_vision, false) THEN true ELSE NULL END,
'function_calling', CASE WHEN COALESCE(default_supports_function_calling, false) THEN true ELSE NULL END,
'extended_thinking', CASE WHEN COALESCE(default_supports_extended_thinking, false) THEN true ELSE NULL END,
'image_generation', CASE WHEN COALESCE(default_supports_image_generation, false) THEN true ELSE NULL END,
'description', description,
'icon_url', icon_url,
'official_url', official_url
))
""")
# 3. 删除旧列
op.drop_column('global_models', 'default_supports_streaming')
op.drop_column('global_models', 'default_supports_vision')
op.drop_column('global_models', 'default_supports_function_calling')
op.drop_column('global_models', 'default_supports_extended_thinking')
op.drop_column('global_models', 'default_supports_image_generation')
op.drop_column('global_models', 'description')
op.drop_column('global_models', 'icon_url')
op.drop_column('global_models', 'official_url')
def downgrade() -> None:
"""回滚迁移:降级到旧版本"""
# 1. 添加旧列
op.add_column('global_models', sa.Column('icon_url', sa.VARCHAR(length=500), nullable=True))
op.add_column('global_models', sa.Column('official_url', sa.VARCHAR(length=500), nullable=True))
op.add_column('global_models', sa.Column('description', sa.TEXT(), nullable=True))
op.add_column('global_models', sa.Column('default_supports_streaming', sa.BOOLEAN(), nullable=True))
op.add_column('global_models', sa.Column('default_supports_vision', sa.BOOLEAN(), nullable=True))
op.add_column('global_models', sa.Column('default_supports_function_calling', sa.BOOLEAN(), nullable=True))
op.add_column('global_models', sa.Column('default_supports_extended_thinking', sa.BOOLEAN(), nullable=True))
op.add_column('global_models', sa.Column('default_supports_image_generation', sa.BOOLEAN(), nullable=True))
# 2. 从 config 恢复数据
op.execute("""
UPDATE global_models
SET
default_supports_streaming = COALESCE((config->>'streaming')::boolean, true),
default_supports_vision = COALESCE((config->>'vision')::boolean, false),
default_supports_function_calling = COALESCE((config->>'function_calling')::boolean, false),
default_supports_extended_thinking = COALESCE((config->>'extended_thinking')::boolean, false),
default_supports_image_generation = COALESCE((config->>'image_generation')::boolean, false),
description = config->>'description',
icon_url = config->>'icon_url',
official_url = config->>'official_url'
""")
# 3. 删除 config 列
op.drop_column('global_models', 'config')
@@ -1,57 +0,0 @@
"""add proxy field to provider_endpoints
Revision ID: f30f9936f6a2
Revises: 1cc6942cf06f
Create Date: 2025-12-18 06:31:58.451112+00:00
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy import inspect
# revision identifiers, used by Alembic.
revision = 'f30f9936f6a2'
down_revision = '1cc6942cf06f'
branch_labels = None
depends_on = None
def column_exists(table_name: str, column_name: str) -> bool:
"""检查列是否存在"""
bind = op.get_bind()
inspector = inspect(bind)
columns = [col['name'] for col in inspector.get_columns(table_name)]
return column_name in columns
def get_column_type(table_name: str, column_name: str) -> str:
"""获取列的类型"""
bind = op.get_bind()
inspector = inspect(bind)
for col in inspector.get_columns(table_name):
if col['name'] == column_name:
return str(col['type']).upper()
return ''
def upgrade() -> None:
"""添加 proxy 字段到 provider_endpoints 表"""
if not column_exists('provider_endpoints', 'proxy'):
# 字段不存在,直接添加 JSONB 类型
op.add_column('provider_endpoints', sa.Column('proxy', JSONB(), nullable=True))
else:
# 字段已存在,检查是否需要转换类型
col_type = get_column_type('provider_endpoints', 'proxy')
if 'JSONB' not in col_type:
# 如果是 JSON 类型,转换为 JSONB
op.execute(
'ALTER TABLE provider_endpoints '
'ALTER COLUMN proxy TYPE JSONB USING proxy::jsonb'
)
def downgrade() -> None:
"""移除 proxy 字段"""
if column_exists('provider_endpoints', 'proxy'):
op.drop_column('provider_endpoints', 'proxy')
@@ -1,86 +0,0 @@
"""add stats_daily_model table and rename provider_model_aliases
Revision ID: a1b2c3d4e5f6
Revises: f30f9936f6a2
Create Date: 2025-12-20 12:00:00.000000+00:00
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy import inspect
# revision identifiers, used by Alembic.
revision = 'a1b2c3d4e5f6'
down_revision = 'f30f9936f6a2'
branch_labels = None
depends_on = None
def table_exists(table_name: str) -> bool:
"""检查表是否存在"""
bind = op.get_bind()
inspector = inspect(bind)
return table_name in inspector.get_table_names()
def column_exists(table_name: str, column_name: str) -> bool:
"""检查列是否存在"""
bind = op.get_bind()
inspector = inspect(bind)
columns = [col['name'] for col in inspector.get_columns(table_name)]
return column_name in columns
def upgrade() -> None:
"""创建 stats_daily_model 表,重命名 provider_model_aliases 为 provider_model_mappings"""
# 1. 创建 stats_daily_model 表
if not table_exists('stats_daily_model'):
op.create_table(
'stats_daily_model',
sa.Column('id', sa.String(36), primary_key=True),
sa.Column('date', sa.DateTime(timezone=True), nullable=False),
sa.Column('model', sa.String(100), nullable=False),
sa.Column('total_requests', sa.Integer(), nullable=False, default=0),
sa.Column('input_tokens', sa.BigInteger(), nullable=False, default=0),
sa.Column('output_tokens', sa.BigInteger(), nullable=False, default=0),
sa.Column('cache_creation_tokens', sa.BigInteger(), nullable=False, default=0),
sa.Column('cache_read_tokens', sa.BigInteger(), nullable=False, default=0),
sa.Column('total_cost', sa.Float(), nullable=False, default=0.0),
sa.Column('avg_response_time_ms', sa.Float(), nullable=False, default=0.0),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False,
server_default=sa.func.now()),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False,
server_default=sa.func.now(), onupdate=sa.func.now()),
sa.UniqueConstraint('date', 'model', name='uq_stats_daily_model'),
)
# 创建索引
op.create_index('idx_stats_daily_model_date', 'stats_daily_model', ['date'])
op.create_index('idx_stats_daily_model_date_model', 'stats_daily_model', ['date', 'model'])
# 2. 重命名 models 表的 provider_model_aliases 为 provider_model_mappings
if column_exists('models', 'provider_model_aliases') and not column_exists('models', 'provider_model_mappings'):
op.alter_column('models', 'provider_model_aliases', new_column_name='provider_model_mappings')
def index_exists(table_name: str, index_name: str) -> bool:
"""检查索引是否存在"""
bind = op.get_bind()
inspector = inspect(bind)
indexes = [idx['name'] for idx in inspector.get_indexes(table_name)]
return index_name in indexes
def downgrade() -> None:
"""删除 stats_daily_model 表,恢复 provider_model_aliases 列名"""
# 恢复列名
if column_exists('models', 'provider_model_mappings') and not column_exists('models', 'provider_model_aliases'):
op.alter_column('models', 'provider_model_mappings', new_column_name='provider_model_aliases')
# 删除表
if table_exists('stats_daily_model'):
if index_exists('stats_daily_model', 'idx_stats_daily_model_date_model'):
op.drop_index('idx_stats_daily_model_date_model', table_name='stats_daily_model')
if index_exists('stats_daily_model', 'idx_stats_daily_model_date'):
op.drop_index('idx_stats_daily_model_date', table_name='stats_daily_model')
op.drop_table('stats_daily_model')
@@ -1,65 +0,0 @@
"""add usage table composite indexes for query optimization
Revision ID: b2c3d4e5f6g7
Revises: a1b2c3d4e5f6
Create Date: 2025-12-20 15:00:00.000000+00:00
"""
from alembic import op
from sqlalchemy import text
# revision identifiers, used by Alembic.
revision = 'b2c3d4e5f6g7'
down_revision = 'a1b2c3d4e5f6'
branch_labels = None
depends_on = None
def upgrade() -> None:
"""为 usage 表添加复合索引以优化常见查询
注意:这些索引已经在 baseline 迁移中创建。
此迁移仅用于从旧版本升级的场景,新安装会跳过。
"""
conn = op.get_bind()
# 检查 usage 表是否存在
result = conn.execute(text(
"SELECT EXISTS (SELECT FROM information_schema.tables WHERE table_name = 'usage')"
))
if not result.scalar():
# 表不存在,跳过
return
# 定义需要创建的索引
indexes = [
("idx_usage_user_created", "ON usage (user_id, created_at)"),
("idx_usage_apikey_created", "ON usage (api_key_id, created_at)"),
("idx_usage_provider_model_created", "ON usage (provider, model, created_at)"),
]
# 分别检查并创建每个索引
for index_name, index_def in indexes:
result = conn.execute(text(
f"SELECT EXISTS (SELECT 1 FROM pg_indexes WHERE indexname = '{index_name}')"
))
if result.scalar():
continue # 索引已存在,跳过
conn.execute(text(f"CREATE INDEX {index_name} {index_def}"))
def downgrade() -> None:
"""删除复合索引"""
conn = op.get_bind()
# 使用 IF EXISTS 避免索引不存在时报错
conn.execute(text(
"DROP INDEX IF EXISTS idx_usage_provider_model_created"
))
conn.execute(text(
"DROP INDEX IF EXISTS idx_usage_apikey_created"
))
conn.execute(text(
"DROP INDEX IF EXISTS idx_usage_user_created"
))
@@ -1,161 +0,0 @@
"""add ldap authentication support
Revision ID: c3d4e5f6g7h8
Revises: b2c3d4e5f6g7
Create Date: 2026-01-01 14:00:00.000000+00:00
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy import text
# revision identifiers, used by Alembic.
revision = 'c3d4e5f6g7h8'
down_revision = 'b2c3d4e5f6g7'
branch_labels = None
depends_on = None
def _type_exists(conn, type_name: str) -> bool:
"""检查 PostgreSQL 类型是否存在"""
result = conn.execute(
text("SELECT 1 FROM pg_type WHERE typname = :name"),
{"name": type_name}
)
return result.scalar() is not None
def _column_exists(conn, table_name: str, column_name: str) -> bool:
"""检查列是否存在"""
result = conn.execute(
text("""
SELECT 1 FROM information_schema.columns
WHERE table_name = :table AND column_name = :column
"""),
{"table": table_name, "column": column_name}
)
return result.scalar() is not None
def _index_exists(conn, index_name: str) -> bool:
"""检查索引是否存在"""
result = conn.execute(
text("SELECT 1 FROM pg_indexes WHERE indexname = :name"),
{"name": index_name}
)
return result.scalar() is not None
def _table_exists(conn, table_name: str) -> bool:
"""检查表是否存在"""
result = conn.execute(
text("""
SELECT 1 FROM information_schema.tables
WHERE table_name = :name AND table_schema = 'public'
"""),
{"name": table_name}
)
return result.scalar() is not None
def upgrade() -> None:
"""添加 LDAP 认证支持
1. 创建 authsource 枚举类型
2. 在 users 表添加 auth_source 字段和 LDAP 标识字段
3. 创建 ldap_configs 表
"""
conn = op.get_bind()
# 1. 创建 authsource 枚举类型(幂等)
if not _type_exists(conn, 'authsource'):
conn.execute(text("CREATE TYPE authsource AS ENUM ('local', 'ldap')"))
# 2. 在 users 表添加字段(幂等)
if not _column_exists(conn, 'users', 'auth_source'):
op.add_column('users', sa.Column(
'auth_source',
sa.Enum('local', 'ldap', name='authsource', create_type=False),
nullable=False,
server_default='local'
))
if not _column_exists(conn, 'users', 'ldap_dn'):
op.add_column('users', sa.Column('ldap_dn', sa.String(length=512), nullable=True))
if not _column_exists(conn, 'users', 'ldap_username'):
op.add_column('users', sa.Column('ldap_username', sa.String(length=255), nullable=True))
# 创建索引(幂等)
if not _index_exists(conn, 'ix_users_ldap_dn'):
op.create_index('ix_users_ldap_dn', 'users', ['ldap_dn'])
if not _index_exists(conn, 'ix_users_ldap_username'):
op.create_index('ix_users_ldap_username', 'users', ['ldap_username'])
# 3. 创建 ldap_configs 表(幂等)
if not _table_exists(conn, 'ldap_configs'):
op.create_table(
'ldap_configs',
sa.Column('id', sa.Integer(), autoincrement=True, nullable=False),
sa.Column('server_url', sa.String(length=255), nullable=False),
sa.Column('bind_dn', sa.String(length=255), nullable=False),
sa.Column('bind_password_encrypted', sa.Text(), nullable=True),
sa.Column('base_dn', sa.String(length=255), nullable=False),
sa.Column('user_search_filter', sa.String(length=500), nullable=False, server_default='(uid={username})'),
sa.Column('username_attr', sa.String(length=50), nullable=False, server_default='uid'),
sa.Column('email_attr', sa.String(length=50), nullable=False, server_default='mail'),
sa.Column('display_name_attr', sa.String(length=50), nullable=False, server_default='cn'),
sa.Column('is_enabled', sa.Boolean(), nullable=False, server_default='false'),
sa.Column('is_exclusive', sa.Boolean(), nullable=False, server_default='false'),
sa.Column('use_starttls', sa.Boolean(), nullable=False, server_default='false'),
sa.Column('connect_timeout', sa.Integer(), nullable=False, server_default='10'),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False, server_default=sa.text('now()')),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False, server_default=sa.text('now()')),
sa.PrimaryKeyConstraint('id')
)
def downgrade() -> None:
"""回滚 LDAP 认证支持
警告:回滚前请确保:
1. 已备份数据库
2. 没有 LDAP 用户需要保留
"""
conn = op.get_bind()
# 检查是否存在 LDAP 用户,防止数据丢失
if _column_exists(conn, 'users', 'auth_source'):
result = conn.execute(text("SELECT COUNT(*) FROM users WHERE auth_source = 'ldap'"))
ldap_user_count = result.scalar()
if ldap_user_count and ldap_user_count > 0:
raise RuntimeError(
f"无法回滚:存在 {ldap_user_count} 个 LDAP 用户。"
f"请先删除或转换这些用户,或使用 --force 参数强制回滚(将丢失数据)。"
)
# 1. 删除 ldap_configs 表(幂等)
if _table_exists(conn, 'ldap_configs'):
op.drop_table('ldap_configs')
# 2. 删除 users 表的 LDAP 相关字段(幂等)
if _index_exists(conn, 'ix_users_ldap_username'):
op.drop_index('ix_users_ldap_username', table_name='users')
if _index_exists(conn, 'ix_users_ldap_dn'):
op.drop_index('ix_users_ldap_dn', table_name='users')
if _column_exists(conn, 'users', 'ldap_username'):
op.drop_column('users', 'ldap_username')
if _column_exists(conn, 'users', 'ldap_dn'):
op.drop_column('users', 'ldap_dn')
if _column_exists(conn, 'users', 'auth_source'):
op.drop_column('users', 'auth_source')
# 3. 删除 authsource 枚举类型(幂等)
# 注意:不使用 CASCADE,因为此时所有依赖应该已被删除
if _type_exists(conn, 'authsource'):
conn.execute(text("DROP TYPE authsource"))
@@ -1,131 +0,0 @@
"""add_management_tokens_table
Revision ID: ad55f1d008b7
Revises: c3d4e5f6g7h8
Create Date: 2026-01-06 15:24:10.660394+00:00
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy import inspect
# revision identifiers, used by Alembic.
revision = 'ad55f1d008b7'
down_revision = 'c3d4e5f6g7h8'
branch_labels = None
depends_on = None
def table_exists(table_name: str) -> bool:
"""检查表是否存在"""
conn = op.get_bind()
inspector = inspect(conn)
return table_name in inspector.get_table_names()
def index_exists(table_name: str, index_name: str) -> bool:
"""检查索引是否存在"""
conn = op.get_bind()
inspector = inspect(conn)
try:
indexes = inspector.get_indexes(table_name)
return any(idx["name"] == index_name for idx in indexes)
except Exception:
return False
def constraint_exists(table_name: str, constraint_name: str) -> bool:
"""检查约束是否存在"""
conn = op.get_bind()
inspector = inspect(conn)
try:
constraints = inspector.get_unique_constraints(table_name)
if any(c["name"] == constraint_name for c in constraints):
return True
# 也检查 check 约束
check_constraints = inspector.get_check_constraints(table_name)
if any(c["name"] == constraint_name for c in check_constraints):
return True
return False
except Exception:
return False
def upgrade() -> None:
"""应用迁移:创建 management_tokens 表"""
# 幂等性检查
if table_exists("management_tokens"):
# 表已存在,检查是否需要添加约束
if not constraint_exists("management_tokens", "uq_management_tokens_user_name"):
op.create_unique_constraint(
"uq_management_tokens_user_name",
"management_tokens",
["user_id", "name"],
)
# 添加 IP 白名单非空检查约束
if not constraint_exists("management_tokens", "check_allowed_ips_not_empty"):
op.create_check_constraint(
"check_allowed_ips_not_empty",
"management_tokens",
"allowed_ips IS NULL OR allowed_ips::text = 'null' OR json_array_length(allowed_ips) > 0",
)
return
op.create_table('management_tokens',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('user_id', sa.String(length=36), nullable=False),
sa.Column('token_hash', sa.String(length=64), nullable=False),
sa.Column('token_prefix', sa.String(length=12), nullable=True),
sa.Column('name', sa.String(length=100), nullable=False),
sa.Column('description', sa.Text(), nullable=True),
sa.Column('allowed_ips', sa.JSON(), nullable=True),
sa.Column('expires_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('last_used_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('last_used_ip', sa.String(length=45), nullable=True),
sa.Column('usage_count', sa.Integer(), server_default='0', nullable=False),
sa.Column('is_active', sa.Boolean(), server_default='true', nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.ForeignKeyConstraint(['user_id'], ['users.id'], ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id')
)
op.create_index('idx_management_tokens_is_active', 'management_tokens', ['is_active'], unique=False)
op.create_index('idx_management_tokens_user_id', 'management_tokens', ['user_id'], unique=False)
op.create_index(op.f('ix_management_tokens_token_hash'), 'management_tokens', ['token_hash'], unique=True)
# 添加用户名称唯一约束
op.create_unique_constraint(
"uq_management_tokens_user_name",
"management_tokens",
["user_id", "name"],
)
# 添加 IP 白名单非空检查约束
# 注意:JSON 类型的 NULL 可能被序列化为 JSON 'null',需要同时处理
op.create_check_constraint(
"check_allowed_ips_not_empty",
"management_tokens",
"allowed_ips IS NULL OR allowed_ips::text = 'null' OR json_array_length(allowed_ips) > 0",
)
def downgrade() -> None:
"""回滚迁移:删除 management_tokens 表"""
# 幂等性检查
if not table_exists("management_tokens"):
return
# 删除约束
if constraint_exists("management_tokens", "check_allowed_ips_not_empty"):
op.drop_constraint("check_allowed_ips_not_empty", "management_tokens", type_="check")
if constraint_exists("management_tokens", "uq_management_tokens_user_name"):
op.drop_constraint("uq_management_tokens_user_name", "management_tokens", type_="unique")
# 删除索引
if index_exists("management_tokens", "ix_management_tokens_token_hash"):
op.drop_index(op.f('ix_management_tokens_token_hash'), table_name='management_tokens')
if index_exists("management_tokens", "idx_management_tokens_user_id"):
op.drop_index('idx_management_tokens_user_id', table_name='management_tokens')
if index_exists("management_tokens", "idx_management_tokens_is_active"):
op.drop_index('idx_management_tokens_is_active', table_name='management_tokens')
# 删除表
op.drop_table('management_tokens')
@@ -1,73 +0,0 @@
"""cleanup ambiguous database fields
Revision ID: 02a45b66b7c4
Revises: ad55f1d008b7
Create Date: 2026-01-07 11:20:12.684426+00:00
变更内容:
1. users 表:重命名 allowed_endpoints 为 allowed_api_formats(修正历史命名错误)
2. api_keys 表:删除 allowed_endpoints 字段(未使用的功能)
3. providers 表:删除 rate_limit 字段(与 rpm_limit 功能重复,且未使用)
4. usage 表:重命名 provider 为 provider_name(避免与 provider_id 外键混淆)
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy import inspect
# revision identifiers, used by Alembic.
revision = '02a45b66b7c4'
down_revision = 'ad55f1d008b7'
branch_labels = None
depends_on = None
def _column_exists(table_name: str, column_name: str) -> bool:
"""检查列是否存在"""
bind = op.get_bind()
inspector = inspect(bind)
columns = [col['name'] for col in inspector.get_columns(table_name)]
return column_name in columns
def upgrade() -> None:
"""
1. users.allowed_endpoints -> allowed_api_formats(重命名)
2. api_keys.allowed_endpoints 删除
3. providers.rate_limit 删除(与 rpm_limit 重复)
4. usage.provider -> provider_name(重命名)
"""
# 1. users 表:重命名 allowed_endpoints 为 allowed_api_formats
if _column_exists('users', 'allowed_endpoints'):
op.alter_column('users', 'allowed_endpoints', new_column_name='allowed_api_formats')
# 2. api_keys 表:删除 allowed_endpoints 字段
if _column_exists('api_keys', 'allowed_endpoints'):
op.drop_column('api_keys', 'allowed_endpoints')
# 3. providers 表:删除 rate_limit 字段(与 rpm_limit 功能重复)
if _column_exists('providers', 'rate_limit'):
op.drop_column('providers', 'rate_limit')
# 4. usage 表:重命名 provider 为 provider_name
if _column_exists('usage', 'provider'):
op.alter_column('usage', 'provider', new_column_name='provider_name')
def downgrade() -> None:
"""回滚:恢复原字段"""
# 4. usage 表:将 provider_name 改回 provider
if _column_exists('usage', 'provider_name'):
op.alter_column('usage', 'provider_name', new_column_name='provider')
# 3. providers 表:恢复 rate_limit 字段
if not _column_exists('providers', 'rate_limit'):
op.add_column('providers', sa.Column('rate_limit', sa.Integer(), nullable=True))
# 2. api_keys 表:恢复 allowed_endpoints 字段
if not _column_exists('api_keys', 'allowed_endpoints'):
op.add_column('api_keys', sa.Column('allowed_endpoints', sa.JSON(), nullable=True))
# 1. users 表:将 allowed_api_formats 改回 allowed_endpoints
if _column_exists('users', 'allowed_api_formats'):
op.alter_column('users', 'allowed_api_formats', new_column_name='allowed_endpoints')
@@ -1,604 +0,0 @@
"""consolidated schema updates
Revision ID: m4n5o6p7q8r9
Revises: 02a45b66b7c4
Create Date: 2026-01-10 20:00:00.000000
This migration consolidates all schema changes from 2026-01-08 to 2026-01-10:
1. provider_api_keys: Key 直接关联 Provider (provider_id, api_formats)
2. provider_api_keys: 添加 rate_multipliers JSON 字段(按格式费率)
3. models: global_model_id 改为可空(支持独立 ProviderModel)
4. providers: 添加 timeout, max_retries, proxy(从 endpoint 迁移)
5. providers: display_name 重命名为 name,删除原 name
6. provider_api_keys: max_concurrent -> rpm_limit(并发改 RPM)
7. provider_api_keys: 健康度改为按格式存储(health_by_format, circuit_breaker_by_format)
8. provider_endpoints: 删除废弃的 rate_limit 列
9. usage: 添加 client_response_headers 字段
10. provider_api_keys: 删除 endpoint_id(Key 不再与 Endpoint 绑定)
11. provider_endpoints: 删除废弃的 max_concurrent 列
12. providers: 删除废弃的 rpm_limit, rpm_used, rpm_reset_at 列
"""
import logging
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
from sqlalchemy.exc import ProgrammingError
from alembic import op
# 配置日志
alembic_logger = logging.getLogger("alembic.runtime.migration")
revision = "m4n5o6p7q8r9"
down_revision = "02a45b66b7c4"
branch_labels = None
depends_on = None
def _column_exists(table_name: str, column_name: str) -> bool:
"""Check if a column exists in the table (bypasses inspector cache)"""
bind = op.get_bind()
result = bind.execute(
sa.text(
"SELECT 1 FROM information_schema.columns "
"WHERE table_name = :table AND column_name = :col"
),
{"table": table_name, "col": column_name},
)
return result.scalar() is not None
def _constraint_exists(table_name: str, constraint_name: str) -> bool:
"""Check if a constraint exists (bypasses inspector cache)"""
bind = op.get_bind()
result = bind.execute(
sa.text(
"SELECT 1 FROM information_schema.table_constraints "
"WHERE table_name = :table AND constraint_name = :name"
),
{"table": table_name, "name": constraint_name},
)
return result.scalar() is not None
def _index_exists(table_name: str, index_name: str) -> bool:
"""Check if an index exists (bypasses inspector cache)"""
bind = op.get_bind()
result = bind.execute(
sa.text("SELECT 1 FROM pg_indexes WHERE indexname = :name"),
{"name": index_name},
)
return result.scalar() is not None
def upgrade() -> None:
"""Apply all consolidated schema changes"""
bind = op.get_bind()
# ========== 1. provider_api_keys: 添加 provider_id 和 api_formats ==========
if not _column_exists("provider_api_keys", "provider_id"):
conn = op.get_bind()
conn.execute(sa.text("SAVEPOINT sp_add_provider_id"))
try:
op.add_column(
"provider_api_keys", sa.Column("provider_id", sa.String(36), nullable=True)
)
conn.execute(sa.text("RELEASE SAVEPOINT sp_add_provider_id"))
except ProgrammingError as exc:
if getattr(getattr(exc, "orig", None), "pgcode", None) == "42701":
conn.execute(sa.text("ROLLBACK TO SAVEPOINT sp_add_provider_id"))
alembic_logger.warning("provider_api_keys.provider_id already exists; skipping add")
else:
conn.execute(sa.text("ROLLBACK TO SAVEPOINT sp_add_provider_id"))
raise
# 数据迁移:从 endpoint 获取 provider_id(如果 endpoint_id 仍存在)
if _column_exists("provider_api_keys", "endpoint_id"):
op.execute("""
UPDATE provider_api_keys k
SET provider_id = e.provider_id
FROM provider_endpoints e
WHERE k.endpoint_id = e.id AND k.provider_id IS NULL
""")
# 检查无法关联的孤儿 Key
result = bind.execute(
sa.text("SELECT COUNT(*) FROM provider_api_keys WHERE provider_id IS NULL")
)
orphan_count = result.scalar() or 0
if orphan_count > 0:
# 使用 logger 记录更明显的告警
alembic_logger.warning("=" * 60)
alembic_logger.warning(
f"[MIGRATION WARNING] 发现 {orphan_count} 个无法关联 Provider 的孤儿 Key"
)
alembic_logger.warning("=" * 60)
alembic_logger.info("正在备份孤儿 Key 到 _orphan_api_keys_backup 表...")
# 先备份孤儿数据到临时表,避免数据丢失
op.execute("""
CREATE TABLE IF NOT EXISTS _orphan_api_keys_backup AS
SELECT *, NOW() as backup_at
FROM provider_api_keys
WHERE provider_id IS NULL
""")
# 记录备份的 Key ID
orphan_ids = bind.execute(
sa.text("SELECT id, name FROM provider_api_keys WHERE provider_id IS NULL")
).fetchall()
alembic_logger.info("备份的孤儿 Key 列表:")
for key_id, key_name in orphan_ids:
alembic_logger.info(f" - Key: {key_name} (ID: {key_id})")
# 删除孤儿数据
op.execute("DELETE FROM provider_api_keys WHERE provider_id IS NULL")
alembic_logger.info(f"已备份并删除 {orphan_count} 个孤儿 Key")
# 提供恢复指南
alembic_logger.warning("-" * 60)
alembic_logger.warning("[恢复指南] 如需恢复孤儿 Key:")
alembic_logger.warning(" 1. 查询备份表: SELECT * FROM _orphan_api_keys_backup;")
alembic_logger.warning(" 2. 确定正确的 provider_id")
alembic_logger.warning(" 3. 执行恢复:")
alembic_logger.warning(" INSERT INTO provider_api_keys (...)")
alembic_logger.warning(" SELECT ... FROM _orphan_api_keys_backup WHERE ...;")
alembic_logger.warning("-" * 60)
# 设置 NOT NULL 并创建外键
op.alter_column("provider_api_keys", "provider_id", nullable=False)
if not _constraint_exists("provider_api_keys", "fk_provider_api_keys_provider"):
op.create_foreign_key(
"fk_provider_api_keys_provider",
"provider_api_keys",
"providers",
["provider_id"],
["id"],
ondelete="CASCADE",
)
if not _index_exists("provider_api_keys", "idx_provider_api_keys_provider_id"):
op.create_index("idx_provider_api_keys_provider_id", "provider_api_keys", ["provider_id"])
if not _column_exists("provider_api_keys", "api_formats"):
op.add_column("provider_api_keys", sa.Column("api_formats", sa.JSON(), nullable=True))
# 数据迁移:从 endpoint 获取 api_format
op.execute("""
UPDATE provider_api_keys k
SET api_formats = json_build_array(e.api_format)
FROM provider_endpoints e
WHERE k.endpoint_id = e.id AND k.api_formats IS NULL
""")
op.alter_column("provider_api_keys", "api_formats", nullable=False, server_default="[]")
# 修改 endpoint_id 为可空,外键改为 SET NULL
if _constraint_exists("provider_api_keys", "provider_api_keys_endpoint_id_fkey"):
op.drop_constraint(
"provider_api_keys_endpoint_id_fkey", "provider_api_keys", type_="foreignkey"
)
op.alter_column("provider_api_keys", "endpoint_id", nullable=True)
# 不再重建外键,因为后面会删除这个字段
# ========== 2. provider_api_keys: 添加 rate_multipliers ==========
if not _column_exists("provider_api_keys", "rate_multipliers"):
op.add_column(
"provider_api_keys",
sa.Column("rate_multipliers", postgresql.JSON(astext_type=sa.Text()), nullable=True),
)
# 数据迁移:将 rate_multiplier 按 api_formats 转换
op.execute("""
UPDATE provider_api_keys
SET rate_multipliers = (
SELECT jsonb_object_agg(elem, rate_multiplier)
FROM jsonb_array_elements_text(api_formats::jsonb) AS elem
)
WHERE api_formats IS NOT NULL
AND api_formats::text != '[]'
AND api_formats::text != 'null'
AND rate_multipliers IS NULL
""")
# ========== 3. models: global_model_id 改为可空 ==========
op.alter_column("models", "global_model_id", existing_type=sa.String(36), nullable=True)
# ========== 4. providers: 添加 timeout, max_retries, proxy ==========
if not _column_exists("providers", "timeout"):
op.add_column(
"providers",
sa.Column("timeout", sa.Integer(), nullable=True, comment="请求超时(秒)"),
)
if not _column_exists("providers", "max_retries"):
op.add_column(
"providers",
sa.Column("max_retries", sa.Integer(), nullable=True, comment="最大重试次数"),
)
if not _column_exists("providers", "proxy"):
op.add_column(
"providers",
sa.Column("proxy", postgresql.JSONB(), nullable=True, comment="代理配置"),
)
# 从端点迁移数据到 provider(动态构建 SQL,仅引用存在的列)
ep_has_timeout = _column_exists("provider_endpoints", "timeout")
ep_has_max_retries = _column_exists("provider_endpoints", "max_retries")
ep_has_proxy = _column_exists("provider_endpoints", "proxy")
set_clauses = []
if _column_exists("providers", "timeout"):
if ep_has_timeout:
set_clauses.append("""
timeout = COALESCE(
p.timeout,
(SELECT MAX(e.timeout) FROM provider_endpoints e WHERE e.provider_id = p.id AND e.timeout IS NOT NULL),
300
)""")
else:
set_clauses.append("timeout = COALESCE(p.timeout, 300)")
if _column_exists("providers", "max_retries"):
if ep_has_max_retries:
set_clauses.append("""
max_retries = COALESCE(
p.max_retries,
(SELECT MAX(e.max_retries) FROM provider_endpoints e WHERE e.provider_id = p.id AND e.max_retries IS NOT NULL),
2
)""")
else:
set_clauses.append("max_retries = COALESCE(p.max_retries, 2)")
if _column_exists("providers", "proxy") and ep_has_proxy:
set_clauses.append("""
proxy = COALESCE(
p.proxy,
(SELECT e.proxy FROM provider_endpoints e WHERE e.provider_id = p.id AND e.proxy IS NOT NULL ORDER BY e.created_at LIMIT 1)
)""")
if set_clauses:
where_parts = []
if _column_exists("providers", "timeout"):
where_parts.append("p.timeout IS NULL")
if _column_exists("providers", "max_retries"):
where_parts.append("p.max_retries IS NULL")
where_clause = " OR ".join(where_parts) if where_parts else "TRUE"
sql = "UPDATE providers p SET " + ", ".join(set_clauses) + " WHERE " + where_clause
op.execute(sql)
# ========== 5. providers: display_name -> name ==========
# 注意:这里假设 display_name 已经被重命名为 name
# 如果 display_name 仍然存在,则需要执行重命名
if _column_exists("providers", "display_name"):
# 删除旧的 name 索引
if _index_exists("providers", "ix_providers_name"):
op.drop_index("ix_providers_name", table_name="providers")
# 如果存在旧的 name 列,先删除
if _column_exists("providers", "name"):
op.drop_column("providers", "name")
# 重命名 display_name 为 name
op.alter_column("providers", "display_name", new_column_name="name")
# 创建新索引
op.create_index("ix_providers_name", "providers", ["name"], unique=True)
# ========== 6. provider_api_keys: max_concurrent -> rpm_limit ==========
if _column_exists("provider_api_keys", "max_concurrent"):
op.alter_column("provider_api_keys", "max_concurrent", new_column_name="rpm_limit")
if _column_exists("provider_api_keys", "learned_max_concurrent"):
op.alter_column(
"provider_api_keys", "learned_max_concurrent", new_column_name="learned_rpm_limit"
)
if _column_exists("provider_api_keys", "last_concurrent_peak"):
op.alter_column(
"provider_api_keys", "last_concurrent_peak", new_column_name="last_rpm_peak"
)
# 删除废弃字段
for col in ["rate_limit", "daily_limit", "monthly_limit"]:
if _column_exists("provider_api_keys", col):
op.drop_column("provider_api_keys", col)
# ========== 7. provider_api_keys: 健康度改为按格式存储 ==========
if not _column_exists("provider_api_keys", "health_by_format"):
op.add_column(
"provider_api_keys",
sa.Column(
"health_by_format",
postgresql.JSONB(astext_type=sa.Text()),
nullable=True,
comment="按API格式存储的健康度数据",
),
)
if not _column_exists("provider_api_keys", "circuit_breaker_by_format"):
op.add_column(
"provider_api_keys",
sa.Column(
"circuit_breaker_by_format",
postgresql.JSONB(astext_type=sa.Text()),
nullable=True,
comment="按API格式存储的熔断器状态",
),
)
# 数据迁移:如果存在旧字段,迁移数据到新结构
if _column_exists("provider_api_keys", "health_score"):
op.execute("""
UPDATE provider_api_keys
SET health_by_format = (
SELECT jsonb_object_agg(
elem,
jsonb_build_object(
'health_score', COALESCE(health_score, 1.0),
'consecutive_failures', COALESCE(consecutive_failures, 0),
'last_failure_at', last_failure_at,
'request_results_window', COALESCE(request_results_window::jsonb, '[]'::jsonb)
)
)
FROM jsonb_array_elements_text(api_formats::jsonb) AS elem
)
WHERE api_formats IS NOT NULL
AND api_formats::text != '[]'
AND health_by_format IS NULL
""")
# Circuit Breaker 迁移策略:
# 不复制旧的 circuit_breaker_open 状态到所有 format,而是全部重置为 closed
# 原因:旧的单一 circuit breaker 状态可能因某一个 format 失败而打开,
# 如果复制到所有 format,会导致其他正常工作的 format 被错误标记为不可用
if _column_exists("provider_api_keys", "circuit_breaker_open"):
op.execute("""
UPDATE provider_api_keys
SET circuit_breaker_by_format = (
SELECT jsonb_object_agg(
elem,
jsonb_build_object(
'open', false,
'open_at', NULL,
'next_probe_at', NULL,
'half_open_until', NULL,
'half_open_successes', 0,
'half_open_failures', 0
)
)
FROM jsonb_array_elements_text(api_formats::jsonb) AS elem
)
WHERE api_formats IS NOT NULL
AND api_formats::text != '[]'
AND circuit_breaker_by_format IS NULL
""")
# 设置默认空对象
op.execute("""
UPDATE provider_api_keys
SET health_by_format = '{}'::jsonb
WHERE health_by_format IS NULL
""")
op.execute("""
UPDATE provider_api_keys
SET circuit_breaker_by_format = '{}'::jsonb
WHERE circuit_breaker_by_format IS NULL
""")
# 创建 GIN 索引
if not _index_exists("provider_api_keys", "ix_provider_api_keys_health_by_format"):
op.create_index(
"ix_provider_api_keys_health_by_format",
"provider_api_keys",
["health_by_format"],
postgresql_using="gin",
)
if not _index_exists("provider_api_keys", "ix_provider_api_keys_circuit_breaker_by_format"):
op.create_index(
"ix_provider_api_keys_circuit_breaker_by_format",
"provider_api_keys",
["circuit_breaker_by_format"],
postgresql_using="gin",
)
# 删除旧字段
old_health_columns = [
"health_score",
"consecutive_failures",
"last_failure_at",
"request_results_window",
"circuit_breaker_open",
"circuit_breaker_open_at",
"next_probe_at",
"half_open_until",
"half_open_successes",
"half_open_failures",
]
for col in old_health_columns:
if _column_exists("provider_api_keys", col):
op.drop_column("provider_api_keys", col)
# ========== 8. provider_endpoints: 删除废弃的 rate_limit 列 ==========
if _column_exists("provider_endpoints", "rate_limit"):
op.drop_column("provider_endpoints", "rate_limit")
# ========== 9. usage: 添加 client_response_headers ==========
if not _column_exists("usage", "client_response_headers"):
op.add_column(
"usage",
sa.Column("client_response_headers", sa.JSON(), nullable=True),
)
# ========== 10. provider_api_keys: 删除 endpoint_id ==========
# Key 不再与 Endpoint 绑定,通过 provider_id + api_formats 关联
if _column_exists("provider_api_keys", "endpoint_id"):
# 查找 endpoint_id 上的外键并删除(用 savepoint 保护,避免事务中止)
conn = op.get_bind()
fk_rows = conn.execute(
sa.text(
"SELECT con.conname FROM pg_constraint con "
"JOIN pg_attribute att ON att.attnum = ANY(con.conkey) "
" AND att.attrelid = con.conrelid "
"WHERE con.conrelid = 'provider_api_keys'::regclass "
" AND con.contype = 'f' AND att.attname = 'endpoint_id'"
)
).fetchall()
for (fk_name,) in fk_rows:
conn.execute(sa.text(f"SAVEPOINT sp_drop_fk_{fk_name}"))
try:
op.drop_constraint(fk_name, "provider_api_keys", type_="foreignkey")
conn.execute(sa.text(f"RELEASE SAVEPOINT sp_drop_fk_{fk_name}"))
except Exception:
conn.execute(sa.text(f"ROLLBACK TO SAVEPOINT sp_drop_fk_{fk_name}"))
op.drop_column("provider_api_keys", "endpoint_id")
# ========== 11. provider_endpoints: 删除废弃的 max_concurrent 列 ==========
if _column_exists("provider_endpoints", "max_concurrent"):
op.drop_column("provider_endpoints", "max_concurrent")
# ========== 12. providers: 删除废弃的 RPM 相关字段 ==========
if _column_exists("providers", "rpm_limit"):
op.drop_column("providers", "rpm_limit")
if _column_exists("providers", "rpm_used"):
op.drop_column("providers", "rpm_used")
if _column_exists("providers", "rpm_reset_at"):
op.drop_column("providers", "rpm_reset_at")
alembic_logger.info("[OK] Consolidated migration completed successfully")
def downgrade() -> None:
"""
Downgrade is complex due to data migrations.
For safety, this only removes new columns without restoring old structure.
Manual intervention may be required for full rollback.
"""
bind = op.get_bind()
# 12. 恢复 providers RPM 相关字段
if not _column_exists("providers", "rpm_limit"):
op.add_column("providers", sa.Column("rpm_limit", sa.Integer(), nullable=True))
if not _column_exists("providers", "rpm_used"):
op.add_column(
"providers",
sa.Column("rpm_used", sa.Integer(), server_default="0", nullable=True),
)
if not _column_exists("providers", "rpm_reset_at"):
op.add_column(
"providers",
sa.Column("rpm_reset_at", sa.DateTime(timezone=True), nullable=True),
)
# 11. 恢复 provider_endpoints.max_concurrent
if not _column_exists("provider_endpoints", "max_concurrent"):
op.add_column(
"provider_endpoints", sa.Column("max_concurrent", sa.Integer(), nullable=True)
)
# 10. 恢复 endpoint_id
if not _column_exists("provider_api_keys", "endpoint_id"):
op.add_column("provider_api_keys", sa.Column("endpoint_id", sa.String(36), nullable=True))
# 9. 删除 client_response_headers
if _column_exists("usage", "client_response_headers"):
op.drop_column("usage", "client_response_headers")
# 8. 恢复 provider_endpoints.rate_limit(如果需要)
if not _column_exists("provider_endpoints", "rate_limit"):
op.add_column("provider_endpoints", sa.Column("rate_limit", sa.Integer(), nullable=True))
# 7. 删除健康度 JSON 字段
bind.execute(sa.text("DROP INDEX IF EXISTS ix_provider_api_keys_health_by_format"))
bind.execute(sa.text("DROP INDEX IF EXISTS ix_provider_api_keys_circuit_breaker_by_format"))
if _column_exists("provider_api_keys", "health_by_format"):
op.drop_column("provider_api_keys", "health_by_format")
if _column_exists("provider_api_keys", "circuit_breaker_by_format"):
op.drop_column("provider_api_keys", "circuit_breaker_by_format")
# 6. rpm_limit -> max_concurrent(简化版:仅重命名)
if _column_exists("provider_api_keys", "rpm_limit"):
op.alter_column("provider_api_keys", "rpm_limit", new_column_name="max_concurrent")
if _column_exists("provider_api_keys", "learned_rpm_limit"):
op.alter_column(
"provider_api_keys", "learned_rpm_limit", new_column_name="learned_max_concurrent"
)
if _column_exists("provider_api_keys", "last_rpm_peak"):
op.alter_column(
"provider_api_keys", "last_rpm_peak", new_column_name="last_concurrent_peak"
)
# 恢复已删除的字段
if not _column_exists("provider_api_keys", "rate_limit"):
op.add_column("provider_api_keys", sa.Column("rate_limit", sa.Integer(), nullable=True))
if not _column_exists("provider_api_keys", "daily_limit"):
op.add_column("provider_api_keys", sa.Column("daily_limit", sa.Integer(), nullable=True))
if not _column_exists("provider_api_keys", "monthly_limit"):
op.add_column("provider_api_keys", sa.Column("monthly_limit", sa.Integer(), nullable=True))
# 5. name -> display_name (需要先删除索引)
if _column_exists("providers", "name") and not _column_exists("providers", "display_name"):
if _index_exists("providers", "ix_providers_name"):
op.drop_index("ix_providers_name", table_name="providers")
op.alter_column("providers", "name", new_column_name="display_name")
if not _column_exists("providers", "name"):
op.add_column("providers", sa.Column("name", sa.String(100), nullable=True))
op.execute("""
UPDATE providers
SET name = LOWER(REPLACE(REPLACE(display_name, ' ', '_'), '-', '_'))
""")
op.alter_column("providers", "name", nullable=False)
if not _index_exists("providers", "ix_providers_name"):
op.create_index("ix_providers_name", "providers", ["name"], unique=True)
# 4. 删除 providers 的 timeout, max_retries, proxy
if _column_exists("providers", "proxy"):
op.drop_column("providers", "proxy")
if _column_exists("providers", "max_retries"):
op.drop_column("providers", "max_retries")
if _column_exists("providers", "timeout"):
op.drop_column("providers", "timeout")
# 3. models: global_model_id 改回 NOT NULL
result = bind.execute(sa.text("SELECT COUNT(*) FROM models WHERE global_model_id IS NULL"))
orphan_model_count = result.scalar() or 0
if orphan_model_count > 0:
alembic_logger.warning(
f"[WARN] 发现 {orphan_model_count} 个无 global_model_id 的独立模型,将被删除"
)
op.execute("DELETE FROM models WHERE global_model_id IS NULL")
alembic_logger.info(f"已删除 {orphan_model_count} 个独立模型")
op.alter_column("models", "global_model_id", nullable=False)
# 2. 删除 rate_multipliers
if _column_exists("provider_api_keys", "rate_multipliers"):
op.drop_column("provider_api_keys", "rate_multipliers")
# 1. 删除 provider_id 和 api_formats
if _index_exists("provider_api_keys", "idx_provider_api_keys_provider_id"):
op.drop_index("idx_provider_api_keys_provider_id", table_name="provider_api_keys")
if _constraint_exists("provider_api_keys", "fk_provider_api_keys_provider"):
op.drop_constraint("fk_provider_api_keys_provider", "provider_api_keys", type_="foreignkey")
if _column_exists("provider_api_keys", "api_formats"):
op.drop_column("provider_api_keys", "api_formats")
if _column_exists("provider_api_keys", "provider_id"):
op.drop_column("provider_api_keys", "provider_id")
# 恢复 endpoint_id 外键(简化版:仅创建外键,不强制 NOT NULL)
if _column_exists("provider_api_keys", "endpoint_id"):
if not _constraint_exists("provider_api_keys", "provider_api_keys_endpoint_id_fkey"):
op.create_foreign_key(
"provider_api_keys_endpoint_id_fkey",
"provider_api_keys",
"provider_endpoints",
["endpoint_id"],
["id"],
ondelete="SET NULL",
)
alembic_logger.info("[OK] Downgrade completed (simplified version)")
@@ -1,95 +0,0 @@
"""add auto_fetch_models and locked_models to provider_api_keys
Revision ID: e4ebe3233b40
Revises: m4n5o6p7q8r9
Create Date: 2026-01-13 17:59:53.119479+00:00
为 provider_api_keys 表添加自动获取模型相关字段:
1. auto_fetch_models: 是否启用自动获取模型
2. last_models_fetch_at: 最后获取时间
3. last_models_fetch_error: 最后获取错误信息
4. locked_models: 被锁定的模型列表(刷新时不会被删除)
注意: downgrade 操作会永久删除 auto_fetch_models 配置和 locked_models 数据
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy import inspect
def _index_exists(index_name: str) -> bool:
"""Check if an index exists"""
bind = op.get_bind()
inspector = inspect(bind)
indexes = inspector.get_indexes("provider_api_keys")
return any(idx["name"] == index_name for idx in indexes)
# revision identifiers, used by Alembic.
revision = 'e4ebe3233b40'
down_revision = 'm4n5o6p7q8r9'
branch_labels = None
depends_on = None
def _column_exists(table_name: str, column_name: str) -> bool:
"""Check if a column exists in the table"""
bind = op.get_bind()
inspector = inspect(bind)
columns = [col["name"] for col in inspector.get_columns(table_name)]
return column_name in columns
def upgrade() -> None:
"""添加自动获取模型相关字段"""
if not _column_exists("provider_api_keys", "auto_fetch_models"):
op.add_column(
"provider_api_keys",
sa.Column("auto_fetch_models", sa.Boolean(), nullable=False, server_default="false"),
)
if not _column_exists("provider_api_keys", "last_models_fetch_at"):
op.add_column(
"provider_api_keys",
sa.Column("last_models_fetch_at", sa.DateTime(timezone=True), nullable=True),
)
if not _column_exists("provider_api_keys", "last_models_fetch_error"):
op.add_column(
"provider_api_keys",
sa.Column("last_models_fetch_error", sa.Text(), nullable=True),
)
if not _column_exists("provider_api_keys", "locked_models"):
op.add_column(
"provider_api_keys",
sa.Column("locked_models", sa.JSON(), nullable=True),
)
# 添加复合索引以优化调度器查询
if not _index_exists("ix_provider_api_keys_auto_fetch_active"):
op.create_index(
"ix_provider_api_keys_auto_fetch_active",
"provider_api_keys",
["auto_fetch_models", "is_active"],
postgresql_where=sa.text("auto_fetch_models = true AND is_active = true"),
)
def downgrade() -> None:
"""移除自动获取模型相关字段"""
# 先删除索引
if _index_exists("ix_provider_api_keys_auto_fetch_active"):
op.drop_index("ix_provider_api_keys_auto_fetch_active", table_name="provider_api_keys")
if _column_exists("provider_api_keys", "locked_models"):
op.drop_column("provider_api_keys", "locked_models")
if _column_exists("provider_api_keys", "last_models_fetch_error"):
op.drop_column("provider_api_keys", "last_models_fetch_error")
if _column_exists("provider_api_keys", "last_models_fetch_at"):
op.drop_column("provider_api_keys", "last_models_fetch_at")
if _column_exists("provider_api_keys", "auto_fetch_models"):
op.drop_column("provider_api_keys", "auto_fetch_models")
@@ -1,104 +0,0 @@
"""add header_rules to provider_endpoints and is_locked to api_keys
Revision ID: 6d579000e511
Revises: e4ebe3233b40
Create Date: 2026-01-15 23:00:00.000000+00:00
变更:
1. provider_endpoints 表: 添加 header_rules 字段,迁移 headers 数据
2. api_keys 表: 添加 is_locked 字段(管理员锁定标志)
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects.postgresql import JSON
# revision identifiers, used by Alembic.
revision = '6d579000e511'
down_revision = 'e4ebe3233b40'
branch_labels = None
depends_on = None
def _column_exists(connection, table: str, column: str) -> bool:
"""检查列是否存在"""
result = connection.execute(
sa.text("""
SELECT 1 FROM information_schema.columns
WHERE table_name = :table AND column_name = :column
"""),
{"table": table, "column": column}
)
return result.fetchone() is not None
def upgrade() -> None:
"""添加 header_rules 字段并迁移现有 headers 数据;添加 is_locked 字段"""
connection = op.get_bind()
# ========== provider_endpoints.header_rules ==========
# 1. 添加 header_rules 列(幂等)
if not _column_exists(connection, 'provider_endpoints', 'header_rules'):
op.add_column('provider_endpoints', sa.Column('header_rules', JSON, nullable=True))
# 2. 批量迁移:headers -> header_rules
# 使用纯 SQL 将 {"k1":"v1", "k2":"v2"} 转换为 [{"action":"set","key":"k1","value":"v1"}, ...]
if _column_exists(connection, 'provider_endpoints', 'headers'):
connection.execute(
sa.text("""
UPDATE provider_endpoints
SET header_rules = (
SELECT jsonb_agg(
jsonb_build_object('action', 'set', 'key', key, 'value', value)
)
FROM jsonb_each_text(headers::jsonb)
)
WHERE headers IS NOT NULL
AND headers::text != '{}'
AND jsonb_typeof(headers::jsonb) = 'object'
AND header_rules IS NULL
""")
)
# 3. 删除旧列
op.drop_column('provider_endpoints', 'headers')
# ========== api_keys.is_locked ==========
if not _column_exists(connection, 'api_keys', 'is_locked'):
op.add_column(
'api_keys',
sa.Column('is_locked', sa.Boolean(), nullable=False, server_default='false')
)
def downgrade() -> None:
"""移除 header_rules 字段,恢复 headers 字段;移除 is_locked 字段"""
connection = op.get_bind()
# ========== api_keys.is_locked ==========
if _column_exists(connection, 'api_keys', 'is_locked'):
op.drop_column('api_keys', 'is_locked')
# ========== provider_endpoints.header_rules ==========
# 1. 添加 headers 列(幂等)
if not _column_exists(connection, 'provider_endpoints', 'headers'):
op.add_column('provider_endpoints', sa.Column('headers', JSON, nullable=True))
# 2. 批量迁移:header_rules -> headers(仅提取 set 操作)
if _column_exists(connection, 'provider_endpoints', 'header_rules'):
connection.execute(
sa.text("""
UPDATE provider_endpoints
SET headers = (
SELECT jsonb_object_agg(rule->>'key', rule->>'value')
FROM jsonb_array_elements(header_rules::jsonb) AS rule
WHERE rule->>'action' = 'set'
AND rule->>'key' IS NOT NULL
)
WHERE header_rules IS NOT NULL
AND jsonb_typeof(header_rules::jsonb) = 'array'
AND jsonb_array_length(header_rules::jsonb) > 0
""")
)
# 3. 删除 header_rules 列
op.drop_column('provider_endpoints', 'header_rules')
@@ -1,127 +0,0 @@
"""add global_priority_by_format and remove deprecated fields
Revision ID: ddd59cdf0349
Revises: 6d579000e511
Create Date: 2026-01-16 12:00:00.000000+00:00
变更:
1. provider_api_keys 表: 添加 global_priority_by_format 字段(按 API 格式的全局优先级)
2. 迁移现有 global_priority 数据到新字段
3. 删除已废弃的 global_priority 字段
4. 删除已废弃的 rate_multiplier 字段(已被 rate_multipliers 替代)
5. 删除已废弃的 providers.timeout 字段(由环境变量控制)
6. 删除已废弃的 provider_endpoints.timeout 字段(由环境变量控制)
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects.postgresql import JSON
# revision identifiers, used by Alembic.
revision = 'ddd59cdf0349'
down_revision = '6d579000e511'
branch_labels = None
depends_on = None
def _column_exists(connection, table: str, column: str) -> bool:
"""检查列是否存在"""
result = connection.execute(
sa.text("""
SELECT 1 FROM information_schema.columns
WHERE table_name = :table AND column_name = :column
"""),
{"table": table, "column": column}
)
return result.fetchone() is not None
def upgrade():
connection = op.get_bind()
# 1. 添加 global_priority_by_format 字段
if not _column_exists(connection, 'provider_api_keys', 'global_priority_by_format'):
op.add_column(
'provider_api_keys',
sa.Column('global_priority_by_format', JSON, nullable=True)
)
# 2. 迁移现有 global_priority 数据到新字段
# 对于有 global_priority 的 Key,将其值应用到所有支持的 api_formats
if _column_exists(connection, 'provider_api_keys', 'global_priority'):
# 将 JSON 数组转换为 text[] 后使用 unnest
connection.execute(sa.text("""
UPDATE provider_api_keys
SET global_priority_by_format = (
SELECT jsonb_object_agg(format, global_priority)
FROM jsonb_array_elements_text(api_formats::jsonb) AS format
)
WHERE global_priority IS NOT NULL
AND api_formats IS NOT NULL
AND jsonb_array_length(api_formats::jsonb) > 0
AND global_priority_by_format IS NULL
"""))
# 3. 删除 global_priority 字段
op.drop_column('provider_api_keys', 'global_priority')
# 4. 删除 rate_multiplier 字段(已被 rate_multipliers 替代)
if _column_exists(connection, 'provider_api_keys', 'rate_multiplier'):
op.drop_column('provider_api_keys', 'rate_multiplier')
# 5. 删除 providers.timeout 字段(由环境变量控制)
if _column_exists(connection, 'providers', 'timeout'):
op.drop_column('providers', 'timeout')
# 6. 删除 provider_endpoints.timeout 字段(由环境变量控制)
if _column_exists(connection, 'provider_endpoints', 'timeout'):
op.drop_column('provider_endpoints', 'timeout')
def downgrade():
connection = op.get_bind()
# 1. 恢复 rate_multiplier 字段
if not _column_exists(connection, 'provider_api_keys', 'rate_multiplier'):
op.add_column(
'provider_api_keys',
sa.Column('rate_multiplier', sa.Float, nullable=False, server_default='1.0')
)
# 2. 恢复 global_priority 字段并迁移数据
if not _column_exists(connection, 'provider_api_keys', 'global_priority'):
op.add_column(
'provider_api_keys',
sa.Column('global_priority', sa.Integer, nullable=True)
)
# 从 global_priority_by_format 迁移数据(取第一个格式的优先级值)
if _column_exists(connection, 'provider_api_keys', 'global_priority_by_format'):
connection.execute(sa.text("""
UPDATE provider_api_keys
SET global_priority = (
SELECT (value::text)::integer
FROM jsonb_each(global_priority_by_format::jsonb)
LIMIT 1
)
WHERE global_priority_by_format IS NOT NULL
AND jsonb_typeof(global_priority_by_format::jsonb) = 'object'
AND global_priority IS NULL
"""))
# 3. 删除 global_priority_by_format 字段
if _column_exists(connection, 'provider_api_keys', 'global_priority_by_format'):
op.drop_column('provider_api_keys', 'global_priority_by_format')
# 4. 恢复 providers.timeout 字段
if not _column_exists(connection, 'providers', 'timeout'):
op.add_column(
'providers',
sa.Column('timeout', sa.Integer, nullable=True, server_default='300')
)
# 5. 恢复 provider_endpoints.timeout 字段
if not _column_exists(connection, 'provider_endpoints', 'timeout'):
op.add_column(
'provider_endpoints',
sa.Column('timeout', sa.Integer, nullable=True, server_default='300')
)
@@ -1,223 +0,0 @@
"""make users email/password nullable add email_verified and oauth tables
Revision ID: 33e347f97c0c
Revises: ddd59cdf0349
Create Date: 2026-01-18 11:18:15.940559+00:00
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy import inspect
# revision identifiers, used by Alembic.
revision = "33e347f97c0c"
down_revision = "ddd59cdf0349"
branch_labels = None
depends_on = None
def column_exists(table_name: str, column_name: str) -> bool:
"""检查列是否存在"""
bind = op.get_bind()
inspector = inspect(bind)
columns = [col["name"] for col in inspector.get_columns(table_name)]
return column_name in columns
def table_exists(table_name: str) -> bool:
"""检查表是否存在"""
bind = op.get_bind()
inspector = inspect(bind)
return table_name in inspector.get_table_names()
def column_is_nullable(table_name: str, column_name: str) -> bool:
"""检查列是否允许 NULL"""
bind = op.get_bind()
inspector = inspect(bind)
for col in inspector.get_columns(table_name):
if col["name"] == column_name:
return col["nullable"]
return False
def enum_value_exists(enum_name: str, value: str) -> bool:
"""检查 PostgreSQL ENUM 是否包含指定值"""
bind = op.get_bind()
if bind.dialect.name != "postgresql":
return True # 非 PostgreSQL 跳过检查
result = bind.execute(
sa.text(
"SELECT 1 FROM pg_enum WHERE enumlabel = :value "
"AND enumtypid = (SELECT oid FROM pg_type WHERE typname = :enum_name)"
),
{"value": value, "enum_name": enum_name},
).first()
return result is not None
def upgrade() -> None:
"""应用迁移:升级到新版本"""
bind = op.get_bind()
# ========== Part 1: users 表修改 ==========
# 1) 新增 email_verified
if not column_exists("users", "email_verified"):
op.add_column("users", sa.Column("email_verified", sa.Boolean(), nullable=True))
# 历史数据回填:已有邮箱的用户默认视为已验证
op.execute(sa.text("UPDATE users SET email_verified = true WHERE email IS NOT NULL"))
op.execute(sa.text("UPDATE users SET email_verified = false WHERE email IS NULL"))
# 收紧约束
op.alter_column("users", "email_verified", existing_type=sa.Boolean(), nullable=False)
# 2) email 放宽为可空
if not column_is_nullable("users", "email"):
op.alter_column(
"users",
"email",
existing_type=sa.String(length=255),
nullable=True,
)
# 3) password_hash 放宽为可空
if not column_is_nullable("users", "password_hash"):
op.alter_column(
"users",
"password_hash",
existing_type=sa.String(length=255),
nullable=True,
)
# ========== Part 2: OAuth 相关 ==========
# 4) 扩展 authsource enum
if bind.dialect.name == "postgresql" and not enum_value_exists("authsource", "oauth"):
ctx = op.get_context()
with ctx.autocommit_block():
op.execute("ALTER TYPE authsource ADD VALUE IF NOT EXISTS 'oauth'")
# 5) OAuth provider 配置表
if not table_exists("oauth_providers"):
op.create_table(
"oauth_providers",
sa.Column("provider_type", sa.String(length=50), primary_key=True),
sa.Column("display_name", sa.String(length=100), nullable=False),
sa.Column("client_id", sa.String(length=255), nullable=False),
sa.Column("client_secret_encrypted", sa.Text(), nullable=True),
sa.Column("authorization_url_override", sa.String(length=500), nullable=True),
sa.Column("token_url_override", sa.String(length=500), nullable=True),
sa.Column("userinfo_url_override", sa.String(length=500), nullable=True),
sa.Column("scopes", sa.JSON(), nullable=True),
sa.Column("redirect_uri", sa.String(length=500), nullable=False),
sa.Column("frontend_callback_url", sa.String(length=500), nullable=False),
sa.Column("attribute_mapping", sa.JSON(), nullable=True),
sa.Column("extra_config", sa.JSON(), nullable=True),
sa.Column(
"is_enabled", sa.Boolean(), nullable=False, server_default=sa.text("false")
),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
nullable=False,
server_default=sa.text("CURRENT_TIMESTAMP"),
),
sa.Column(
"updated_at",
sa.DateTime(timezone=True),
nullable=False,
server_default=sa.text("CURRENT_TIMESTAMP"),
),
)
# 6) 用户 OAuth 绑定关系表
if not table_exists("user_oauth_links"):
op.create_table(
"user_oauth_links",
sa.Column("id", sa.String(length=36), primary_key=True),
sa.Column(
"user_id",
sa.String(length=36),
sa.ForeignKey("users.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column(
"provider_type",
sa.String(length=50),
sa.ForeignKey("oauth_providers.provider_type", ondelete="CASCADE"),
nullable=False,
),
sa.Column("provider_user_id", sa.String(length=255), nullable=False),
sa.Column("provider_username", sa.String(length=255), nullable=True),
sa.Column("provider_email", sa.String(length=255), nullable=True),
sa.Column("extra_data", sa.JSON(), nullable=True),
sa.Column(
"linked_at",
sa.DateTime(timezone=True),
nullable=False,
server_default=sa.text("CURRENT_TIMESTAMP"),
),
sa.Column("last_login_at", sa.DateTime(timezone=True), nullable=True),
sa.UniqueConstraint(
"provider_type", "provider_user_id", name="uq_oauth_provider_user"
),
sa.UniqueConstraint("user_id", "provider_type", name="uq_user_oauth_provider"),
)
op.create_index("ix_user_oauth_links_user_id", "user_oauth_links", ["user_id"])
op.create_index(
"ix_user_oauth_links_provider_type", "user_oauth_links", ["provider_type"]
)
def downgrade() -> None:
"""回滚迁移:降级到旧版本"""
bind = op.get_bind()
# ========== Part 2: OAuth 相关(先删除,因为有外键依赖) ==========
if table_exists("user_oauth_links"):
op.drop_index("ix_user_oauth_links_provider_type", table_name="user_oauth_links")
op.drop_index("ix_user_oauth_links_user_id", table_name="user_oauth_links")
op.drop_table("user_oauth_links")
if table_exists("oauth_providers"):
op.drop_table("oauth_providers")
# 注意:Postgres 不支持从 ENUM 删除值,authsource 不回退
# ========== Part 1: users 表修改 ==========
# 降级前检查:避免把包含 NULL 的列强制改回 NOT NULL
has_null_email = bind.execute(
sa.text("SELECT 1 FROM users WHERE email IS NULL LIMIT 1")
).first()
if has_null_email:
raise RuntimeError("Cannot downgrade: users.email contains NULL values")
has_null_password = bind.execute(
sa.text("SELECT 1 FROM users WHERE password_hash IS NULL LIMIT 1")
).first()
if has_null_password:
raise RuntimeError("Cannot downgrade: users.password_hash contains NULL values")
# 恢复 NOT NULL 约束
if column_is_nullable("users", "email"):
op.alter_column(
"users",
"email",
existing_type=sa.String(length=255),
nullable=False,
)
if column_is_nullable("users", "password_hash"):
op.alter_column(
"users",
"password_hash",
existing_type=sa.String(length=255),
nullable=False,
)
if column_exists("users", "email_verified"):
op.drop_column("users", "email_verified")
@@ -1,65 +0,0 @@
"""add_stats_daily_provider_table
Revision ID: c868729753ad
Revises: 33e347f97c0c
Create Date: 2026-01-19 05:19:49.634662+00:00
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy import inspect
# revision identifiers, used by Alembic.
revision = 'c868729753ad'
down_revision = '33e347f97c0c'
branch_labels = None
depends_on = None
def table_exists(table_name: str) -> bool:
"""检查表是否存在"""
bind = op.get_bind()
inspector = inspect(bind)
return table_name in inspector.get_table_names()
def index_exists(table_name: str, index_name: str) -> bool:
"""检查索引是否存在"""
bind = op.get_bind()
inspector = inspect(bind)
indexes = [idx['name'] for idx in inspector.get_indexes(table_name)]
return index_name in indexes
def upgrade() -> None:
"""应用迁移:升级到新版本"""
if not table_exists('stats_daily_provider'):
op.create_table(
'stats_daily_provider',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('date', sa.DateTime(timezone=True), nullable=False),
sa.Column('provider_name', sa.String(length=100), nullable=False),
sa.Column('total_requests', sa.Integer(), nullable=False),
sa.Column('input_tokens', sa.BigInteger(), nullable=False),
sa.Column('output_tokens', sa.BigInteger(), nullable=False),
sa.Column('cache_creation_tokens', sa.BigInteger(), nullable=False),
sa.Column('cache_read_tokens', sa.BigInteger(), nullable=False),
sa.Column('total_cost', sa.Float(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint('id'),
sa.UniqueConstraint('date', 'provider_name', name='uq_stats_daily_provider')
)
op.create_index('idx_stats_daily_provider_date', 'stats_daily_provider', ['date'], unique=False)
op.create_index('idx_stats_daily_provider_date_provider', 'stats_daily_provider', ['date', 'provider_name'], unique=False)
def downgrade() -> None:
"""回滚迁移:降级到旧版本"""
if table_exists('stats_daily_provider'):
if index_exists('stats_daily_provider', 'idx_stats_daily_provider_date_provider'):
op.drop_index('idx_stats_daily_provider_date_provider', table_name='stats_daily_provider')
if index_exists('stats_daily_provider', 'idx_stats_daily_provider_date'):
op.drop_index('idx_stats_daily_provider_date', table_name='stats_daily_provider')
op.drop_table('stats_daily_provider')
@@ -1,51 +0,0 @@
"""add_format_acceptance_config_to_provider_endpoints
Revision ID: 4b4c7b0df1a2
Revises: c868729753ad
Create Date: 2026-01-21 18:45:00+00:00
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy import inspect
# revision identifiers, used by Alembic.
revision = "4b4c7b0df1a2"
down_revision = "c868729753ad"
branch_labels = None
depends_on = None
def table_exists(table_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
return table_name in inspector.get_table_names()
def column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
columns = [col["name"] for col in inspector.get_columns(table_name)]
return column_name in columns
def upgrade() -> None:
if not table_exists("provider_endpoints"):
return
if column_exists("provider_endpoints", "format_acceptance_config"):
return
op.add_column(
"provider_endpoints",
sa.Column("format_acceptance_config", sa.JSON(), nullable=True),
)
def downgrade() -> None:
if not table_exists("provider_endpoints"):
return
if not column_exists("provider_endpoints", "format_acceptance_config"):
return
op.drop_column("provider_endpoints", "format_acceptance_config")
@@ -1,114 +0,0 @@
"""add_format_conversion_tracking_and_model_filter_patterns_and_provider_timeout
Revision ID: f7c8d9e0a1b2
Revises: 4b4c7b0df1a2
Create Date: 2026-01-27 10:00:00+00:00
Changes:
1. usage 表: 添加 endpoint_api_format 和 has_format_conversion 字段
2. provider_api_keys 表: 添加 model_include_patterns 和 model_exclude_patterns 字段
- 支持通配符规则自动过滤从上游获取的模型列表
- 包含规则和排除规则(支持 * 和 ? 通配符)
3. providers 表: 添加 stream_first_byte_timeout 和 request_timeout 字段
- 允许每个提供商单独配置超时时间
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy import inspect
# revision identifiers, used by Alembic.
revision = "f7c8d9e0a1b2"
down_revision = "4b4c7b0df1a2"
branch_labels = None
depends_on = None
def table_exists(table_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
return table_name in inspector.get_table_names()
def column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
columns = [col["name"] for col in inspector.get_columns(table_name)]
return column_name in columns
def upgrade() -> None:
# === usage 表: 格式转换追踪 ===
if table_exists("usage"):
# 添加 endpoint_api_format 字段(端点原生 API 格式)
if not column_exists("usage", "endpoint_api_format"):
op.add_column(
"usage",
sa.Column("endpoint_api_format", sa.String(50), nullable=True),
)
# 添加 has_format_conversion 字段(是否发生了格式转换)
if not column_exists("usage", "has_format_conversion"):
op.add_column(
"usage",
sa.Column("has_format_conversion", sa.Boolean(), nullable=True, server_default="false"),
)
# === provider_api_keys 表: 模型过滤规则 ===
if table_exists("provider_api_keys"):
# 添加 model_include_patterns 字段(包含规则,支持 * 和 ? 通配符)
if not column_exists("provider_api_keys", "model_include_patterns"):
op.add_column(
"provider_api_keys",
sa.Column("model_include_patterns", sa.JSON(), nullable=True),
)
# 添加 model_exclude_patterns 字段(排除规则,支持 * 和 ? 通配符)
if not column_exists("provider_api_keys", "model_exclude_patterns"):
op.add_column(
"provider_api_keys",
sa.Column("model_exclude_patterns", sa.JSON(), nullable=True),
)
# === providers 表: 超时配置 ===
if table_exists("providers"):
# 添加 stream_first_byte_timeout 字段(流式请求首字节超时)
if not column_exists("providers", "stream_first_byte_timeout"):
op.add_column(
"providers",
sa.Column("stream_first_byte_timeout", sa.Float(), nullable=True),
)
# 添加 request_timeout 字段(非流式请求整体超时)
if not column_exists("providers", "request_timeout"):
op.add_column(
"providers",
sa.Column("request_timeout", sa.Float(), nullable=True),
)
def downgrade() -> None:
# === providers 表: 移除超时配置 ===
if table_exists("providers"):
if column_exists("providers", "request_timeout"):
op.drop_column("providers", "request_timeout")
if column_exists("providers", "stream_first_byte_timeout"):
op.drop_column("providers", "stream_first_byte_timeout")
# === provider_api_keys 表: 移除模型过滤规则 ===
if table_exists("provider_api_keys"):
if column_exists("provider_api_keys", "model_exclude_patterns"):
op.drop_column("provider_api_keys", "model_exclude_patterns")
if column_exists("provider_api_keys", "model_include_patterns"):
op.drop_column("provider_api_keys", "model_include_patterns")
# === usage 表: 移除格式转换追踪 ===
if table_exists("usage"):
if column_exists("usage", "has_format_conversion"):
op.drop_column("usage", "has_format_conversion")
if column_exists("usage", "endpoint_api_format"):
op.drop_column("usage", "endpoint_api_format")
@@ -1,58 +0,0 @@
"""add_keep_priority_on_conversion_to_providers
Revision ID: 364680d1bc99
Revises: f7c8d9e0a1b2
Create Date: 2026-01-28 12:00:00+00:00
Changes:
1. providers 表: 添加 keep_priority_on_conversion 字段
- 格式转换时是否保持提供商原优先级
- 默认 False:需要格式转换时,候选会被降级到不需要转换的候选之后
- 设为 True:即使需要格式转换,也保持原优先级排名
"""
from alembic import op
import sqlalchemy as sa
from sqlalchemy import inspect
# revision identifiers, used by Alembic.
revision = "364680d1bc99"
down_revision = "f7c8d9e0a1b2"
branch_labels = None
depends_on = None
def table_exists(table_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
return table_name in inspector.get_table_names()
def column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
columns = [col["name"] for col in inspector.get_columns(table_name)]
return column_name in columns
def upgrade() -> None:
# === providers 表: 添加格式转换优先级保持配置 ===
if table_exists("providers"):
if not column_exists("providers", "keep_priority_on_conversion"):
op.add_column(
"providers",
sa.Column(
"keep_priority_on_conversion",
sa.Boolean(),
nullable=False,
server_default="false",
),
)
def downgrade() -> None:
# === providers 表: 移除格式转换优先级保持配置 ===
if table_exists("providers"):
if column_exists("providers", "keep_priority_on_conversion"):
op.drop_column("providers", "keep_priority_on_conversion")
@@ -1,51 +0,0 @@
"""Add auth_type and auth_config fields to provider_api_keys table
Revision ID: 7f6f8065f517
Revises: 364680d1bc99
Create Date: 2026-01-30 10:00:00.000000
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
from sqlalchemy import inspect
# revision identifiers, used by Alembic.
revision: str = "7f6f8065f517"
down_revision: Union[str, None] = "364680d1bc99"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def column_exists(table_name: str, column_name: str) -> bool:
"""检查列是否已存在"""
bind = op.get_bind()
inspector = inspect(bind)
columns = [col["name"] for col in inspector.get_columns(table_name)]
return column_name in columns
def upgrade() -> None:
# 添加 auth_type 字段,默认值为 "api_key"
if not column_exists("provider_api_keys", "auth_type"):
op.add_column(
"provider_api_keys",
sa.Column("auth_type", sa.String(20), nullable=False, server_default="api_key"),
)
# 添加 auth_config 字段(Text,存储加密后的认证配置)
if not column_exists("provider_api_keys", "auth_config"):
op.add_column(
"provider_api_keys",
sa.Column("auth_config", sa.Text, nullable=True),
)
def downgrade() -> None:
if column_exists("provider_api_keys", "auth_config"):
op.drop_column("provider_api_keys", "auth_config")
if column_exists("provider_api_keys", "auth_type"):
op.drop_column("provider_api_keys", "auth_type")
@@ -1,112 +0,0 @@
"""Add video_tasks table
Revision ID: b6f1a2c5d8e9
Revises: 7f6f8065f517
Create Date: 2026-01-30 18:00:00.000000
"""
from typing import Sequence, Union
import sqlalchemy as sa
from sqlalchemy import inspect
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "b6f1a2c5d8e9"
down_revision: Union[str, None] = "7f6f8065f517"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def table_exists(table_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
return table_name in inspector.get_table_names()
def upgrade() -> None:
if table_exists("video_tasks"):
return
op.create_table(
"video_tasks",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("external_task_id", sa.String(200), nullable=True, index=False),
sa.Column("user_id", sa.String(36), sa.ForeignKey("users.id"), nullable=False),
sa.Column("api_key_id", sa.String(36), sa.ForeignKey("api_keys.id"), nullable=True),
sa.Column("provider_id", sa.String(36), sa.ForeignKey("providers.id"), nullable=True),
sa.Column(
"endpoint_id", sa.String(36), sa.ForeignKey("provider_endpoints.id"), nullable=True
),
sa.Column("key_id", sa.String(36), sa.ForeignKey("provider_api_keys.id"), nullable=True),
sa.Column("client_api_format", sa.String(50), nullable=False),
sa.Column("provider_api_format", sa.String(50), nullable=False),
sa.Column("format_converted", sa.Boolean(), server_default=sa.false()),
sa.Column("model", sa.String(100), nullable=False),
sa.Column("prompt", sa.Text(), nullable=False),
sa.Column("original_request_body", sa.JSON(), nullable=True),
sa.Column("converted_request_body", sa.JSON(), nullable=True),
sa.Column("duration_seconds", sa.Integer(), server_default=sa.text("4")),
sa.Column("resolution", sa.String(20), server_default=sa.text("'720p'")),
sa.Column("aspect_ratio", sa.String(10), server_default=sa.text("'16:9'")),
sa.Column("size", sa.String(20), nullable=True),
sa.Column("status", sa.String(20), server_default=sa.text("'pending'")),
sa.Column("progress_percent", sa.Integer(), server_default=sa.text("0")),
sa.Column("progress_message", sa.String(500), nullable=True),
sa.Column("video_url", sa.String(2000), nullable=True),
sa.Column("video_urls", sa.JSON(), nullable=True),
sa.Column("thumbnail_url", sa.String(2000), nullable=True),
sa.Column("video_size_bytes", sa.BigInteger(), nullable=True),
sa.Column("video_expires_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("stored_video_path", sa.String(500), nullable=True),
sa.Column("storage_provider", sa.String(50), nullable=True),
sa.Column("error_code", sa.String(50), nullable=True),
sa.Column("error_message", sa.Text(), nullable=True),
sa.Column("retry_count", sa.Integer(), server_default=sa.text("0")),
sa.Column("max_retries", sa.Integer(), server_default=sa.text("3")),
sa.Column("poll_interval_seconds", sa.Integer(), server_default=sa.text("10")),
sa.Column("next_poll_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("poll_count", sa.Integer(), server_default=sa.text("0")),
sa.Column("max_poll_count", sa.Integer(), server_default=sa.text("360")),
sa.Column(
"remixed_from_task_id",
sa.String(36),
sa.ForeignKey("video_tasks.id", ondelete="SET NULL"),
nullable=True,
),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("CURRENT_TIMESTAMP"),
),
sa.Column("submitted_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("completed_at", sa.DateTime(timezone=True), nullable=True),
sa.Column(
"updated_at",
sa.DateTime(timezone=True),
server_default=sa.text("CURRENT_TIMESTAMP"),
),
)
op.create_index("idx_video_tasks_user_status", "video_tasks", ["user_id", "status"])
op.create_index("idx_video_tasks_next_poll", "video_tasks", ["next_poll_at"])
op.create_index("idx_video_tasks_external_id", "video_tasks", ["external_task_id"])
# 唯一约束:同一用户不能有重复的 external_task_id
op.create_unique_constraint(
"uq_video_tasks_user_external_id",
"video_tasks",
["user_id", "external_task_id"],
)
def downgrade() -> None:
if not table_exists("video_tasks"):
return
op.drop_constraint("uq_video_tasks_user_external_id", "video_tasks", type_="unique")
op.drop_index("idx_video_tasks_external_id", table_name="video_tasks")
op.drop_index("idx_video_tasks_next_poll", table_name="video_tasks")
op.drop_index("idx_video_tasks_user_status", table_name="video_tasks")
op.drop_table("video_tasks")
@@ -1,180 +0,0 @@
"""Add billing system tables and video_tasks.request_metadata
Revision ID: c8d2e4f6a1b3
Revises: b6f1a2c5d8e9
Create Date: 2026-01-31 12:00:00.000000
"""
from __future__ import annotations
from typing import Sequence, Union
import sqlalchemy as sa
from sqlalchemy import inspect
from sqlalchemy.dialects.postgresql import JSONB
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "c8d2e4f6a1b3"
down_revision: Union[str, None] = "b6f1a2c5d8e9"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def table_exists(table_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
return table_name in inspector.get_table_names()
def column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
columns = [col["name"] for col in inspector.get_columns(table_name)]
return column_name in columns
def index_exists(table_name: str, index_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
try:
indexes = inspector.get_indexes(table_name)
except Exception:
return False
return any(idx.get("name") == index_name for idx in indexes)
def upgrade() -> None:
# ==================== video_tasks.request_metadata ====================
if not column_exists("video_tasks", "request_metadata"):
op.add_column(
"video_tasks",
sa.Column("request_metadata", sa.JSON(), nullable=True),
)
# ==================== billing_rules ====================
if not table_exists("billing_rules"):
op.create_table(
"billing_rules",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column(
"global_model_id",
sa.String(36),
sa.ForeignKey("global_models.id", ondelete="CASCADE"),
nullable=True,
),
sa.Column(
"model_id",
sa.String(36),
sa.ForeignKey("models.id", ondelete="CASCADE"),
nullable=True,
),
sa.Column("name", sa.String(100), nullable=False),
sa.Column("task_type", sa.String(20), nullable=False, server_default="chat"),
sa.Column("expression", sa.Text(), nullable=False),
sa.Column("variables", JSONB, nullable=False, server_default=sa.text("'{}'::jsonb")),
sa.Column(
"dimension_mappings", JSONB, nullable=False, server_default=sa.text("'{}'::jsonb")
),
sa.Column("is_enabled", sa.Boolean(), nullable=False, server_default=sa.text("true")),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
nullable=False,
server_default=sa.text("now()"),
),
sa.Column(
"updated_at",
sa.DateTime(timezone=True),
nullable=False,
server_default=sa.text("now()"),
),
sa.CheckConstraint(
"(global_model_id IS NOT NULL AND model_id IS NULL) OR "
"(global_model_id IS NULL AND model_id IS NOT NULL)",
name="chk_billing_rules_model_ref",
),
)
# Partial unique indexes for enabled rules
if table_exists("billing_rules"):
if not index_exists("billing_rules", "uq_billing_rules_global_model_task"):
op.create_index(
"uq_billing_rules_global_model_task",
"billing_rules",
["global_model_id", "task_type"],
unique=True,
postgresql_where=sa.text("is_enabled = TRUE AND global_model_id IS NOT NULL"),
)
if not index_exists("billing_rules", "uq_billing_rules_model_task"):
op.create_index(
"uq_billing_rules_model_task",
"billing_rules",
["model_id", "task_type"],
unique=True,
postgresql_where=sa.text("is_enabled = TRUE AND model_id IS NOT NULL"),
)
# ==================== dimension_collectors ====================
if not table_exists("dimension_collectors"):
op.create_table(
"dimension_collectors",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("api_format", sa.String(50), nullable=False),
sa.Column("task_type", sa.String(20), nullable=False),
sa.Column("dimension_name", sa.String(100), nullable=False),
sa.Column("source_type", sa.String(20), nullable=False),
sa.Column("source_path", sa.String(200), nullable=True),
sa.Column("value_type", sa.String(20), nullable=False, server_default="float"),
sa.Column("transform_expression", sa.Text(), nullable=True),
sa.Column("default_value", sa.String(100), nullable=True),
sa.Column("priority", sa.Integer(), nullable=False, server_default="0"),
sa.Column("is_enabled", sa.Boolean(), nullable=False, server_default=sa.text("true")),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
nullable=False,
server_default=sa.text("now()"),
),
sa.Column(
"updated_at",
sa.DateTime(timezone=True),
nullable=False,
server_default=sa.text("now()"),
),
sa.CheckConstraint(
"(source_type = 'computed' AND source_path IS NULL AND transform_expression IS NOT NULL) OR "
"(source_type != 'computed' AND source_path IS NOT NULL)",
name="chk_dimension_collectors_source_config",
),
)
if table_exists("dimension_collectors"):
if not index_exists("dimension_collectors", "uq_dimension_collectors_enabled"):
op.create_index(
"uq_dimension_collectors_enabled",
"dimension_collectors",
["api_format", "task_type", "dimension_name", "priority"],
unique=True,
postgresql_where=sa.text("is_enabled = TRUE"),
)
def downgrade() -> None:
# Drop in reverse order
if table_exists("dimension_collectors"):
if index_exists("dimension_collectors", "uq_dimension_collectors_enabled"):
op.drop_index("uq_dimension_collectors_enabled", table_name="dimension_collectors")
op.drop_table("dimension_collectors")
if table_exists("billing_rules"):
if index_exists("billing_rules", "uq_billing_rules_model_task"):
op.drop_index("uq_billing_rules_model_task", table_name="billing_rules")
if index_exists("billing_rules", "uq_billing_rules_global_model_task"):
op.drop_index("uq_billing_rules_global_model_task", table_name="billing_rules")
op.drop_table("billing_rules")
if column_exists("video_tasks", "request_metadata"):
op.drop_column("video_tasks", "request_metadata")
@@ -1,462 +0,0 @@
"""Add api_family/endpoint_kind and migrate api_format to endpoint signature keys
Revision ID: cf40e6a5c5b1
Revises: c8d2e4f6a1b3
Create Date: 2026-01-31 15:30:00.000000
"""
from __future__ import annotations
import json
from datetime import datetime, timezone
from typing import Sequence, Union
from uuid import uuid4
import sqlalchemy as sa
from sqlalchemy import inspect, text
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "cf40e6a5c5b1"
down_revision: Union[str, None] = "c8d2e4f6a1b3"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def _json_loads(val):
if val is None:
return None
if isinstance(val, (dict, list)):
return val
if isinstance(val, str):
try:
return json.loads(val)
except Exception:
return None
return None
def _json_dumps(val):
"""将 dict/list 转为 JSON 字符串,None 保持 None"""
if val is None:
return None
if isinstance(val, str):
return val
return json.dumps(val)
def _normalize_signature(value: str | None) -> str | None:
"""
Normalize legacy api_format / signature-ish strings to canonical signature key.
- canonical: `<family>:<kind>` (lowercase)
- legacy examples: "OPENAI", "OPENAI_CLI", "GEMINI_VIDEO"
"""
if value is None:
return None
raw = str(value).strip()
if not raw:
return None
if ":" in raw:
fam, kind = raw.split(":", 1)
fam = fam.strip().lower()
kind = kind.strip().lower()
if not fam or not kind:
return None
return f"{fam}:{kind}"
upper = raw.upper()
if upper.startswith("CLAUDE"):
fam = "claude"
elif upper.startswith("OPENAI"):
fam = "openai"
elif upper.startswith("GEMINI"):
fam = "gemini"
else:
return None
kind = "chat"
if upper.endswith("_CLI"):
kind = "cli"
elif upper.endswith("_VIDEO"):
kind = "video"
return f"{fam}:{kind}"
def _normalize_signature_list(values) -> list[str] | None:
if values is None:
return None
if isinstance(values, str):
values = _json_loads(values)
if not isinstance(values, list):
return None
out: list[str] = []
seen: set[str] = set()
for v in values:
sig = _normalize_signature(str(v) if v is not None else None)
if not sig:
continue
if sig in seen:
continue
seen.add(sig)
out.append(sig)
return out
def _normalize_signature_dict(values) -> dict | None:
if values is None:
return None
if isinstance(values, str):
values = _json_loads(values)
if not isinstance(values, dict):
return None
out: dict = {}
for k, v in values.items():
sig = _normalize_signature(str(k) if k is not None else None)
if not sig:
continue
out[sig] = v
return out
def _add_video_variants(formats: list[str]) -> list[str]:
"""
保持原有格式,不自动补齐 video 变体。
"""
return formats
def table_exists(table_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
return table_name in inspector.get_table_names()
def column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
columns = [col["name"] for col in inspector.get_columns(table_name)]
return column_name in columns
def index_exists(table_name: str, index_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
try:
indexes = inspector.get_indexes(table_name)
except Exception:
return False
return any(idx.get("name") == index_name for idx in indexes)
def _migrate_format_acceptance_config(cfg) -> dict | None:
cfg_obj = _json_loads(cfg)
if not isinstance(cfg_obj, dict):
return cfg_obj if cfg_obj is None else None
for key in ("accept_formats", "reject_formats"):
raw = cfg_obj.get(key)
if not isinstance(raw, list):
continue
normalized = _normalize_signature_list(raw) or []
cfg_obj[key] = normalized
return cfg_obj
def migrate_provider_endpoints(connection) -> None:
"""
- 将 provider_endpoints.api_format 统一迁移为 signature key(小写)
- 填充/校准 api_family / endpoint_kind
- 迁移 format_acceptance_config 中的 accept/reject formats
"""
rows = connection.execute(text("""
SELECT
id,
api_format,
api_family,
endpoint_kind,
format_acceptance_config
FROM provider_endpoints
""")).fetchall()
for row in rows:
sig = _normalize_signature(row.api_format)
if not sig:
continue
fam, kind = sig.split(":", 1)
cfg = _migrate_format_acceptance_config(row.format_acceptance_config)
connection.execute(
text("""
UPDATE provider_endpoints
SET
api_format = :api_format,
api_family = :api_family,
endpoint_kind = :endpoint_kind,
format_acceptance_config = CAST(:format_acceptance_config AS json)
WHERE id = :id
"""),
{
"id": row.id,
"api_format": sig,
"api_family": fam,
"endpoint_kind": kind,
"format_acceptance_config": _json_dumps(cfg),
},
)
def create_video_endpoints(connection) -> None:
"""
不再自动创建 video endpoint,保持原有配置。
"""
pass
def migrate_provider_api_keys(connection) -> None:
"""
迁移 provider_api_keys:
- api_formats -> signature keys(并补齐 video 变体)
- dict 字段 key -> signature keys(rate_multipliers/global_priority/health/circuit_breaker)
- rate_multipliers/global_priority_by_format 复制 chat -> video(如 openai:chat -> openai:video)
"""
rows = connection.execute(text("""
SELECT
id,
api_formats,
rate_multipliers,
global_priority_by_format,
health_by_format,
circuit_breaker_by_format
FROM provider_api_keys
""")).fetchall()
for row in rows:
api_formats = _normalize_signature_list(row.api_formats)
if api_formats is not None:
api_formats = _add_video_variants(api_formats)
rate_multipliers = _normalize_signature_dict(row.rate_multipliers)
global_priority_by_format = _normalize_signature_dict(row.global_priority_by_format)
health_by_format = _normalize_signature_dict(row.health_by_format)
circuit_breaker_by_format = _normalize_signature_dict(row.circuit_breaker_by_format)
connection.execute(
text("""
UPDATE provider_api_keys
SET
api_formats = CAST(:api_formats AS json),
rate_multipliers = CAST(:rate_multipliers AS json),
global_priority_by_format = CAST(:global_priority_by_format AS json),
health_by_format = CAST(:health_by_format AS json),
circuit_breaker_by_format = CAST(:circuit_breaker_by_format AS json)
WHERE id = :id
"""),
{
"id": row.id,
"api_formats": _json_dumps(api_formats),
"rate_multipliers": _json_dumps(rate_multipliers),
"global_priority_by_format": _json_dumps(global_priority_by_format),
"health_by_format": _json_dumps(health_by_format),
"circuit_breaker_by_format": _json_dumps(circuit_breaker_by_format),
},
)
def migrate_allowed_api_formats(connection, *, table_name: str) -> None:
"""迁移 users/api_keys.allowed_api_formats 为 signature keys(并补齐 video 变体)。"""
if not table_exists(table_name):
return
rows = connection.execute(text(f"""
SELECT id, allowed_api_formats
FROM {table_name}
""")).fetchall()
for row in rows:
allowed = _normalize_signature_list(row.allowed_api_formats)
if allowed is None:
continue
allowed = _add_video_variants(allowed)
connection.execute(
text(f"""
UPDATE {table_name}
SET allowed_api_formats = CAST(:allowed_api_formats AS json)
WHERE id = :id
"""),
{"id": row.id, "allowed_api_formats": _json_dumps(allowed)},
)
def migrate_video_tasks(connection) -> None:
"""
video_tasks.*_api_format 迁移为 signature keys。
注意:video_tasks 表天然是 video 任务,因此将 openai/gemini 的 kind 强制归一为 video,
以兼容历史上复用 chat 格式存储的旧记录。
"""
if not table_exists("video_tasks"):
return
rows = connection.execute(text("""
SELECT id, client_api_format, provider_api_format
FROM video_tasks
""")).fetchall()
for row in rows:
client_sig = _normalize_signature(row.client_api_format) or ""
provider_sig = _normalize_signature(row.provider_api_format) or ""
def _force_video(sig: str) -> str:
if not sig or ":" not in sig:
return sig
fam, _kind = sig.split(":", 1)
fam = fam.strip().lower()
if fam in ("openai", "gemini"):
return f"{fam}:video"
return sig
client_sig = _force_video(client_sig)
provider_sig = _force_video(provider_sig)
if not client_sig or not provider_sig:
continue
connection.execute(
text("""
UPDATE video_tasks
SET client_api_format = :client_api_format,
provider_api_format = :provider_api_format
WHERE id = :id
"""),
{
"id": row.id,
"client_api_format": client_sig,
"provider_api_format": provider_sig,
},
)
def migrate_model_provider_mappings(connection) -> None:
"""迁移 models.provider_model_mappings[*].api_formats 为 signature keys。"""
if not table_exists("models"):
return
rows = connection.execute(text("""
SELECT id, provider_model_mappings
FROM models
WHERE provider_model_mappings IS NOT NULL
""")).fetchall()
for row in rows:
mappings = _json_loads(row.provider_model_mappings)
if not isinstance(mappings, list):
continue
changed = False
new_mappings: list = []
for item in mappings:
if not isinstance(item, dict):
new_mappings.append(item)
continue
raw_formats = item.get("api_formats")
if isinstance(raw_formats, list):
normalized = _normalize_signature_list(raw_formats) or []
# 内容比较(而非引用比较),避免已迁移数据被无意义地重复 UPDATE
if set(normalized) != set(raw_formats):
changed = True
item = dict(item)
item["api_formats"] = normalized
new_mappings.append(item)
if not changed:
continue
connection.execute(
text("""
UPDATE models
SET provider_model_mappings = CAST(:provider_model_mappings AS json)
WHERE id = :id
"""),
{"id": row.id, "provider_model_mappings": _json_dumps(new_mappings)},
)
def migrate_dimension_collectors(connection) -> None:
"""迁移 dimension_collectors.api_format 为 signature keys(如果存在历史数据)。"""
if not table_exists("dimension_collectors"):
return
rows = connection.execute(text("""
SELECT id, api_format
FROM dimension_collectors
WHERE api_format IS NOT NULL
""")).fetchall()
for row in rows:
sig = _normalize_signature(row.api_format)
if not sig:
continue
connection.execute(
text("""
UPDATE dimension_collectors
SET api_format = :api_format
WHERE id = :id
"""),
{"id": row.id, "api_format": sig},
)
def upgrade() -> None:
if not table_exists("provider_endpoints"):
return
# ==================== provider_endpoints.api_family / endpoint_kind ====================
if not column_exists("provider_endpoints", "api_family"):
op.add_column("provider_endpoints", sa.Column("api_family", sa.String(50), nullable=True))
if not column_exists("provider_endpoints", "endpoint_kind"):
op.add_column(
"provider_endpoints", sa.Column("endpoint_kind", sa.String(50), nullable=True)
)
# ==================== idx_provider_family_kind ====================
if not index_exists("provider_endpoints", "idx_provider_family_kind"):
op.create_index(
"idx_provider_family_kind",
"provider_endpoints",
["provider_id", "api_family", "endpoint_kind"],
)
# ==================== data migrations (idempotent) ====================
conn = op.get_bind()
migrate_provider_endpoints(conn)
create_video_endpoints(conn)
if table_exists("provider_api_keys"):
migrate_provider_api_keys(conn)
migrate_allowed_api_formats(conn, table_name="users")
migrate_allowed_api_formats(conn, table_name="api_keys")
migrate_video_tasks(conn)
migrate_model_provider_mappings(conn)
migrate_dimension_collectors(conn)
def downgrade() -> None:
# Drop index/columns only; data changes are intentionally kept (safe rollback strategy).
if table_exists("provider_endpoints"):
if index_exists("provider_endpoints", "idx_provider_family_kind"):
op.drop_index("idx_provider_family_kind", table_name="provider_endpoints")
if column_exists("provider_endpoints", "endpoint_kind"):
op.drop_column("provider_endpoints", "endpoint_kind")
if column_exists("provider_endpoints", "api_family"):
op.drop_column("provider_endpoints", "api_family")
@@ -1,329 +0,0 @@
"""Add usage billing, video_tasks fields, gemini_file_mappings, provider format conversion, and indexes
Revision ID: a2f1b3c4d5e6
Revises: cf40e6a5c5b1
Create Date: 2026-02-01 12:00:00+00:00
Changes:
1. usage 表:
- 添加 billing_status (pending/settled/void),用于表示结算状态
- 添加 finalized_at,用于记录结算完成时间
- 添加 (provider_name, created_at) 和 (model, created_at) 索引
2. video_tasks 表:
- 添加 request_id(全局唯一),用于与 Usage/RequestCandidate 建立稳定关联
- 添加 short_id (Gemini-style short ID)
3. gemini_file_mappings 表:
- 创建新表用于文件映射
- 添加 source_hash 字段用于关联相同源文件
4. providers 表:
- 添加 enable_format_conversion 开关字段
5. request_candidates 表:
- 添加 created_at 索引
"""
from __future__ import annotations
import secrets
import string
from typing import Sequence, Union
import sqlalchemy as sa
from sqlalchemy import inspect, text
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "a2f1b3c4d5e6"
down_revision: Union[str, None] = "cf40e6a5c5b1"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def table_exists(table_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
inspector.clear_cache()
return table_name in inspector.get_table_names()
def column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
# Clear cached schema info to get fresh data
inspector.clear_cache()
columns = [col["name"] for col in inspector.get_columns(table_name)]
return column_name in columns
def index_exists(table_name: str, index_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
inspector.clear_cache()
indexes = inspector.get_indexes(table_name)
return any(idx.get("name") == index_name for idx in indexes)
def unique_constraint_exists(table_name: str, constraint_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
inspector.clear_cache()
constraints = inspector.get_unique_constraints(table_name)
return any(c.get("name") == constraint_name for c in constraints)
def generate_short_id(length: int = 12) -> str:
"""Generate a Gemini-style short ID (lowercase letters + digits)"""
alphabet = string.ascii_lowercase + string.digits
return "".join(secrets.choice(alphabet) for _ in range(length))
def upgrade() -> None:
bind = op.get_bind()
dialect = bind.dialect.name
# =========================================================================
# 1. usage 表: billing_status + finalized_at + 索引
# =========================================================================
if table_exists("usage"):
if not column_exists("usage", "billing_status"):
op.add_column(
"usage",
sa.Column(
"billing_status",
sa.String(20),
nullable=False,
server_default="settled",
),
)
if not column_exists("usage", "finalized_at"):
op.add_column(
"usage",
sa.Column("finalized_at", sa.DateTime(timezone=True), nullable=True),
)
if not index_exists("usage", "idx_usage_billing_status"):
op.create_index("idx_usage_billing_status", "usage", ["billing_status"])
# (provider_name, created_at) — provider list / dashboard queries
if (
column_exists("usage", "provider_name")
and column_exists("usage", "created_at")
and not index_exists("usage", "idx_usage_provider_created")
):
op.create_index("idx_usage_provider_created", "usage", ["provider_name", "created_at"])
# (model, created_at) — model analytics / recent requests queries
if (
column_exists("usage", "model")
and column_exists("usage", "created_at")
and not index_exists("usage", "idx_usage_model_created")
):
op.create_index("idx_usage_model_created", "usage", ["model", "created_at"])
# =========================================================================
# 2. video_tasks 表: request_id + short_id
# =========================================================================
if table_exists("video_tasks"):
# --- request_id ---
if not column_exists("video_tasks", "request_id"):
op.add_column(
"video_tasks",
sa.Column("request_id", sa.String(100), nullable=True),
)
# 回填 request_id
if dialect == "postgresql":
op.execute("""
UPDATE video_tasks
SET request_id = COALESCE(request_metadata->>'request_id', id)
WHERE request_id IS NULL
""")
elif dialect == "sqlite":
op.execute("""
UPDATE video_tasks
SET request_id = COALESCE(json_extract(request_metadata, '$.request_id'), id)
WHERE request_id IS NULL
""")
else:
op.execute("""
UPDATE video_tasks
SET request_id = id
WHERE request_id IS NULL
""")
if dialect == "postgresql":
op.alter_column("video_tasks", "request_id", nullable=False)
if not index_exists("video_tasks", "idx_video_tasks_request_id"):
op.create_index("idx_video_tasks_request_id", "video_tasks", ["request_id"])
if not unique_constraint_exists("video_tasks", "uq_video_tasks_request_id"):
op.create_unique_constraint(
"uq_video_tasks_request_id",
"video_tasks",
["request_id"],
)
# --- short_id ---
if not column_exists("video_tasks", "short_id"):
op.add_column(
"video_tasks",
sa.Column("short_id", sa.String(16), nullable=True),
)
# Populate existing rows with unique short_ids
result = bind.execute(text("SELECT id FROM video_tasks WHERE short_id IS NULL"))
for row in result:
short_id = generate_short_id()
bind.execute(
text("UPDATE video_tasks SET short_id = :short_id WHERE id = :id"),
{"short_id": short_id, "id": row[0]},
)
op.alter_column("video_tasks", "short_id", nullable=False)
op.create_index("ix_video_tasks_short_id", "video_tasks", ["short_id"], unique=True)
# =========================================================================
# 3. gemini_file_mappings 表
# =========================================================================
if not table_exists("gemini_file_mappings"):
op.create_table(
"gemini_file_mappings",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("file_name", sa.String(255), nullable=False, unique=True),
sa.Column(
"key_id",
sa.String(36),
sa.ForeignKey("provider_api_keys.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column(
"user_id",
sa.String(36),
sa.ForeignKey("users.id", ondelete="CASCADE"),
nullable=True,
),
sa.Column("display_name", sa.String(255), nullable=True),
sa.Column("mime_type", sa.String(100), nullable=True),
sa.Column("source_hash", sa.String(64), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False),
)
op.create_index("ix_gemini_file_mappings_id", "gemini_file_mappings", ["id"])
op.create_index(
"ix_gemini_file_mappings_file_name", "gemini_file_mappings", ["file_name"], unique=True
)
op.create_index("ix_gemini_file_mappings_key_id", "gemini_file_mappings", ["key_id"])
op.create_index("ix_gemini_file_mappings_user_id", "gemini_file_mappings", ["user_id"])
op.create_index("idx_gemini_file_mappings_expires", "gemini_file_mappings", ["expires_at"])
op.create_index(
"idx_gemini_file_mappings_source_hash", "gemini_file_mappings", ["source_hash"]
)
else:
# 表已存在,只添加 source_hash
if not column_exists("gemini_file_mappings", "source_hash"):
op.add_column(
"gemini_file_mappings",
sa.Column("source_hash", sa.String(64), nullable=True),
)
op.create_index(
"idx_gemini_file_mappings_source_hash",
"gemini_file_mappings",
["source_hash"],
)
# =========================================================================
# 4. providers 表: enable_format_conversion
# =========================================================================
if table_exists("providers") and not column_exists("providers", "enable_format_conversion"):
op.add_column(
"providers",
sa.Column(
"enable_format_conversion",
sa.Boolean(),
nullable=False,
server_default=sa.text("false"),
),
)
# =========================================================================
# 5. request_candidates 表: created_at 索引
# =========================================================================
if table_exists("request_candidates"):
if not index_exists("request_candidates", "idx_request_candidates_created_at"):
op.create_index(
"idx_request_candidates_created_at",
"request_candidates",
["created_at"],
unique=False,
)
def downgrade() -> None:
bind = op.get_bind()
dialect = bind.dialect.name
# =========================================================================
# 5. request_candidates 表回滚
# =========================================================================
if table_exists("request_candidates"):
if index_exists("request_candidates", "idx_request_candidates_created_at"):
op.drop_index("idx_request_candidates_created_at", table_name="request_candidates")
# =========================================================================
# 4. providers 表回滚
# =========================================================================
if table_exists("providers") and column_exists("providers", "enable_format_conversion"):
op.drop_column("providers", "enable_format_conversion")
# =========================================================================
# 3. gemini_file_mappings 表回滚
# =========================================================================
if table_exists("gemini_file_mappings"):
op.drop_index("idx_gemini_file_mappings_source_hash", table_name="gemini_file_mappings")
op.drop_index("idx_gemini_file_mappings_expires", table_name="gemini_file_mappings")
op.drop_index("ix_gemini_file_mappings_user_id", table_name="gemini_file_mappings")
op.drop_index("ix_gemini_file_mappings_key_id", table_name="gemini_file_mappings")
op.drop_index("ix_gemini_file_mappings_file_name", table_name="gemini_file_mappings")
op.drop_index("ix_gemini_file_mappings_id", table_name="gemini_file_mappings")
op.drop_table("gemini_file_mappings")
# =========================================================================
# 2. video_tasks 表回滚
# =========================================================================
if table_exists("video_tasks"):
# short_id
if column_exists("video_tasks", "short_id"):
if index_exists("video_tasks", "ix_video_tasks_short_id"):
op.drop_index("ix_video_tasks_short_id", table_name="video_tasks")
op.drop_column("video_tasks", "short_id")
# request_id
if column_exists("video_tasks", "request_id"):
if dialect == "postgresql":
if unique_constraint_exists("video_tasks", "uq_video_tasks_request_id"):
op.drop_constraint("uq_video_tasks_request_id", "video_tasks", type_="unique")
if index_exists("video_tasks", "idx_video_tasks_request_id"):
op.drop_index("idx_video_tasks_request_id", table_name="video_tasks")
op.drop_column("video_tasks", "request_id")
# =========================================================================
# 1. usage 表回滚
# =========================================================================
if table_exists("usage"):
if index_exists("usage", "idx_usage_model_created"):
op.drop_index("idx_usage_model_created", table_name="usage")
if index_exists("usage", "idx_usage_provider_created"):
op.drop_index("idx_usage_provider_created", table_name="usage")
if index_exists("usage", "idx_usage_billing_status"):
op.drop_index("idx_usage_billing_status", table_name="usage")
if column_exists("usage", "finalized_at"):
op.drop_column("usage", "finalized_at")
if column_exists("usage", "billing_status"):
op.drop_column("usage", "billing_status")
@@ -1,60 +0,0 @@
"""Add video_duration_seconds to video_tasks and body_rules to provider_endpoints
Revision ID: b3c4d5e6f7a8
Revises: a2f1b3c4d5e6
Create Date: 2026-02-03 15:00:00.000000
"""
from __future__ import annotations
from typing import Sequence, Union
import sqlalchemy as sa
from sqlalchemy import inspect
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "b3c4d5e6f7a8"
down_revision: Union[str, None] = "a2f1b3c4d5e6"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def _column_exists(table_name: str, column_name: str) -> bool:
"""Check if a column exists in a table."""
bind = op.get_bind()
inspector = inspect(bind)
columns = [col["name"] for col in inspector.get_columns(table_name)]
return column_name in columns
def upgrade() -> None:
# 1. Add video_duration_seconds to video_tasks
if not _column_exists("video_tasks", "video_duration_seconds"):
op.add_column(
"video_tasks",
sa.Column("video_duration_seconds", sa.Float(), nullable=True),
)
# 2. Add body_rules to provider_endpoints
# 请求体规则支持三种操作:
# - set: 设置/覆盖字段 {"action": "set", "path": "metadata", "value": {"custom": "val"}}
# - drop: 删除字段 {"action": "drop", "path": "unwanted_field"}
# - rename: 重命名字段 {"action": "rename", "from": "old_key", "to": "new_key"}
if not _column_exists("provider_endpoints", "body_rules"):
op.add_column(
"provider_endpoints",
sa.Column("body_rules", sa.JSON(), nullable=True),
)
def downgrade() -> None:
# Remove body_rules from provider_endpoints
if _column_exists("provider_endpoints", "body_rules"):
op.drop_column("provider_endpoints", "body_rules")
# Remove video_duration_seconds from video_tasks
if _column_exists("video_tasks", "video_duration_seconds"):
op.drop_column("video_tasks", "video_duration_seconds")
@@ -1,347 +0,0 @@
"""add_stats_hourly_and_daily_complete_flag
Revision ID: c4e8f9a1b2c3
Revises: b3c4d5e6f7a8
Create Date: 2026-02-04 12:00:00.000000
"""
from __future__ import annotations
from typing import Sequence, Union
import sqlalchemy as sa
from sqlalchemy import inspect
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "c4e8f9a1b2c3"
down_revision: Union[str, None] = "b3c4d5e6f7a8"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def _table_exists(table_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
return table_name in inspector.get_table_names()
def _index_exists(table_name: str, index_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
indexes = [idx["name"] for idx in inspector.get_indexes(table_name)]
return index_name in indexes
def _column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
# Use information_schema for more reliable detection (inspector can have caching issues)
result = bind.execute(
sa.text(
"SELECT EXISTS ("
"SELECT 1 FROM information_schema.columns "
"WHERE table_name = :table AND column_name = :column"
")"
),
{"table": table_name, "column": column_name},
)
return bool(result.scalar())
def upgrade() -> None:
if _table_exists("stats_daily"):
if not _column_exists("stats_daily", "is_complete"):
op.add_column(
"stats_daily",
sa.Column("is_complete", sa.Boolean(), nullable=False, server_default=sa.false()),
)
op.execute("UPDATE stats_daily SET is_complete = true")
if not _column_exists("stats_daily", "aggregated_at"):
op.add_column(
"stats_daily",
sa.Column("aggregated_at", sa.DateTime(timezone=True), nullable=True),
)
if not _table_exists("stats_hourly"):
op.create_table(
"stats_hourly",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("hour_utc", sa.DateTime(timezone=True), nullable=False),
sa.Column("total_requests", sa.Integer(), nullable=False),
sa.Column("success_requests", sa.Integer(), nullable=False),
sa.Column("error_requests", sa.Integer(), nullable=False),
sa.Column("input_tokens", sa.BigInteger(), nullable=False),
sa.Column("output_tokens", sa.BigInteger(), nullable=False),
sa.Column("cache_creation_tokens", sa.BigInteger(), nullable=False),
sa.Column("cache_read_tokens", sa.BigInteger(), nullable=False),
sa.Column("total_cost", sa.Float(), nullable=False),
sa.Column("actual_total_cost", sa.Float(), nullable=False),
sa.Column("avg_response_time_ms", sa.Float(), nullable=False),
sa.Column("is_complete", sa.Boolean(), nullable=False),
sa.Column("aggregated_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("hour_utc", name="uq_stats_hourly_hour"),
)
op.create_index("idx_stats_hourly_hour", "stats_hourly", ["hour_utc"], unique=False)
if not _table_exists("stats_hourly_user"):
op.create_table(
"stats_hourly_user",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("hour_utc", sa.DateTime(timezone=True), nullable=False),
sa.Column("user_id", sa.String(length=36), nullable=False),
sa.Column("total_requests", sa.Integer(), nullable=False),
sa.Column("success_requests", sa.Integer(), nullable=False),
sa.Column("error_requests", sa.Integer(), nullable=False),
sa.Column("input_tokens", sa.BigInteger(), nullable=False),
sa.Column("output_tokens", sa.BigInteger(), nullable=False),
sa.Column("total_cost", sa.Float(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("hour_utc", "user_id", name="uq_stats_hourly_user"),
)
op.create_index(
"idx_stats_hourly_user_hour", "stats_hourly_user", ["hour_utc"], unique=False
)
op.create_index(
"idx_stats_hourly_user_user_hour",
"stats_hourly_user",
["user_id", "hour_utc"],
unique=False,
)
if not _table_exists("stats_hourly_model"):
op.create_table(
"stats_hourly_model",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("hour_utc", sa.DateTime(timezone=True), nullable=False),
sa.Column("model", sa.String(length=100), nullable=False),
sa.Column("total_requests", sa.Integer(), nullable=False),
sa.Column("input_tokens", sa.BigInteger(), nullable=False),
sa.Column("output_tokens", sa.BigInteger(), nullable=False),
sa.Column("total_cost", sa.Float(), nullable=False),
sa.Column("avg_response_time_ms", sa.Float(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("hour_utc", "model", name="uq_stats_hourly_model"),
)
op.create_index(
"idx_stats_hourly_model_hour", "stats_hourly_model", ["hour_utc"], unique=False
)
op.create_index(
"idx_stats_hourly_model_model_hour",
"stats_hourly_model",
["model", "hour_utc"],
unique=False,
)
if not _table_exists("stats_hourly_provider"):
op.create_table(
"stats_hourly_provider",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("hour_utc", sa.DateTime(timezone=True), nullable=False),
sa.Column("provider_name", sa.String(length=100), nullable=False),
sa.Column("total_requests", sa.Integer(), nullable=False),
sa.Column("input_tokens", sa.BigInteger(), nullable=False),
sa.Column("output_tokens", sa.BigInteger(), nullable=False),
sa.Column("total_cost", sa.Float(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("hour_utc", "provider_name", name="uq_stats_hourly_provider"),
)
op.create_index(
"idx_stats_hourly_provider_hour",
"stats_hourly_provider",
["hour_utc"],
unique=False,
)
if not _table_exists("stats_daily_api_key"):
op.create_table(
"stats_daily_api_key",
sa.Column("id", sa.String(length=36), primary_key=True),
sa.Column(
"api_key_id",
sa.String(length=36),
sa.ForeignKey("api_keys.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column("date", sa.DateTime(timezone=True), nullable=False),
sa.Column("total_requests", sa.Integer(), nullable=False, server_default="0"),
sa.Column("success_requests", sa.Integer(), nullable=False, server_default="0"),
sa.Column("error_requests", sa.Integer(), nullable=False, server_default="0"),
sa.Column("input_tokens", sa.BigInteger(), nullable=False, server_default="0"),
sa.Column("output_tokens", sa.BigInteger(), nullable=False, server_default="0"),
sa.Column("cache_creation_tokens", sa.BigInteger(), nullable=False, server_default="0"),
sa.Column("cache_read_tokens", sa.BigInteger(), nullable=False, server_default="0"),
sa.Column("total_cost", sa.Float(), nullable=False, server_default="0"),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
nullable=False,
server_default=sa.text("CURRENT_TIMESTAMP"),
),
sa.Column(
"updated_at",
sa.DateTime(timezone=True),
nullable=False,
server_default=sa.text("CURRENT_TIMESTAMP"),
),
sa.UniqueConstraint("api_key_id", "date", name="uq_stats_daily_api_key"),
)
if _table_exists("stats_daily_api_key"):
if not _index_exists("stats_daily_api_key", "idx_stats_daily_api_key_date"):
op.create_index("idx_stats_daily_api_key_date", "stats_daily_api_key", ["date"])
if not _index_exists("stats_daily_api_key", "idx_stats_daily_api_key_key_date"):
op.create_index(
"idx_stats_daily_api_key_key_date",
"stats_daily_api_key",
["api_key_id", "date"],
)
if not _index_exists("stats_daily_api_key", "idx_stats_daily_api_key_date_requests"):
op.create_index(
"idx_stats_daily_api_key_date_requests",
"stats_daily_api_key",
["date", "total_requests"],
)
if not _index_exists("stats_daily_api_key", "idx_stats_daily_api_key_date_cost"):
op.create_index(
"idx_stats_daily_api_key_date_cost",
"stats_daily_api_key",
["date", "total_cost"],
)
if _table_exists("usage"):
if not _column_exists("usage", "error_category"):
op.add_column(
"usage",
sa.Column("error_category", sa.String(length=50), nullable=True),
)
op.create_index("idx_usage_error_category", "usage", ["error_category"], unique=False)
if _table_exists("stats_daily"):
for name in (
"p50_response_time_ms",
"p90_response_time_ms",
"p99_response_time_ms",
"p50_first_byte_time_ms",
"p90_first_byte_time_ms",
"p99_first_byte_time_ms",
):
if not _column_exists("stats_daily", name):
op.add_column("stats_daily", sa.Column(name, sa.Integer(), nullable=True))
if not _table_exists("stats_daily_error"):
op.create_table(
"stats_daily_error",
sa.Column("id", sa.String(length=36), primary_key=True),
sa.Column("date", sa.DateTime(timezone=True), nullable=False),
sa.Column("error_category", sa.String(length=50), nullable=False),
sa.Column("provider_name", sa.String(length=100), nullable=True),
sa.Column("model", sa.String(length=100), nullable=True),
sa.Column("count", sa.Integer(), nullable=False, server_default="0"),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
nullable=False,
server_default=sa.text("CURRENT_TIMESTAMP"),
),
sa.Column(
"updated_at",
sa.DateTime(timezone=True),
nullable=False,
server_default=sa.text("CURRENT_TIMESTAMP"),
),
sa.UniqueConstraint(
"date",
"error_category",
"provider_name",
"model",
name="uq_stats_daily_error",
),
)
if _table_exists("stats_daily_error"):
if not _index_exists("stats_daily_error", "idx_stats_daily_error_date"):
op.create_index("idx_stats_daily_error_date", "stats_daily_error", ["date"])
if not _index_exists("stats_daily_error", "idx_stats_daily_error_category"):
op.create_index(
"idx_stats_daily_error_category",
"stats_daily_error",
["date", "error_category"],
)
def downgrade() -> None:
if _table_exists("stats_daily_error"):
if _index_exists("stats_daily_error", "idx_stats_daily_error_category"):
op.drop_index("idx_stats_daily_error_category", table_name="stats_daily_error")
if _index_exists("stats_daily_error", "idx_stats_daily_error_date"):
op.drop_index("idx_stats_daily_error_date", table_name="stats_daily_error")
op.drop_table("stats_daily_error")
if _table_exists("stats_daily"):
for name in (
"p50_response_time_ms",
"p90_response_time_ms",
"p99_response_time_ms",
"p50_first_byte_time_ms",
"p90_first_byte_time_ms",
"p99_first_byte_time_ms",
):
if _column_exists("stats_daily", name):
op.drop_column("stats_daily", name)
if _table_exists("usage") and _column_exists("usage", "error_category"):
if _index_exists("usage", "idx_usage_error_category"):
op.drop_index("idx_usage_error_category", table_name="usage")
op.drop_column("usage", "error_category")
if _table_exists("stats_daily_api_key"):
if _index_exists("stats_daily_api_key", "idx_stats_daily_api_key_date_cost"):
op.drop_index("idx_stats_daily_api_key_date_cost", table_name="stats_daily_api_key")
if _index_exists("stats_daily_api_key", "idx_stats_daily_api_key_date_requests"):
op.drop_index("idx_stats_daily_api_key_date_requests", table_name="stats_daily_api_key")
if _index_exists("stats_daily_api_key", "idx_stats_daily_api_key_key_date"):
op.drop_index("idx_stats_daily_api_key_key_date", table_name="stats_daily_api_key")
if _index_exists("stats_daily_api_key", "idx_stats_daily_api_key_date"):
op.drop_index("idx_stats_daily_api_key_date", table_name="stats_daily_api_key")
op.drop_table("stats_daily_api_key")
if _table_exists("stats_hourly_provider"):
if _index_exists("stats_hourly_provider", "idx_stats_hourly_provider_hour"):
op.drop_index("idx_stats_hourly_provider_hour", table_name="stats_hourly_provider")
op.drop_table("stats_hourly_provider")
if _table_exists("stats_hourly_model"):
if _index_exists("stats_hourly_model", "idx_stats_hourly_model_model_hour"):
op.drop_index("idx_stats_hourly_model_model_hour", table_name="stats_hourly_model")
if _index_exists("stats_hourly_model", "idx_stats_hourly_model_hour"):
op.drop_index("idx_stats_hourly_model_hour", table_name="stats_hourly_model")
op.drop_table("stats_hourly_model")
if _table_exists("stats_hourly_user"):
if _index_exists("stats_hourly_user", "idx_stats_hourly_user_user_hour"):
op.drop_index("idx_stats_hourly_user_user_hour", table_name="stats_hourly_user")
if _index_exists("stats_hourly_user", "idx_stats_hourly_user_hour"):
op.drop_index("idx_stats_hourly_user_hour", table_name="stats_hourly_user")
op.drop_table("stats_hourly_user")
if _table_exists("stats_hourly"):
if _index_exists("stats_hourly", "idx_stats_hourly_hour"):
op.drop_index("idx_stats_hourly_hour", table_name="stats_hourly")
op.drop_table("stats_hourly")
if _table_exists("stats_daily"):
if _column_exists("stats_daily", "aggregated_at"):
op.drop_column("stats_daily", "aggregated_at")
if _column_exists("stats_daily", "is_complete"):
op.drop_column("stats_daily", "is_complete")
@@ -1,205 +0,0 @@
"""Add provider_type, upstream_metadata, oauth_invalid fields and expand string columns to TEXT
- Add providers.provider_type (String(20), server_default="custom")
- Add provider_api_keys.upstream_metadata (JSON, nullable)
- Add provider_api_keys.oauth_invalid_at (DateTime, nullable) - OAuth Token 失效时间
- Add provider_api_keys.oauth_invalid_reason (String(255), nullable) - OAuth Token 失效原因
- Expand multiple VARCHAR columns to TEXT for long values (OAuth tokens, LDAP DN, URLs, etc.)
Revision ID: b5c6d7e8f9a0
Revises: c4e8f9a1b2c3
Create Date: 2026-02-04 15:00:00.000000
"""
from typing import Sequence, Union
import sqlalchemy as sa
from sqlalchemy import inspect
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "b5c6d7e8f9a0"
down_revision: Union[str, None] = "c4e8f9a1b2c3"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
# 需要扩展为 TEXT 的列(表名, 列名, 原始类型长度)
COLUMNS_TO_EXPAND = [
("provider_api_keys", "api_key", 500), # OAuth tokens can be very long
(
"provider_api_keys",
"auth_config",
None,
), # 确保 auth_config 是 TEXT 类型(可能从 JSON 迁移过来)
("ldap_configs", "bind_dn", 255), # LDAP DN can be deeply nested
("ldap_configs", "base_dn", 255), # LDAP DN can be deeply nested
("ldap_configs", "user_search_filter", 500), # Complex LDAP filters
("oauth_providers", "client_id", 255), # Some OAuth providers use JWT client_id
]
def column_exists(table_name: str, column_name: str) -> bool:
"""检查列是否已存在"""
bind = op.get_bind()
inspector = inspect(bind)
columns = [col["name"] for col in inspector.get_columns(table_name)]
return column_name in columns
def table_exists(table_name: str) -> bool:
"""检查表是否存在"""
bind = op.get_bind()
inspector = inspect(bind)
return table_name in inspector.get_table_names()
def is_sqlite() -> bool:
"""检查是否为 SQLite 数据库"""
bind = op.get_bind()
return bind.dialect.name == "sqlite"
def get_column_type(table_name: str, column_name: str) -> str | None:
"""获取列的数据类型"""
bind = op.get_bind()
inspector = inspect(bind)
for col in inspector.get_columns(table_name):
if col["name"] == column_name:
return str(col["type"]).upper()
return None
def expand_column_to_text(table_name: str, column_name: str, original_length: int | None) -> None:
"""将 VARCHAR 列扩展为 TEXT(兼容 SQLite)"""
if not table_exists(table_name):
return
if not column_exists(table_name, column_name):
return
# 检查当前列类型,如果已经是 TEXT 则跳过
col_type = get_column_type(table_name, column_name)
if col_type and "TEXT" in col_type:
return
# 如果是 JSON 类型(可能是历史遗留),先将 JSON 数据转为文本表示再变更类型
is_json_col = col_type and "JSON" in col_type
if is_json_col and not is_sqlite():
# PostgreSQL: 先用 CAST 把 JSON 值转为 TEXT,保留数据
op.execute(
sa.text(
f"ALTER TABLE {table_name} ALTER COLUMN {column_name} "
f"TYPE TEXT USING {column_name}::TEXT"
)
)
return
if is_sqlite():
# SQLite 不支持直接 ALTER COLUMN,需要用 batch 模式
# batch 模式会自动处理 JSON->TEXT 的数据迁移
with op.batch_alter_table(table_name) as batch_op:
batch_op.alter_column(
column_name,
type_=sa.Text(),
existing_type=sa.String(original_length) if original_length else sa.Text(),
)
else:
op.alter_column(
table_name,
column_name,
type_=sa.Text(),
existing_type=sa.String(original_length) if original_length else sa.Text(),
existing_nullable=True,
)
def shrink_column_to_varchar(
table_name: str, column_name: str, target_length: int, nullable: bool = False
) -> None:
"""将 TEXT 列缩小为 VARCHAR(兼容 SQLite)
WARNING: 如果数据超过 target_length 会失败
"""
if not table_exists(table_name):
return
if not column_exists(table_name, column_name):
return
if is_sqlite():
with op.batch_alter_table(table_name) as batch_op:
batch_op.alter_column(
column_name,
type_=sa.String(target_length),
existing_type=sa.Text(),
existing_nullable=nullable,
)
else:
op.alter_column(
table_name,
column_name,
type_=sa.String(target_length),
existing_type=sa.Text(),
existing_nullable=nullable,
)
def upgrade() -> None:
# Add providers.provider_type
if not column_exists("providers", "provider_type"):
op.add_column(
"providers",
sa.Column("provider_type", sa.String(20), nullable=False, server_default="custom"),
)
# Add provider_api_keys.upstream_metadata
if not column_exists("provider_api_keys", "upstream_metadata"):
op.add_column(
"provider_api_keys",
sa.Column("upstream_metadata", sa.JSON(), nullable=True),
)
# Add provider_api_keys.oauth_invalid_at
if not column_exists("provider_api_keys", "oauth_invalid_at"):
op.add_column(
"provider_api_keys",
sa.Column("oauth_invalid_at", sa.DateTime(timezone=True), nullable=True),
)
# Add provider_api_keys.oauth_invalid_reason
if not column_exists("provider_api_keys", "oauth_invalid_reason"):
op.add_column(
"provider_api_keys",
sa.Column("oauth_invalid_reason", sa.String(255), nullable=True),
)
# Expand VARCHAR columns to TEXT
for table_name, column_name, original_length in COLUMNS_TO_EXPAND:
expand_column_to_text(table_name, column_name, original_length)
def downgrade() -> None:
# Shrink TEXT columns back to VARCHAR
# WARNING: Downgrade may fail if any values exceed original length
for table_name, column_name, original_length in reversed(COLUMNS_TO_EXPAND):
# 跳过没有原始长度的列(如 auth_config,由其他迁移创建)
if original_length is None:
continue
shrink_column_to_varchar(table_name, column_name, original_length)
# Drop provider_api_keys.oauth_invalid_reason
if column_exists("provider_api_keys", "oauth_invalid_reason"):
op.drop_column("provider_api_keys", "oauth_invalid_reason")
# Drop provider_api_keys.oauth_invalid_at
if column_exists("provider_api_keys", "oauth_invalid_at"):
op.drop_column("provider_api_keys", "oauth_invalid_at")
# Drop provider_api_keys.upstream_metadata
if column_exists("provider_api_keys", "upstream_metadata"):
op.drop_column("provider_api_keys", "upstream_metadata")
# Drop providers.provider_type
if column_exists("providers", "provider_type"):
op.drop_column("providers", "provider_type")
@@ -1,254 +0,0 @@
"""Antigravity endpoint signature to gemini:chat & add proxy_nodes table (with manual fields)
Revision ID: e1b2c3d4f5a6
Revises: b5c6d7e8f9a0
Create Date: 2026-02-06 23:45:00.000000
"""
from __future__ import annotations
from collections.abc import Sequence
import sqlalchemy as sa
from sqlalchemy import inspect, text
from sqlalchemy.dialects import postgresql
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "e1b2c3d4f5a6"
down_revision: str | None = "b5c6d7e8f9a0"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def table_exists(table_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
return table_name in inspector.get_table_names()
def upgrade() -> None:
conn = op.get_bind()
# =========================================================================
# Part 1: Antigravity endpoint signature migration (gemini:cli -> gemini:chat)
# =========================================================================
# --- provider_endpoints ---
# Update only when there is no conflicting gemini:chat endpoint for the same provider
# (provider_endpoints has a unique constraint on (provider_id, api_format)).
conn.execute(text("""
UPDATE provider_endpoints pe
SET
api_format = 'gemini:chat',
api_family = 'gemini',
endpoint_kind = 'chat'
WHERE pe.api_format = 'gemini:cli'
AND pe.provider_id IN (
SELECT id FROM providers WHERE lower(provider_type) = 'antigravity'
)
AND NOT EXISTS (
SELECT 1 FROM provider_endpoints pe2
WHERE pe2.provider_id = pe.provider_id
AND pe2.api_format = 'gemini:chat'
)
"""))
# Best-effort normalization for already-existing Antigravity gemini:chat endpoints.
conn.execute(text("""
UPDATE provider_endpoints pe
SET
api_family = 'gemini',
endpoint_kind = 'chat'
WHERE pe.api_format = 'gemini:chat'
AND pe.provider_id IN (
SELECT id FROM providers WHERE lower(provider_type) = 'antigravity'
)
"""))
# --- provider_api_keys.api_formats (JSON array) ---
# Replace "gemini:cli" with "gemini:chat" in the JSON array for Antigravity keys.
# Uses text-level replace on the serialized JSON -- safe because the value is a
# simple string with no special characters that could cause ambiguous replacements.
conn.execute(text("""
UPDATE provider_api_keys pak
SET api_formats = replace(pak.api_formats::text, '"gemini:cli"', '"gemini:chat"')::json
WHERE pak.provider_id IN (
SELECT id FROM providers WHERE lower(provider_type) = 'antigravity'
)
AND pak.api_formats IS NOT NULL
AND pak.api_formats::text LIKE '%"gemini:cli"%'
"""))
# =========================================================================
# Part 2: Create proxy_nodes table with manual proxy fields (idempotent)
# =========================================================================
# Create ENUM type (idempotent)
op.execute(
"DO $$ BEGIN "
"CREATE TYPE proxynodestatus AS ENUM ('online', 'unhealthy', 'offline'); "
"EXCEPTION WHEN duplicate_object THEN NULL; "
"END $$"
)
if table_exists("proxy_nodes"):
# Table already exists — ensure manual proxy columns are present
inspector = inspect(conn)
existing_columns = {c["name"] for c in inspector.get_columns("proxy_nodes")}
# ip 列扩容:手动节点的 ip 存储 "socks5://hostname" 形式,45 字符可能不够
ip_col = next((c for c in inspector.get_columns("proxy_nodes") if c["name"] == "ip"), None)
if ip_col and hasattr(ip_col["type"], "length") and (ip_col["type"].length or 0) < 512:
op.alter_column("proxy_nodes", "ip", type_=sa.String(512), existing_nullable=False)
manual_columns = [
("is_manual", sa.Boolean(), False, sa.text("false"), "是否为手动添加的代理节点"),
("proxy_url", sa.String(500), True, None, "手动节点的完整代理 URL"),
("proxy_username", sa.String(255), True, None, "手动节点的代理用户名"),
("proxy_password", sa.String(500), True, None, "手动节点的代理密码"),
]
for col_name, col_type, nullable, default, comment in manual_columns:
if col_name not in existing_columns:
op.add_column(
"proxy_nodes",
sa.Column(
col_name,
col_type, # type: ignore[arg-type]
nullable=nullable,
server_default=default,
comment=comment,
),
)
return
op.create_table(
"proxy_nodes",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("name", sa.String(100), nullable=False),
sa.Column("ip", sa.String(512), nullable=False),
sa.Column("port", sa.Integer(), nullable=False),
sa.Column("region", sa.String(100), nullable=True),
sa.Column(
"status",
postgresql.ENUM(
"online",
"unhealthy",
"offline",
name="proxynodestatus",
create_type=False,
),
nullable=False,
server_default=sa.text("'online'"),
),
sa.Column(
"registered_by",
sa.String(36),
sa.ForeignKey("users.id", ondelete="SET NULL"),
nullable=True,
),
sa.Column("last_heartbeat_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("heartbeat_interval", sa.Integer(), nullable=False, server_default=sa.text("30")),
sa.Column("active_connections", sa.Integer(), nullable=False, server_default=sa.text("0")),
sa.Column("total_requests", sa.BigInteger(), nullable=False, server_default=sa.text("0")),
sa.Column("avg_latency_ms", sa.Float(), nullable=True),
# --- Manual proxy node fields ---
sa.Column(
"is_manual",
sa.Boolean(),
nullable=False,
server_default=sa.text("false"),
comment="是否为手动添加的代理节点",
),
sa.Column(
"proxy_url",
sa.String(500),
nullable=True,
comment="手动节点的完整代理 URL",
),
sa.Column(
"proxy_username",
sa.String(255),
nullable=True,
comment="手动节点的代理用户名",
),
sa.Column(
"proxy_password",
sa.String(500),
nullable=True,
comment="手动节点的代理密码",
),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("CURRENT_TIMESTAMP"),
nullable=False,
),
sa.Column(
"updated_at",
sa.DateTime(timezone=True),
server_default=sa.text("CURRENT_TIMESTAMP"),
nullable=False,
),
sa.UniqueConstraint("ip", "port", name="uq_proxy_node_ip_port"),
)
def downgrade() -> None:
conn = op.get_bind()
# =========================================================================
# Part 2 rollback: Drop proxy_nodes table (and manual columns if present)
# =========================================================================
if table_exists("proxy_nodes"):
op.drop_table("proxy_nodes")
# Best-effort: drop type (only used by proxy_nodes)
op.execute("DROP TYPE IF EXISTS proxynodestatus")
# =========================================================================
# Part 1 rollback: Revert Antigravity endpoint signature (gemini:chat -> gemini:cli)
# =========================================================================
# --- provider_endpoints ---
conn.execute(text("""
UPDATE provider_endpoints pe
SET
api_format = 'gemini:cli',
api_family = 'gemini',
endpoint_kind = 'cli'
WHERE pe.api_format = 'gemini:chat'
AND pe.provider_id IN (
SELECT id FROM providers WHERE lower(provider_type) = 'antigravity'
)
AND NOT EXISTS (
SELECT 1 FROM provider_endpoints pe2
WHERE pe2.provider_id = pe.provider_id
AND pe2.api_format = 'gemini:cli'
)
"""))
# Best-effort normalization for already-existing Antigravity gemini:cli endpoints.
conn.execute(text("""
UPDATE provider_endpoints pe
SET
api_family = 'gemini',
endpoint_kind = 'cli'
WHERE pe.api_format = 'gemini:cli'
AND pe.provider_id IN (
SELECT id FROM providers WHERE lower(provider_type) = 'antigravity'
)
"""))
# --- provider_api_keys.api_formats (JSON array) ---
conn.execute(text("""
UPDATE provider_api_keys pak
SET api_formats = replace(pak.api_formats::text, '"gemini:chat"', '"gemini:cli"')::json
WHERE pak.provider_id IN (
SELECT id FROM providers WHERE lower(provider_type) = 'antigravity'
)
AND pak.api_formats IS NOT NULL
AND pak.api_formats::text LIKE '%"gemini:chat"%'
"""))
@@ -1,61 +0,0 @@
"""Add remote_config and config_version to proxy_nodes
Revision ID: 3aff3ffc4a0e
Revises: e1b2c3d4f5a6
Create Date: 2026-02-07 15:00:00.000000
"""
from __future__ import annotations
from collections.abc import Sequence
import sqlalchemy as sa
from sqlalchemy import inspect
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "3aff3ffc4a0e"
down_revision: str | None = "e1b2c3d4f5a6"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
columns = [c["name"] for c in inspector.get_columns(table_name)]
return column_name in columns
def upgrade() -> None:
if not column_exists("proxy_nodes", "remote_config"):
op.add_column(
"proxy_nodes",
sa.Column(
"remote_config",
sa.JSON(),
nullable=True,
comment="管理端下发的远程配置 (allowed_ports, log_level, heartbeat_interval, timestamp_tolerance)",
),
)
if not column_exists("proxy_nodes", "config_version"):
op.add_column(
"proxy_nodes",
sa.Column(
"config_version",
sa.Integer(),
nullable=False,
server_default="0",
comment="远程配置版本号,每次更新 +1",
),
)
def downgrade() -> None:
if column_exists("proxy_nodes", "config_version"):
op.drop_column("proxy_nodes", "config_version")
if column_exists("proxy_nodes", "remote_config"):
op.drop_column("proxy_nodes", "remote_config")
@@ -1,61 +0,0 @@
"""Add tls_enabled and tls_cert_fingerprint to proxy_nodes
Revision ID: 4b5c6d7e8f9a
Revises: 3aff3ffc4a0e
Create Date: 2026-02-07 18:00:00.000000
"""
from __future__ import annotations
from collections.abc import Sequence
import sqlalchemy as sa
from sqlalchemy import inspect
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "4b5c6d7e8f9a"
down_revision: str | None = "3aff3ffc4a0e"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
columns = [c["name"] for c in inspector.get_columns(table_name)]
return column_name in columns
def upgrade() -> None:
if not column_exists("proxy_nodes", "tls_enabled"):
op.add_column(
"proxy_nodes",
sa.Column(
"tls_enabled",
sa.Boolean(),
nullable=False,
server_default="false",
comment="是否启用 TLS 加密",
),
)
if not column_exists("proxy_nodes", "tls_cert_fingerprint"):
op.add_column(
"proxy_nodes",
sa.Column(
"tls_cert_fingerprint",
sa.String(128),
nullable=True,
comment="TLS 证书 SHA-256 指纹(hex)",
),
)
def downgrade() -> None:
if column_exists("proxy_nodes", "tls_cert_fingerprint"):
op.drop_column("proxy_nodes", "tls_cert_fingerprint")
if column_exists("proxy_nodes", "tls_enabled"):
op.drop_column("proxy_nodes", "tls_enabled")
@@ -1,60 +0,0 @@
"""Add hardware_info and estimated_max_concurrency to proxy_nodes
Revision ID: 5c6d7e8f9a0b
Revises: 4b5c6d7e8f9a
Create Date: 2026-02-08 12:00:00.000000
"""
from __future__ import annotations
from collections.abc import Sequence
import sqlalchemy as sa
from sqlalchemy import inspect
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "5c6d7e8f9a0b"
down_revision: str | None = "4b5c6d7e8f9a"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
columns = [c["name"] for c in inspector.get_columns(table_name)]
return column_name in columns
def upgrade() -> None:
if not column_exists("proxy_nodes", "hardware_info"):
op.add_column(
"proxy_nodes",
sa.Column(
"hardware_info",
sa.JSON(),
nullable=True,
comment="硬件信息 (cpu_cores, total_memory_mb, os_info, fd_limit, ...)",
),
)
if not column_exists("proxy_nodes", "estimated_max_concurrency"):
op.add_column(
"proxy_nodes",
sa.Column(
"estimated_max_concurrency",
sa.Integer(),
nullable=True,
comment="基于硬件估算的最大并发连接数",
),
)
def downgrade() -> None:
if column_exists("proxy_nodes", "estimated_max_concurrency"):
op.drop_column("proxy_nodes", "estimated_max_concurrency")
if column_exists("proxy_nodes", "hardware_info"):
op.drop_column("proxy_nodes", "hardware_info")
@@ -1,47 +0,0 @@
"""Add proxy column to provider_api_keys for per-key proxy configuration
Revision ID: 6d7e8f9a0b1c
Revises: 5c6d7e8f9a0b
Create Date: 2026-02-08 15:00:00.000000
"""
from __future__ import annotations
from collections.abc import Sequence
import sqlalchemy as sa
from sqlalchemy import inspect
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "6d7e8f9a0b1c"
down_revision: str | None = "5c6d7e8f9a0b"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
columns = [c["name"] for c in inspector.get_columns(table_name)]
return column_name in columns
def upgrade() -> None:
if not column_exists("provider_api_keys", "proxy"):
op.add_column(
"provider_api_keys",
sa.Column(
"proxy",
sa.JSON(),
nullable=True,
comment="Key 级别代理配置(覆盖 Provider 级别代理),如 {node_id, enabled}",
),
)
def downgrade() -> None:
if column_exists("provider_api_keys", "proxy"):
op.drop_column("provider_api_keys", "proxy")
@@ -1,46 +0,0 @@
"""Add provider_request_body and client_response_body columns to usage table
Revision ID: 7e8f9a0b1c2d
Revises: 6d7e8f9a0b1c
Create Date: 2026-02-20 18:00:00.000000
"""
from __future__ import annotations
from collections.abc import Sequence
from sqlalchemy import text
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "7e8f9a0b1c2d"
down_revision: str | None = "6d7e8f9a0b1c"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
conn = op.get_bind()
# Use PostgreSQL native IF NOT EXISTS to avoid duplicate-column races
# when migrations are triggered concurrently (e.g. startup + manual run).
conn.execute(text("ALTER TABLE usage ADD COLUMN IF NOT EXISTS provider_request_body JSON"))
conn.execute(
text("ALTER TABLE usage ADD COLUMN IF NOT EXISTS provider_request_body_compressed BYTEA")
)
conn.execute(text("ALTER TABLE usage ADD COLUMN IF NOT EXISTS client_response_body JSON"))
conn.execute(
text("ALTER TABLE usage ADD COLUMN IF NOT EXISTS client_response_body_compressed BYTEA")
)
def downgrade() -> None:
conn = op.get_bind()
for col in (
"client_response_body_compressed",
"client_response_body",
"provider_request_body_compressed",
"provider_request_body",
):
conn.execute(text(f"ALTER TABLE usage DROP COLUMN IF EXISTS {col}"))
@@ -1,91 +0,0 @@
"""Add api_family and endpoint_kind columns to usage table
Revision ID: 8f9a0b1c2d3e
Revises: 7e8f9a0b1c2d
Create Date: 2026-02-21 15:00:00.000000
"""
from __future__ import annotations
from collections.abc import Sequence
import sqlalchemy as sa
from sqlalchemy import inspect, text
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "8f9a0b1c2d3e"
down_revision: str | None = "7e8f9a0b1c2d"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
# Usage 表新增列
NEW_COLUMNS = [
("api_family", sa.String(50)),
("endpoint_kind", sa.String(50)),
("provider_api_family", sa.String(50)),
("provider_endpoint_kind", sa.String(50)),
]
# 新增索引
NEW_INDEXES = [
("idx_usage_api_family", "usage", ["api_family"]),
("idx_usage_endpoint_kind", "usage", ["endpoint_kind"]),
("idx_usage_family_kind", "usage", ["api_family", "endpoint_kind"]),
]
def upgrade() -> None:
conn = op.get_bind()
# 使用 PostgreSQL 原生 IF NOT EXISTS,比 inspect 更可靠(避免同一事务内缓存问题)
col_definitions = {
"api_family": "VARCHAR(50)",
"endpoint_kind": "VARCHAR(50)",
"provider_api_family": "VARCHAR(50)",
"provider_endpoint_kind": "VARCHAR(50)",
}
for col_name, col_type_sql in col_definitions.items():
conn.execute(text(f"ALTER TABLE usage ADD COLUMN IF NOT EXISTS {col_name} {col_type_sql}"))
# 数据迁移:从 api_format 解析 api_family + endpoint_kind
conn.execute(text("""
UPDATE usage SET
api_family = lower(split_part(api_format, ':', 1)),
endpoint_kind = lower(split_part(api_format, ':', 2))
WHERE api_format IS NOT NULL
AND api_format LIKE '%%:%%'
AND api_family IS NULL
"""))
conn.execute(text("""
UPDATE usage SET
provider_api_family = lower(split_part(endpoint_api_format, ':', 1)),
provider_endpoint_kind = lower(split_part(endpoint_api_format, ':', 2))
WHERE endpoint_api_format IS NOT NULL
AND endpoint_api_format LIKE '%%:%%'
AND provider_api_family IS NULL
"""))
# 创建索引
inspector = inspect(conn)
existing_indexes = {idx["name"] for idx in inspector.get_indexes("usage")}
for idx_name, table, columns in NEW_INDEXES:
if idx_name not in existing_indexes:
op.create_index(idx_name, table, columns)
def downgrade() -> None:
conn = op.get_bind()
inspector = inspect(conn)
existing_indexes = {idx["name"] for idx in inspector.get_indexes("usage")}
for idx_name, _, _ in reversed(NEW_INDEXES):
if idx_name in existing_indexes:
op.drop_index(idx_name, table_name="usage")
existing_columns = {col["name"] for col in inspector.get_columns("usage")}
for col_name, _ in reversed(NEW_COLUMNS):
if col_name in existing_columns:
op.drop_column("usage", col_name)
@@ -1,106 +0,0 @@
"""Add tunnel mode fields and remove IP forwarding fields
Revision ID: 9a0b1c2d3e4f
Revises: 8f9a0b1c2d3e
Create Date: 2026-02-24 17:00:00.000000
"""
from __future__ import annotations
from collections.abc import Sequence
import sqlalchemy as sa
from sqlalchemy import inspect
from alembic import op
revision: str = "9a0b1c2d3e4f"
down_revision: str | None = "8f9a0b1c2d3e"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
columns = [c["name"] for c in inspector.get_columns(table_name)]
return column_name in columns
def upgrade() -> None:
# 添加 tunnel 模式字段
if not column_exists("proxy_nodes", "tunnel_mode"):
op.add_column(
"proxy_nodes",
sa.Column(
"tunnel_mode",
sa.Boolean(),
nullable=False,
server_default=sa.text("false"),
comment="是否使用 WebSocket 隧道模式",
),
)
if not column_exists("proxy_nodes", "tunnel_connected"):
op.add_column(
"proxy_nodes",
sa.Column(
"tunnel_connected",
sa.Boolean(),
nullable=False,
server_default=sa.text("false"),
comment="隧道是否已连接",
),
)
if not column_exists("proxy_nodes", "tunnel_connected_at"):
op.add_column(
"proxy_nodes",
sa.Column(
"tunnel_connected_at",
sa.DateTime(timezone=True),
nullable=True,
comment="隧道最近一次建立时间",
),
)
# tunnel 模式节点不需要 port,将其置零
op.execute("UPDATE proxy_nodes SET port = 0 WHERE tunnel_mode = true")
# 移除旧的 IP 转发字段
if column_exists("proxy_nodes", "tls_enabled"):
op.drop_column("proxy_nodes", "tls_enabled")
if column_exists("proxy_nodes", "tls_cert_fingerprint"):
op.drop_column("proxy_nodes", "tls_cert_fingerprint")
def downgrade() -> None:
# 恢复 IP 转发字段
if not column_exists("proxy_nodes", "tls_cert_fingerprint"):
op.add_column(
"proxy_nodes",
sa.Column(
"tls_cert_fingerprint",
sa.String(128),
nullable=True,
comment="TLS 证书 SHA-256 指纹(hex)",
),
)
if not column_exists("proxy_nodes", "tls_enabled"):
op.add_column(
"proxy_nodes",
sa.Column(
"tls_enabled",
sa.Boolean(),
nullable=False,
server_default=sa.text("false"),
comment="是否启用 TLS 加密",
),
)
# 移除 tunnel 模式字段
if column_exists("proxy_nodes", "tunnel_connected_at"):
op.drop_column("proxy_nodes", "tunnel_connected_at")
if column_exists("proxy_nodes", "tunnel_connected"):
op.drop_column("proxy_nodes", "tunnel_connected")
if column_exists("proxy_nodes", "tunnel_mode"):
op.drop_column("proxy_nodes", "tunnel_mode")
@@ -1,224 +0,0 @@
"""Add cache_creation columns, clean up capability settings, add user_model_usage_counts,
enforce global_model_id NOT NULL
1. Add cache_creation_input_tokens_5m and cache_creation_input_tokens_1h to usage table.
2. Clean up cache_1h/context_1m/gemini_files from user-configurable settings
(now auto-detected via REQUEST_PARAM mode).
3. Create user_model_usage_counts table for per-user per-model atomic usage counters.
4. Enforce models.global_model_id NOT NULL (delete orphan models without global model).
Revision ID: b2c3d4e5f6a7
Revises: 9a0b1c2d3e4f
Create Date: 2026-02-28 14:00:00.000000
"""
from __future__ import annotations
import json
import uuid
from collections.abc import Sequence
from datetime import datetime, timezone
import sqlalchemy as sa
from sqlalchemy import inspect
from alembic import op
revision: str = "b2c3d4e5f6a7"
down_revision: str | None = "9a0b1c2d3e4f"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
insp = inspect(bind)
columns = [c["name"] for c in insp.get_columns(table_name)]
return column_name in columns
def table_exists(table_name: str) -> bool:
bind = op.get_bind()
insp = inspect(bind)
return table_name in insp.get_table_names()
def index_exists(table_name: str, index_name: str) -> bool:
bind = op.get_bind()
insp = inspect(bind)
return any(idx["name"] == index_name for idx in insp.get_indexes(table_name))
def upgrade() -> None:
# --- 1. Add cache_creation columns ---
if not column_exists("usage", "cache_creation_input_tokens_5m"):
op.add_column(
"usage",
sa.Column(
"cache_creation_input_tokens_5m",
sa.Integer(),
nullable=False,
server_default=sa.text("0"),
comment="5min TTL cache creation input tokens",
),
)
if not column_exists("usage", "cache_creation_input_tokens_1h"):
op.add_column(
"usage",
sa.Column(
"cache_creation_input_tokens_1h",
sa.Integer(),
nullable=False,
server_default=sa.text("0"),
comment="1h TTL cache creation input tokens",
),
)
# --- 2. Clean up stale capability settings (pure Python, DB-agnostic) ---
stale_keys = {"cache_1h", "context_1m", "gemini_files"}
conn = op.get_bind()
# ApiKey.force_capabilities: dict-like JSON, remove stale keys
rows = conn.execute(
sa.text("SELECT id, force_capabilities FROM api_keys WHERE force_capabilities IS NOT NULL")
).fetchall()
for row in rows:
raw = row[1]
if raw is None:
continue
data = raw if isinstance(raw, dict) else json.loads(raw)
cleaned = {k: v for k, v in data.items() if k not in stale_keys}
new_val = json.dumps(cleaned) if cleaned else None
conn.execute(
sa.text("UPDATE api_keys SET force_capabilities = :val WHERE id = :id"),
{"val": new_val, "id": row[0]},
)
# User.model_capability_settings: nested dict {model_key: {cap: val}}, remove stale keys
rows = conn.execute(
sa.text(
"SELECT id, model_capability_settings FROM users"
" WHERE model_capability_settings IS NOT NULL"
)
).fetchall()
for row in rows:
raw = row[1]
if raw is None:
continue
data = raw if isinstance(raw, dict) else json.loads(raw)
cleaned = {}
for model_key, caps in data.items():
cap_cleaned = {k: v for k, v in caps.items() if k not in stale_keys}
if cap_cleaned:
cleaned[model_key] = cap_cleaned
new_val = json.dumps(cleaned) if cleaned else None
conn.execute(
sa.text("UPDATE users SET model_capability_settings = :val WHERE id = :id"),
{"val": new_val, "id": row[0]},
)
# GlobalModel.supported_capabilities: JSON array, remove stale entries
rows = conn.execute(
sa.text(
"SELECT id, supported_capabilities FROM global_models"
" WHERE supported_capabilities IS NOT NULL"
)
).fetchall()
for row in rows:
raw = row[1]
if raw is None:
continue
data = raw if isinstance(raw, list) else json.loads(raw)
cleaned = [c for c in data if c not in stale_keys]
new_val = json.dumps(cleaned) if cleaned else None
conn.execute(
sa.text("UPDATE global_models SET supported_capabilities = :val WHERE id = :id"),
{"val": new_val, "id": row[0]},
)
# --- 3. Create user_model_usage_counts table ---
if not table_exists("user_model_usage_counts"):
op.create_table(
"user_model_usage_counts",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column(
"user_id",
sa.String(36),
sa.ForeignKey("users.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column("model", sa.String(100), nullable=False),
sa.Column("usage_count", sa.Integer, nullable=False, server_default="0"),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
nullable=False,
server_default=sa.func.now(),
),
sa.Column(
"updated_at",
sa.DateTime(timezone=True),
nullable=False,
server_default=sa.func.now(),
),
sa.UniqueConstraint("user_id", "model", name="uq_user_model_usage_count"),
)
if not index_exists("user_model_usage_counts", "idx_user_model_usage_user"):
op.create_index("idx_user_model_usage_user", "user_model_usage_counts", ["user_id"])
if not index_exists("user_model_usage_counts", "idx_user_model_usage_model"):
op.create_index("idx_user_model_usage_model", "user_model_usage_counts", ["model"])
# Backfill from existing usage records (truncate first for idempotency)
conn.execute(sa.text("DELETE FROM user_model_usage_counts"))
rows = conn.execute(
sa.text(
"SELECT user_id, model, COUNT(*) AS cnt FROM usage"
" WHERE user_id IS NOT NULL GROUP BY user_id, model"
)
).fetchall()
now = datetime.now(timezone.utc)
for row in rows:
conn.execute(
sa.text(
"INSERT INTO user_model_usage_counts"
" (id, user_id, model, usage_count, created_at, updated_at)"
" VALUES (:id, :user_id, :model, :cnt, :now, :now)"
),
{
"id": str(uuid.uuid4()),
"user_id": row[0],
"model": row[1],
"cnt": row[2],
"now": now,
},
)
# --- 4. Enforce models.global_model_id NOT NULL ---
conn = op.get_bind()
insp = inspect(conn)
model_cols = {c["name"]: c for c in insp.get_columns("models")}
if model_cols.get("global_model_id", {}).get("nullable", True):
op.execute("DELETE FROM models WHERE global_model_id IS NULL")
op.alter_column("models", "global_model_id", existing_type=sa.String(36), nullable=False)
def downgrade() -> None:
# Revert models.global_model_id to nullable
if column_exists("models", "global_model_id"):
op.alter_column("models", "global_model_id", existing_type=sa.String(36), nullable=True)
# Drop user_model_usage_counts
if table_exists("user_model_usage_counts"):
if index_exists("user_model_usage_counts", "idx_user_model_usage_model"):
op.drop_index("idx_user_model_usage_model", table_name="user_model_usage_counts")
if index_exists("user_model_usage_counts", "idx_user_model_usage_user"):
op.drop_index("idx_user_model_usage_user", table_name="user_model_usage_counts")
op.drop_table("user_model_usage_counts")
# Drop cache_creation columns
if column_exists("usage", "cache_creation_input_tokens_1h"):
op.drop_column("usage", "cache_creation_input_tokens_1h")
if column_exists("usage", "cache_creation_input_tokens_5m"):
op.drop_column("usage", "cache_creation_input_tokens_5m")
# capability settings cleanup is not reversible
@@ -1,153 +0,0 @@
"""proxy_node_metrics_and_events
Revision ID: 48afe197cc15
Revises: b2c3d4e5f6a7
Create Date: 2026-02-28 04:33:11.201185+00:00
"""
import sqlalchemy as sa
from sqlalchemy import inspect
from alembic import op
# revision identifiers, used by Alembic.
revision = "48afe197cc15"
down_revision = "b2c3d4e5f6a7"
branch_labels = None
depends_on = None
def _column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
insp = inspect(bind)
columns = [c["name"] for c in insp.get_columns(table_name)]
return column_name in columns
def _table_exists(table_name: str) -> bool:
bind = op.get_bind()
insp = inspect(bind)
return table_name in insp.get_table_names()
def _enum_has_value(enum_name: str, value: str) -> bool:
"""检查 PostgreSQL 枚举类型是否包含指定值"""
bind = op.get_bind()
result = bind.execute(
sa.text(
"SELECT 1 FROM pg_enum e JOIN pg_type t ON e.enumtypid = t.oid"
" WHERE t.typname = :enum_name AND e.enumlabel = :value"
),
{"enum_name": enum_name, "value": value},
)
return result.fetchone() is not None
def upgrade() -> None:
# proxy_nodes: 将已废弃的 unhealthy 状态迁移为 offline,然后从枚举中移除
if _enum_has_value("proxynodestatus", "unhealthy"):
op.execute("UPDATE proxy_nodes SET status = 'offline' WHERE status = 'unhealthy'")
op.execute("ALTER TYPE proxynodestatus RENAME TO proxynodestatus_old")
op.execute("CREATE TYPE proxynodestatus AS ENUM ('online', 'offline')")
# 必须先移除旧枚举类型的 DEFAULT,否则 ALTER TYPE 会因无法转换默认值而报错
op.execute("ALTER TABLE proxy_nodes ALTER COLUMN status DROP DEFAULT")
op.execute(
"ALTER TABLE proxy_nodes ALTER COLUMN status TYPE proxynodestatus"
" USING status::text::proxynodestatus"
)
op.execute("ALTER TABLE proxy_nodes ALTER COLUMN status SET DEFAULT 'online'::proxynodestatus")
op.execute("DROP TYPE proxynodestatus_old")
# proxy_nodes: 新增错误指标字段
if not _column_exists("proxy_nodes", "failed_requests"):
op.add_column(
"proxy_nodes",
sa.Column(
"failed_requests",
sa.BigInteger(),
nullable=False,
server_default="0",
comment="累计失败请求数",
),
)
if not _column_exists("proxy_nodes", "dns_failures"):
op.add_column(
"proxy_nodes",
sa.Column(
"dns_failures",
sa.BigInteger(),
nullable=False,
server_default="0",
comment="累计 DNS 失败数",
),
)
if not _column_exists("proxy_nodes", "stream_errors"):
op.add_column(
"proxy_nodes",
sa.Column(
"stream_errors",
sa.BigInteger(),
nullable=False,
server_default="0",
comment="累计流错误数",
),
)
# proxy_node_events: 连接事件表
if not _table_exists("proxy_node_events"):
op.create_table(
"proxy_node_events",
sa.Column("id", sa.BigInteger(), autoincrement=True, nullable=False),
sa.Column("node_id", sa.String(length=36), nullable=False),
sa.Column(
"event_type",
sa.String(length=20),
nullable=False,
comment="事件类型: connected, disconnected, error",
),
sa.Column(
"detail",
sa.String(length=500),
nullable=True,
comment="事件详情(如断开原因)",
),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["node_id"], ["proxy_nodes.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
)
op.create_index(
"idx_proxy_node_events_node_created",
"proxy_node_events",
["node_id", "created_at"],
)
op.create_index(
op.f("ix_proxy_node_events_node_id"),
"proxy_node_events",
["node_id"],
)
def downgrade() -> None:
# 恢复 proxynodestatus 枚举,加回 unhealthy
if not _enum_has_value("proxynodestatus", "unhealthy"):
op.execute("ALTER TYPE proxynodestatus RENAME TO proxynodestatus_old")
op.execute("CREATE TYPE proxynodestatus AS ENUM ('online', 'unhealthy', 'offline')")
op.execute("ALTER TABLE proxy_nodes ALTER COLUMN status DROP DEFAULT")
op.execute(
"ALTER TABLE proxy_nodes ALTER COLUMN status TYPE proxynodestatus"
" USING status::text::proxynodestatus"
)
op.execute("ALTER TABLE proxy_nodes ALTER COLUMN status SET DEFAULT 'online'::proxynodestatus")
op.execute("DROP TYPE proxynodestatus_old")
if _table_exists("proxy_node_events"):
op.drop_index(op.f("ix_proxy_node_events_node_id"), table_name="proxy_node_events")
op.drop_index("idx_proxy_node_events_node_created", table_name="proxy_node_events")
op.drop_table("proxy_node_events")
if _column_exists("proxy_nodes", "stream_errors"):
op.drop_column("proxy_nodes", "stream_errors")
if _column_exists("proxy_nodes", "dns_failures"):
op.drop_column("proxy_nodes", "dns_failures")
if _column_exists("proxy_nodes", "failed_requests"):
op.drop_column("proxy_nodes", "failed_requests")
@@ -1,49 +0,0 @@
"""add_request_candidates_composite_indexes
Revision ID: 00b9161b8729
Revises: 48afe197cc15
Create Date: 2026-02-28 14:48:00.000000+00:00
"""
from sqlalchemy import inspect
from alembic import op
# revision identifiers, used by Alembic.
revision = "00b9161b8729"
down_revision = "48afe197cc15"
branch_labels = None
depends_on = None
def _index_exists(index_name: str) -> bool:
bind = op.get_bind()
insp = inspect(bind)
indexes = insp.get_indexes("request_candidates")
return any(idx["name"] == index_name for idx in indexes)
def upgrade() -> None:
# (request_id, status) - fallback/retry 查询优化
if not _index_exists("idx_rc_request_id_status"):
op.create_index(
"idx_rc_request_id_status",
"request_candidates",
["request_id", "status"],
)
# (provider_id, status, created_at) - provider 聚合统计优化
if not _index_exists("idx_rc_provider_status_created"):
op.create_index(
"idx_rc_provider_status_created",
"request_candidates",
["provider_id", "status", "created_at"],
)
def downgrade() -> None:
if _index_exists("idx_rc_provider_status_created"):
op.drop_index("idx_rc_provider_status_created", table_name="request_candidates")
if _index_exists("idx_rc_request_id_status"):
op.drop_index("idx_rc_request_id_status", table_name="request_candidates")
@@ -1,263 +0,0 @@
"""vertex_ai_provider_type
Migrate legacy Vertex auth_type/provider_type into the new model:
- provider_type=vertex_ai
- auth_type=service_account (legacy vertex_ai renamed)
- fixed Vertex endpoints: gemini:chat + claude:chat
Revision ID: 2a624af8dd3a
Revises: 00b9161b8729
Create Date: 2026-02-28 15:00:00.000000+00:00
"""
from __future__ import annotations
import uuid
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = "2a624af8dd3a"
down_revision = "00b9161b8729"
branch_labels = None
depends_on = None
_VERTEX_BASE_URL = "https://aiplatform.googleapis.com"
_VERTEX_ENDPOINTS: tuple[tuple[str, str, str], ...] = (
("gemini:chat", "gemini", "chat"),
("claude:chat", "claude", "chat"),
)
_VERTEX_KEY_FORMATS_SA = '["gemini:chat","claude:chat"]'
_VERTEX_KEY_FORMATS_API_KEY = '["gemini:chat"]'
def _select_vertex_provider_ids(conn: sa.Connection) -> list[str]:
"""Collect providers that should be treated as Vertex after migration."""
rows = conn.execute(sa.text("""
SELECT DISTINCT p.id
FROM providers p
LEFT JOIN provider_api_keys pak ON pak.provider_id = p.id
WHERE lower(COALESCE(p.provider_type, '')) = 'vertex_ai'
OR pak.auth_type = 'vertex_ai'
"""))
return [str(row[0]) for row in rows if row[0]]
def _ensure_fixed_vertex_endpoints(conn: sa.Connection, provider_ids: list[str]) -> None:
"""Ensure every Vertex provider has fixed gemini:chat + claude:chat endpoints."""
for provider_id in provider_ids:
provider_max_retries = (
conn.execute(
sa.text("""
SELECT COALESCE(max_retries, 2)
FROM providers
WHERE id = :provider_id
"""),
{"provider_id": provider_id},
).scalar()
or 2
)
for api_format, api_family, endpoint_kind in _VERTEX_ENDPOINTS:
# Normalize existing fixed endpoint fields.
conn.execute(
sa.text("""
UPDATE provider_endpoints
SET
api_family = :api_family,
endpoint_kind = :endpoint_kind,
base_url = :base_url,
custom_path = NULL,
is_active = TRUE,
updated_at = CURRENT_TIMESTAMP
WHERE provider_id = :provider_id
AND api_format = :api_format
"""),
{
"provider_id": provider_id,
"api_format": api_format,
"api_family": api_family,
"endpoint_kind": endpoint_kind,
"base_url": _VERTEX_BASE_URL,
},
)
exists = conn.execute(
sa.text("""
SELECT 1
FROM provider_endpoints
WHERE provider_id = :provider_id
AND api_format = :api_format
LIMIT 1
"""),
{"provider_id": provider_id, "api_format": api_format},
).first()
if not exists:
conn.execute(
sa.text("""
INSERT INTO provider_endpoints (
id,
provider_id,
api_format,
api_family,
endpoint_kind,
base_url,
custom_path,
header_rules,
body_rules,
max_retries,
is_active,
config,
format_acceptance_config,
proxy,
created_at,
updated_at
)
VALUES (
:id,
:provider_id,
:api_format,
:api_family,
:endpoint_kind,
:base_url,
NULL,
NULL,
NULL,
:max_retries,
TRUE,
NULL,
NULL,
NULL,
CURRENT_TIMESTAMP,
CURRENT_TIMESTAMP
)
"""),
{
"id": str(uuid.uuid4()),
"provider_id": provider_id,
"api_format": api_format,
"api_family": api_family,
"endpoint_kind": endpoint_kind,
"base_url": _VERTEX_BASE_URL,
"max_retries": int(provider_max_retries),
},
)
# Vertex fixed-provider model: disable non-fixed endpoints.
conn.execute(
sa.text("""
UPDATE provider_endpoints
SET
is_active = FALSE,
updated_at = CURRENT_TIMESTAMP
WHERE provider_id = :provider_id
AND api_format NOT IN ('gemini:chat', 'claude:chat')
"""),
{"provider_id": provider_id},
)
def _normalize_vertex_key_formats(conn: sa.Connection, provider_ids: list[str]) -> None:
"""Normalize key.api_formats for Vertex keys by auth type."""
for provider_id in provider_ids:
# Service Account (and legacy vertex_ai) keys: allow Gemini + Claude models.
conn.execute(
sa.text("""
UPDATE provider_api_keys
SET
api_formats = CAST(:api_formats AS json),
updated_at = CURRENT_TIMESTAMP
WHERE provider_id = :provider_id
AND auth_type IN ('service_account', 'vertex_ai')
"""),
{
"provider_id": provider_id,
"api_formats": _VERTEX_KEY_FORMATS_SA,
},
)
# API Key mode on Vertex 仅支持 Gemini(Google publisher)。
conn.execute(
sa.text("""
UPDATE provider_api_keys
SET
api_formats = CAST(:api_formats AS json),
updated_at = CURRENT_TIMESTAMP
WHERE provider_id = :provider_id
AND auth_type = 'api_key'
"""),
{
"provider_id": provider_id,
"api_formats": _VERTEX_KEY_FORMATS_API_KEY,
},
)
def upgrade() -> None:
conn = op.get_bind()
# 1) 收集目标 Provider(兼容重复执行,先识别 legacy/new 两种来源)。
provider_ids = _select_vertex_provider_ids(conn)
# 2) 先重命名 auth_type(legacy vertex_ai -> service_account)。
conn.execute(sa.text("""
UPDATE provider_api_keys
SET auth_type = 'service_account'
WHERE auth_type = 'vertex_ai'
"""))
if not provider_ids:
return
# 3) 归一 provider_type,并启用格式转换(Vertex 同时承载 Gemini/Claude)。
for provider_id in provider_ids:
conn.execute(
sa.text("""
UPDATE providers
SET
provider_type = 'vertex_ai',
enable_format_conversion = TRUE
WHERE id = :provider_id
"""),
{"provider_id": provider_id},
)
# 4) 固定端点落地:gemini:chat + claude:chat。
_ensure_fixed_vertex_endpoints(conn, provider_ids)
# 5) 归一 key 的 api_formats,避免调度命中旧格式。
_normalize_vertex_key_formats(conn, provider_ids)
def downgrade() -> None:
conn = op.get_bind()
provider_rows = conn.execute(sa.text("""
SELECT id
FROM providers
WHERE lower(COALESCE(provider_type, '')) = 'vertex_ai'
"""))
provider_ids = [str(row[0]) for row in provider_rows if row[0]]
if provider_ids:
for provider_id in provider_ids:
conn.execute(
sa.text("""
UPDATE provider_api_keys
SET auth_type = 'vertex_ai'
WHERE provider_id = :provider_id
AND auth_type = 'service_account'
"""),
{"provider_id": provider_id},
)
conn.execute(
sa.text("""
UPDATE providers
SET provider_type = 'custom'
WHERE id = :provider_id
"""),
{"provider_id": provider_id},
)
@@ -1,199 +0,0 @@
"""backfill_codex_compact_endpoint
Backfill Codex reverse-proxy endpoints:
- ensure `openai:cli` endpoint is pinned to force_stream
- ensure `openai:compact` endpoint exists
Revision ID: f0c3a7b9d1e2
Revises: 2a624af8dd3a
Create Date: 2026-03-01 17:00:00.000000+00:00
"""
from __future__ import annotations
import json
import uuid
from typing import Any
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = "f0c3a7b9d1e2"
down_revision = "2a624af8dd3a"
branch_labels = None
depends_on = None
_CODEX_BASE_URL = "https://chatgpt.com/backend-api/codex"
_COMPACT_FORMAT = "openai:compact"
_CLI_FORMAT = "openai:cli"
_FORCE_STREAM = "force_stream"
def _find_codex_provider_ids(conn: sa.Connection) -> list[str]:
"""Find Codex providers (by provider_type or legacy base_url pattern)."""
rows = conn.execute(sa.text("""
SELECT DISTINCT p.id
FROM providers p
LEFT JOIN provider_endpoints pe ON pe.provider_id = p.id
WHERE lower(COALESCE(p.provider_type, '')) = 'codex'
OR (
lower(COALESCE(pe.api_format, '')) = 'openai:cli'
AND lower(COALESCE(pe.base_url, '')) LIKE '%/backend-api/codex%'
)
"""))
return [str(r[0]) for r in rows if r[0]]
def _get_cli_endpoint(conn: sa.Connection, provider_id: str) -> dict[str, Any] | None:
"""Load existing openai:cli endpoint for the provider."""
row = (
conn.execute(
sa.text("""
SELECT base_url, header_rules, body_rules, max_retries, proxy, config
FROM provider_endpoints
WHERE provider_id = :pid AND api_format = :fmt
LIMIT 1
"""),
{"pid": provider_id, "fmt": _CLI_FORMAT},
)
.mappings()
.first()
)
return dict(row) if row else None
def _pin_cli_force_stream(conn: sa.Connection, provider_id: str, cli: dict[str, Any]) -> None:
"""Set upstream_stream_policy=force_stream on existing cli endpoint."""
cfg = dict(cli.get("config") or {}) if isinstance(cli.get("config"), dict) else {}
cfg.pop("upstreamStreamPolicy", None)
cfg.pop("upstream_stream", None)
cfg["upstream_stream_policy"] = _FORCE_STREAM
conn.execute(
sa.text("""
UPDATE provider_endpoints
SET api_family = 'openai',
endpoint_kind = 'cli',
config = CAST(:config AS json),
updated_at = CURRENT_TIMESTAMP
WHERE provider_id = :pid AND api_format = :fmt
"""),
{
"pid": provider_id,
"fmt": _CLI_FORMAT,
"config": json.dumps(cfg, ensure_ascii=False),
},
)
def _ensure_compact_endpoint(conn: sa.Connection, provider_id: str, cli: dict[str, Any]) -> None:
"""Create openai:compact endpoint if missing (clone from cli)."""
exists = conn.execute(
sa.text(
"SELECT 1 FROM provider_endpoints WHERE provider_id = :pid AND api_format = :fmt LIMIT 1"
),
{"pid": provider_id, "fmt": _COMPACT_FORMAT},
).first()
if exists:
# Already exists, just ensure api_family/endpoint_kind are set.
conn.execute(
sa.text("""
UPDATE provider_endpoints
SET api_family = 'openai', endpoint_kind = 'compact',
updated_at = CURRENT_TIMESTAMP
WHERE provider_id = :pid AND api_format = :fmt
"""),
{"pid": provider_id, "fmt": _COMPACT_FORMAT},
)
return
# Clone from cli endpoint, strip stream policy.
cfg = dict(cli.get("config") or {}) if isinstance(cli.get("config"), dict) else {}
for k in ("upstream_stream_policy", "upstreamStreamPolicy", "upstream_stream"):
cfg.pop(k, None)
def _json(val: Any) -> str | None:
return json.dumps(val, ensure_ascii=False) if val is not None else None
conn.execute(
sa.text("""
INSERT INTO provider_endpoints (
id, provider_id, api_format, api_family, endpoint_kind,
base_url, custom_path, header_rules, body_rules,
max_retries, is_active, config, format_acceptance_config,
proxy, created_at, updated_at
) VALUES (
:id, :pid, :fmt, 'openai', 'compact',
:base_url, NULL, CAST(:header_rules AS json), CAST(:body_rules AS json),
:max_retries, TRUE, CAST(:config AS json), NULL,
CAST(:proxy AS jsonb), CURRENT_TIMESTAMP, CURRENT_TIMESTAMP
)
"""),
{
"id": str(uuid.uuid4()),
"pid": provider_id,
"fmt": _COMPACT_FORMAT,
"base_url": cli.get("base_url") or _CODEX_BASE_URL,
"header_rules": _json(cli.get("header_rules")),
"body_rules": _json(cli.get("body_rules")),
"max_retries": cli.get("max_retries") or 2,
"config": _json(cfg or None),
"proxy": _json(cli.get("proxy")),
},
)
def _add_compact_to_key_formats(conn: sa.Connection, provider_id: str) -> None:
"""Ensure provider keys include openai:compact in api_formats."""
rows = (
conn.execute(
sa.text("SELECT id, api_formats FROM provider_api_keys WHERE provider_id = :pid"),
{"pid": provider_id},
)
.mappings()
.all()
)
for row in rows:
raw = row["api_formats"]
formats: list[str] = []
if isinstance(raw, list):
for item in raw:
v = str(item or "").strip().lower()
if v and v not in formats:
formats.append(v)
if _COMPACT_FORMAT in formats:
continue
# Insert compact right after cli, or at end.
if _CLI_FORMAT in formats:
idx = formats.index(_CLI_FORMAT) + 1
formats.insert(idx, _COMPACT_FORMAT)
else:
formats.append(_COMPACT_FORMAT)
conn.execute(
sa.text("""
UPDATE provider_api_keys
SET api_formats = CAST(:fmts AS json), updated_at = CURRENT_TIMESTAMP
WHERE id = :id
"""),
{"id": row["id"], "fmts": json.dumps(formats, ensure_ascii=False)},
)
def upgrade() -> None:
conn = op.get_bind()
for provider_id in _find_codex_provider_ids(conn):
cli = _get_cli_endpoint(conn, provider_id)
if not cli:
continue # No cli endpoint to clone from; skip.
_pin_cli_force_stream(conn, provider_id, cli)
_ensure_compact_endpoint(conn, provider_id, cli)
_add_compact_to_key_formats(conn, provider_id)
def downgrade() -> None:
# Data backfill: no-op to avoid deleting user-managed data.
return
@@ -1,29 +0,0 @@
"""add_proxy_metadata_to_proxy_nodes
Revision ID: 1d2e3f4a5b6c
Revises: f0c3a7b9d1e2
Create Date: 2026-03-02 13:00:00.000000+00:00
"""
from __future__ import annotations
from collections.abc import Sequence
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "1d2e3f4a5b6c"
down_revision: str | None = "f0c3a7b9d1e2"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
op.execute("ALTER TABLE public.proxy_nodes ADD COLUMN IF NOT EXISTS proxy_metadata json")
op.execute(
"COMMENT ON COLUMN public.proxy_nodes.proxy_metadata IS 'aether-proxy 上报元数据(版本等)'"
)
def downgrade() -> None:
op.execute("ALTER TABLE public.proxy_nodes DROP COLUMN IF EXISTS proxy_metadata")
@@ -1,72 +0,0 @@
"""backfill_codex_default_body_rules
Backfill default body_rules for codex providers with openai:cli endpoints
that currently have body_rules IS NULL.
Rules:
- drop max_output_tokens
- drop temperature
- drop top_p
- set store = false
- set instructions = "You are GPT-5." (when instructions not exists)
Revision ID: dd0278c0a28c
Revises: 1d2e3f4a5b6c
Create Date: 2026-03-02 15:00:00.000000+00:00
"""
from __future__ import annotations
import json
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = "dd0278c0a28c"
down_revision = "1d2e3f4a5b6c"
branch_labels = None
depends_on = None
_TARGET_FORMATS = ("openai:cli",)
_DEFAULT_BODY_RULES = [
{"action": "drop", "path": "max_output_tokens"},
{"action": "drop", "path": "temperature"},
{"action": "drop", "path": "top_p"},
{"action": "set", "path": "store", "value": False},
{
"action": "set",
"path": "instructions",
"value": "You are GPT-5.",
"condition": {"path": "instructions", "op": "not_exists"},
},
]
def upgrade() -> None:
conn = op.get_bind()
# 幂等性: 仅回填 codex 提供商中 body_rules 为空(SQL NULL 或 JSON null)的记录
rules_json = json.dumps(_DEFAULT_BODY_RULES, ensure_ascii=False)
result = conn.execute(
sa.text("""
UPDATE provider_endpoints pe
SET body_rules = CAST(:rules AS json),
updated_at = CURRENT_TIMESTAMP
FROM providers p
WHERE pe.provider_id = p.id
AND p.provider_type = :ptype
AND pe.api_format = :fmt
AND (pe.body_rules IS NULL OR pe.body_rules::text = 'null')
"""),
{"rules": rules_json, "ptype": "codex", "fmt": _TARGET_FORMATS[0]},
)
if result.rowcount:
print(f" backfilled body_rules for {result.rowcount} endpoint(s)")
def downgrade() -> None:
# Data backfill: no-op to avoid removing user-customized rules.
return
@@ -1,45 +0,0 @@
"""add_idx_usage_provider_key
Add composite index on usage(provider_id, provider_api_key_id) to support
the pool management page's per-key usage stats aggregation query.
Revision ID: 0ba031f328de
Revises: dd0278c0a28c
Create Date: 2026-03-03 10:00:00.000000+00:00
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
revision = "0ba031f328de"
down_revision = "dd0278c0a28c"
branch_labels = None
depends_on = None
INDEX_NAME = "idx_usage_provider_key"
TABLE = "usage"
COLUMNS = ["provider_id", "provider_api_key_id"]
def upgrade() -> None:
bind = op.get_bind()
result = bind.execute(
sa.text("SELECT 1 FROM pg_indexes WHERE indexname = :name"),
{"name": INDEX_NAME},
).fetchone()
if result:
return
op.create_index(INDEX_NAME, TABLE, COLUMNS)
def downgrade() -> None:
bind = op.get_bind()
result = bind.execute(
sa.text("SELECT 1 FROM pg_indexes WHERE indexname = :name"),
{"name": INDEX_NAME},
).fetchone()
if not result:
return
op.drop_index(INDEX_NAME, table_name=TABLE)
@@ -1,45 +0,0 @@
"""add_idx_usage_status_user_created
Add composite index on usage(status, user_id, created_at) to speed up
interval timeline and active usage analytics queries.
Revision ID: 5f1d2e3c4b5a
Revises: 0ba031f328de
Create Date: 2026-03-03 17:30:00.000000+00:00
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
revision = "5f1d2e3c4b5a"
down_revision = "0ba031f328de"
branch_labels = None
depends_on = None
INDEX_NAME = "idx_usage_status_user_created"
TABLE = "usage"
COLUMNS = ["status", "user_id", "created_at"]
def upgrade() -> None:
bind = op.get_bind()
result = bind.execute(
sa.text("SELECT 1 FROM pg_indexes WHERE indexname = :name"),
{"name": INDEX_NAME},
).fetchone()
if result:
return
op.create_index(INDEX_NAME, TABLE, COLUMNS)
def downgrade() -> None:
bind = op.get_bind()
result = bind.execute(
sa.text("SELECT 1 FROM pg_indexes WHERE indexname = :name"),
{"name": INDEX_NAME},
).fetchone()
if not result:
return
op.drop_index(INDEX_NAME, table_name=TABLE)
@@ -1,41 +0,0 @@
"""add fingerprint column to provider_api_keys
Revision ID: 6a9b8c7d5e4f
Revises: 5f1d2e3c4b5a
Create Date: 2026-03-04 23:50:00.000000
"""
from __future__ import annotations
from collections.abc import Sequence
import sqlalchemy as sa
from sqlalchemy import inspect
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "6a9b8c7d5e4f"
down_revision: str | None = "5f1d2e3c4b5a"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
columns = [c["name"] for c in inspector.get_columns(table_name)]
return column_name in columns
def upgrade() -> None:
if not column_exists("provider_api_keys", "fingerprint"):
op.add_column(
"provider_api_keys",
sa.Column("fingerprint", sa.JSON(), nullable=True),
)
def downgrade() -> None:
if column_exists("provider_api_keys", "fingerprint"):
op.drop_column("provider_api_keys", "fingerprint")
@@ -1,65 +0,0 @@
"""remove standalone api key locking
Revision ID: 7c91d2e4f8a1
Revises: 6f7a8b9c0d1e
Create Date: 2026-03-05 17:00:00.000000+00:00
"""
from __future__ import annotations
from collections.abc import Sequence
import sqlalchemy as sa
from sqlalchemy import inspect
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "7c91d2e4f8a1"
down_revision: str | None = "6f7a8b9c0d1e"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
_CONSTRAINT_NAME = "ck_api_keys_standalone_not_locked"
def _column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
insp = inspect(bind)
insp.clear_cache()
return column_name in [c["name"] for c in insp.get_columns(table_name)]
def _check_constraint_exists(table_name: str, constraint_name: str) -> bool:
bind = op.get_bind()
insp = inspect(bind)
insp.clear_cache()
return any(c.get("name") == constraint_name for c in insp.get_check_constraints(table_name))
def upgrade() -> None:
if not (
_column_exists("api_keys", "is_standalone")
and _column_exists("api_keys", "is_locked")
and _column_exists("api_keys", "is_active")
):
return
op.execute(sa.text("""
UPDATE api_keys
SET is_active = FALSE,
is_locked = FALSE
WHERE is_standalone IS TRUE AND is_locked IS TRUE
"""))
if not _check_constraint_exists("api_keys", _CONSTRAINT_NAME):
op.create_check_constraint(
_CONSTRAINT_NAME,
"api_keys",
"(NOT is_standalone) OR (NOT is_locked)",
)
def downgrade() -> None:
if _check_constraint_exists("api_keys", _CONSTRAINT_NAME):
op.drop_constraint(_CONSTRAINT_NAME, "api_keys", type_="check")
@@ -1,220 +0,0 @@
"""tighten wallet transaction snapshots and remove wallet version
Revision ID: 8e71f2a4c9b0
Revises: 7c91d2e4f8a1
Create Date: 2026-03-07 13:00:00.000000+00:00
"""
from __future__ import annotations
from collections.abc import Sequence
import sqlalchemy as sa
from sqlalchemy import inspect
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "8e71f2a4c9b0"
down_revision: str | None = "7c91d2e4f8a1"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
_WALLET_TX_BEFORE_CHECK = "ck_wallet_tx_balance_before_consistent"
_WALLET_TX_AFTER_CHECK = "ck_wallet_tx_balance_after_consistent"
_WALLET_LIMIT_MODE_INDEX = "idx_wallets_limit_mode"
def _table_exists(table_name: str) -> bool:
bind = op.get_bind()
insp = inspect(bind)
insp.clear_cache()
return table_name in insp.get_table_names()
def _column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
insp = inspect(bind)
insp.clear_cache()
return column_name in [c["name"] for c in insp.get_columns(table_name)]
def _index_exists(table_name: str, index_name: str) -> bool:
bind = op.get_bind()
insp = inspect(bind)
insp.clear_cache()
return any(index.get("name") == index_name for index in insp.get_indexes(table_name))
def _check_constraint_exists(table_name: str, constraint_name: str) -> bool:
bind = op.get_bind()
insp = inspect(bind)
insp.clear_cache()
return any(c.get("name") == constraint_name for c in insp.get_check_constraints(table_name))
def _tighten_wallet_transaction_snapshots() -> None:
if not _table_exists("wallet_transactions"):
return
required_columns = {
"balance_before",
"balance_after",
"recharge_balance_before",
"recharge_balance_after",
"gift_balance_before",
"gift_balance_after",
}
existing_columns = {
column["name"] for column in inspect(op.get_bind()).get_columns("wallet_transactions")
}
if not required_columns.issubset(existing_columns):
return
op.execute(
sa.text(
"""
UPDATE wallet_transactions
SET recharge_balance_before = balance_before
WHERE recharge_balance_before IS NULL
"""
)
)
op.execute(
sa.text(
"""
UPDATE wallet_transactions
SET recharge_balance_after = balance_after
WHERE recharge_balance_after IS NULL
"""
)
)
op.execute(
sa.text(
"""
UPDATE wallet_transactions
SET gift_balance_before = 0
WHERE gift_balance_before IS NULL
"""
)
)
op.execute(
sa.text(
"""
UPDATE wallet_transactions
SET gift_balance_after = 0
WHERE gift_balance_after IS NULL
"""
)
)
op.execute(
sa.text(
"""
UPDATE wallet_transactions
SET balance_before = recharge_balance_before + gift_balance_before,
balance_after = recharge_balance_after + gift_balance_after
"""
)
)
if not _check_constraint_exists("wallet_transactions", _WALLET_TX_BEFORE_CHECK):
op.create_check_constraint(
_WALLET_TX_BEFORE_CHECK,
"wallet_transactions",
"balance_before = recharge_balance_before + gift_balance_before",
)
if not _check_constraint_exists("wallet_transactions", _WALLET_TX_AFTER_CHECK):
op.create_check_constraint(
_WALLET_TX_AFTER_CHECK,
"wallet_transactions",
"balance_after = recharge_balance_after + gift_balance_after",
)
op.alter_column(
"wallet_transactions",
"recharge_balance_before",
existing_type=sa.Numeric(20, 8),
nullable=False,
)
op.alter_column(
"wallet_transactions",
"recharge_balance_after",
existing_type=sa.Numeric(20, 8),
nullable=False,
)
op.alter_column(
"wallet_transactions",
"gift_balance_before",
existing_type=sa.Numeric(20, 8),
nullable=False,
)
op.alter_column(
"wallet_transactions",
"gift_balance_after",
existing_type=sa.Numeric(20, 8),
nullable=False,
)
def _drop_wallet_cleanup_artifacts() -> None:
if not _table_exists("wallets"):
return
if _index_exists("wallets", _WALLET_LIMIT_MODE_INDEX):
op.drop_index(_WALLET_LIMIT_MODE_INDEX, table_name="wallets")
if _column_exists("wallets", "version"):
op.drop_column("wallets", "version")
def upgrade() -> None:
_tighten_wallet_transaction_snapshots()
_drop_wallet_cleanup_artifacts()
def downgrade() -> None:
if _table_exists("wallets"):
if not _column_exists("wallets", "version"):
op.add_column(
"wallets",
sa.Column("version", sa.Integer(), nullable=False, server_default="0"),
)
if not _index_exists("wallets", _WALLET_LIMIT_MODE_INDEX):
op.create_index(_WALLET_LIMIT_MODE_INDEX, "wallets", ["limit_mode"])
if not _table_exists("wallet_transactions"):
return
if _column_exists("wallet_transactions", "recharge_balance_before"):
op.alter_column(
"wallet_transactions",
"recharge_balance_before",
existing_type=sa.Numeric(20, 8),
nullable=True,
)
if _column_exists("wallet_transactions", "recharge_balance_after"):
op.alter_column(
"wallet_transactions",
"recharge_balance_after",
existing_type=sa.Numeric(20, 8),
nullable=True,
)
if _column_exists("wallet_transactions", "gift_balance_before"):
op.alter_column(
"wallet_transactions",
"gift_balance_before",
existing_type=sa.Numeric(20, 8),
nullable=True,
)
if _column_exists("wallet_transactions", "gift_balance_after"):
op.alter_column(
"wallet_transactions",
"gift_balance_after",
existing_type=sa.Numeric(20, 8),
nullable=True,
)
if _check_constraint_exists("wallet_transactions", _WALLET_TX_AFTER_CHECK):
op.drop_constraint(_WALLET_TX_AFTER_CHECK, "wallet_transactions", type_="check")
if _check_constraint_exists("wallet_transactions", _WALLET_TX_BEFORE_CHECK):
op.drop_constraint(_WALLET_TX_BEFORE_CHECK, "wallet_transactions", type_="check")
@@ -1,80 +0,0 @@
"""add missing foreign key indexes for cascade delete performance
Revision ID: 2d932114930d
Revises: 8e71f2a4c9b0
Create Date: 2026-03-07 16:28:48.633531+00:00
"""
from alembic import op
from sqlalchemy import inspect
# revision identifiers, used by Alembic.
revision = '2d932114930d'
down_revision = '8e71f2a4c9b0'
branch_labels = None
depends_on = None
def _index_exists(table_name: str, index_name: str) -> bool:
bind = op.get_bind()
insp = inspect(bind)
return any(idx["name"] == index_name for idx in insp.get_indexes(table_name))
def _create_index_if_not_exists(index_name: str, table_name: str, columns: list[str]) -> None:
if not _index_exists(table_name, index_name):
op.create_index(op.f(index_name), table_name, columns, unique=False)
def _drop_index_if_exists(index_name: str, table_name: str) -> None:
if _index_exists(table_name, index_name):
op.drop_index(op.f(index_name), table_name=table_name)
# (index_name, table_name, columns)
_INDEXES = [
# api_keys.user_id (CASCADE -> users.id)
('ix_api_keys_user_id', 'api_keys', ['user_id']),
# usage: wallet_id, provider_endpoint_id, provider_api_key_id (SET NULL)
('ix_usage_wallet_id', 'usage', ['wallet_id']),
('ix_usage_provider_endpoint_id', 'usage', ['provider_endpoint_id']),
('ix_usage_provider_api_key_id', 'usage', ['provider_api_key_id']),
# wallet_transactions.operator_id (SET NULL -> users.id)
('ix_wallet_transactions_operator_id', 'wallet_transactions', ['operator_id']),
# payment_callbacks.payment_order_id (SET NULL -> payment_orders.id)
('ix_payment_callbacks_payment_order_id', 'payment_callbacks', ['payment_order_id']),
# refund_requests: payment_order_id, requested_by, approved_by, processed_by (SET NULL)
('ix_refund_requests_payment_order_id', 'refund_requests', ['payment_order_id']),
('ix_refund_requests_requested_by', 'refund_requests', ['requested_by']),
('ix_refund_requests_approved_by', 'refund_requests', ['approved_by']),
('ix_refund_requests_processed_by', 'refund_requests', ['processed_by']),
# proxy_nodes.registered_by (SET NULL -> users.id)
('ix_proxy_nodes_registered_by', 'proxy_nodes', ['registered_by']),
# video_tasks: api_key_id, provider_id, endpoint_id, key_id, remixed_from_task_id
('ix_video_tasks_api_key_id', 'video_tasks', ['api_key_id']),
('ix_video_tasks_provider_id', 'video_tasks', ['provider_id']),
('ix_video_tasks_endpoint_id', 'video_tasks', ['endpoint_id']),
('ix_video_tasks_key_id', 'video_tasks', ['key_id']),
('ix_video_tasks_remixed_from_task_id', 'video_tasks', ['remixed_from_task_id']),
# user_preferences.default_provider_id (-> providers.id)
('ix_user_preferences_default_provider_id', 'user_preferences', ['default_provider_id']),
# announcements.author_id (SET NULL -> users.id)
('ix_announcements_author_id', 'announcements', ['author_id']),
# announcement_reads.announcement_id (-> announcements.id)
('ix_announcement_reads_announcement_id', 'announcement_reads', ['announcement_id']),
# request_candidates: user_id, api_key_id, endpoint_id, key_id (CASCADE)
('ix_request_candidates_user_id', 'request_candidates', ['user_id']),
('ix_request_candidates_api_key_id', 'request_candidates', ['api_key_id']),
('ix_request_candidates_endpoint_id', 'request_candidates', ['endpoint_id']),
('ix_request_candidates_key_id', 'request_candidates', ['key_id']),
]
def upgrade() -> None:
for index_name, table_name, columns in _INDEXES:
_create_index_if_not_exists(index_name, table_name, columns)
def downgrade() -> None:
for index_name, table_name, _columns in reversed(_INDEXES):
_drop_index_if_exists(index_name, table_name)
@@ -1,224 +0,0 @@
"""usage stats retention: SET NULL on delete and add name snapshots
Revision ID: 45b118150a78
Revises: 2d932114930d
Create Date: 2026-03-08 03:48:49.622091+00:00
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = "45b118150a78"
down_revision = "2d932114930d"
branch_labels = None
depends_on = None
_TABLES = ["usage", "stats_user_daily", "stats_daily_api_key"]
# ---------------------------------------------------------------------------
# Inline helpers
# ---------------------------------------------------------------------------
class _SchemaCache:
def __init__(self) -> None:
self._columns: dict[str, dict[str, str]] = {}
self._fk_rules: dict[tuple[str, str], str] = {}
self._fk_loaded_tables: set[str] = set()
def load_columns(self, tables: list[str]) -> None:
need = [t for t in tables if t not in self._columns]
if not need:
return
bind = op.get_bind()
rows = bind.execute(
sa.text(
"SELECT table_name, column_name, data_type "
"FROM information_schema.columns "
"WHERE table_name = ANY(:tables) "
" AND table_schema = current_schema()"
),
{"tables": need},
).fetchall()
for t in need:
self._columns.setdefault(t, {})
for table, col, dtype in rows:
self._columns[table][col] = dtype
def load_fk_rules(self, tables: list[str]) -> None:
need = [t for t in tables if t not in self._fk_loaded_tables]
if not need:
return
bind = op.get_bind()
rows = bind.execute(
sa.text(
"SELECT tc.table_name, tc.constraint_name, rc.delete_rule "
"FROM information_schema.referential_constraints rc "
"JOIN information_schema.table_constraints tc "
" ON rc.constraint_name = tc.constraint_name "
" AND rc.constraint_schema = tc.constraint_schema "
"WHERE tc.table_name = ANY(:tables) "
" AND tc.table_schema = current_schema()"
),
{"tables": need},
).fetchall()
for table, name, rule in rows:
self._fk_rules[(table, name)] = rule
self._fk_loaded_tables.update(need)
def column_exists(self, table: str, column: str) -> bool:
return column in self._columns.get(table, {})
def fk_ondelete(self, table: str, constraint: str) -> str | None:
return self._fk_rules.get((table, constraint))
def _fk_exists(constraint_name: str, table_name: str) -> bool:
bind = op.get_bind()
result = bind.execute(
sa.text(
"SELECT 1 FROM pg_constraint c "
"JOIN pg_class r ON c.conrelid = r.oid "
"JOIN pg_namespace n ON r.relnamespace = n.oid "
"WHERE c.conname = :name AND r.relname = :table "
" AND n.nspname = current_schema() AND c.contype = 'f'"
),
{"name": constraint_name, "table": table_name},
)
return result.scalar() is not None
def _replace_fk_if_needed(
cache: _SchemaCache,
constraint_name: str,
table_name: str,
ref_table: str,
local_cols: list[str],
remote_cols: list[str],
desired_ondelete: str,
) -> None:
current = cache.fk_ondelete(table_name, constraint_name)
if current and current.upper() == desired_ondelete.upper():
return
if current or _fk_exists(constraint_name, table_name):
op.drop_constraint(constraint_name, table_name, type_="foreignkey")
op.create_foreign_key(
constraint_name,
table_name,
ref_table,
local_cols,
remote_cols,
ondelete=desired_ondelete,
)
# ---------------------------------------------------------------------------
def upgrade() -> None:
c = _SchemaCache()
c.load_columns(_TABLES)
c.load_fk_rules(["stats_user_daily", "stats_daily_api_key"])
# --- Usage: add name snapshot columns ---
if not c.column_exists("usage", "username"):
op.add_column(
"usage", sa.Column("username", sa.String(100), nullable=True, comment="用户名快照")
)
if not c.column_exists("usage", "api_key_name"):
op.add_column(
"usage",
sa.Column("api_key_name", sa.String(200), nullable=True, comment="API Key 名称快照"),
)
# --- StatsUserDaily: CASCADE -> SET NULL, add username snapshot ---
_replace_fk_if_needed(
c,
"stats_user_daily_user_id_fkey",
"stats_user_daily",
"users",
["user_id"],
["id"],
"SET NULL",
)
op.alter_column("stats_user_daily", "user_id", existing_type=sa.String(36), nullable=True)
if not c.column_exists("stats_user_daily", "username"):
op.add_column(
"stats_user_daily",
sa.Column(
"username",
sa.String(100),
nullable=True,
comment="用户名快照(删除用户后仍可追溯)",
),
)
# --- StatsDailyApiKey: CASCADE -> SET NULL, add api_key_name snapshot ---
_replace_fk_if_needed(
c,
"stats_daily_api_key_api_key_id_fkey",
"stats_daily_api_key",
"api_keys",
["api_key_id"],
["id"],
"SET NULL",
)
op.alter_column("stats_daily_api_key", "api_key_id", existing_type=sa.String(36), nullable=True)
if not c.column_exists("stats_daily_api_key", "api_key_name"):
op.add_column(
"stats_daily_api_key",
sa.Column(
"api_key_name",
sa.String(200),
nullable=True,
comment="API Key 名称快照(删除 Key 后仍可追溯)",
),
)
def downgrade() -> None:
c = _SchemaCache()
c.load_columns(["stats_daily_api_key", "stats_user_daily", "usage"])
c.load_fk_rules(["stats_daily_api_key", "stats_user_daily"])
# --- Remove snapshot columns ---
if c.column_exists("stats_daily_api_key", "api_key_name"):
op.drop_column("stats_daily_api_key", "api_key_name")
if c.column_exists("stats_user_daily", "username"):
op.drop_column("stats_user_daily", "username")
if c.column_exists("usage", "api_key_name"):
op.drop_column("usage", "api_key_name")
if c.column_exists("usage", "username"):
op.drop_column("usage", "username")
# --- StatsDailyApiKey: SET NULL -> CASCADE ---
_replace_fk_if_needed(
c,
"stats_daily_api_key_api_key_id_fkey",
"stats_daily_api_key",
"api_keys",
["api_key_id"],
["id"],
"CASCADE",
)
op.alter_column(
"stats_daily_api_key", "api_key_id", existing_type=sa.String(36), nullable=False
)
# --- StatsUserDaily: SET NULL -> CASCADE ---
_replace_fk_if_needed(
c,
"stats_user_daily_user_id_fkey",
"stats_user_daily",
"users",
["user_id"],
["id"],
"CASCADE",
)
op.alter_column("stats_user_daily", "user_id", existing_type=sa.String(36), nullable=False)
@@ -1,257 +0,0 @@
"""request_candidates/video_tasks retention: SET NULL and add snapshots
Revision ID: 13a4c8f6d9e0
Revises: 45b118150a78
Create Date: 2026-03-08 12:15:00.000000+00:00
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = "13a4c8f6d9e0"
down_revision = "45b118150a78"
branch_labels = None
depends_on = None
_TABLES = ["request_candidates", "video_tasks"]
# ---------------------------------------------------------------------------
# Inline helpers
# ---------------------------------------------------------------------------
class _SchemaCache:
def __init__(self) -> None:
self._columns: dict[str, dict[str, str]] = {}
self._fk_rules: dict[tuple[str, str], str] = {}
self._fk_loaded_tables: set[str] = set()
def load_columns(self, tables: list[str]) -> None:
need = [t for t in tables if t not in self._columns]
if not need:
return
bind = op.get_bind()
rows = bind.execute(
sa.text(
"SELECT table_name, column_name, data_type "
"FROM information_schema.columns "
"WHERE table_name = ANY(:tables) "
" AND table_schema = current_schema()"
),
{"tables": need},
).fetchall()
for t in need:
self._columns.setdefault(t, {})
for table, col, dtype in rows:
self._columns[table][col] = dtype
def load_fk_rules(self, tables: list[str]) -> None:
need = [t for t in tables if t not in self._fk_loaded_tables]
if not need:
return
bind = op.get_bind()
rows = bind.execute(
sa.text(
"SELECT tc.table_name, tc.constraint_name, rc.delete_rule "
"FROM information_schema.referential_constraints rc "
"JOIN information_schema.table_constraints tc "
" ON rc.constraint_name = tc.constraint_name "
" AND rc.constraint_schema = tc.constraint_schema "
"WHERE tc.table_name = ANY(:tables) "
" AND tc.table_schema = current_schema()"
),
{"tables": need},
).fetchall()
for table, name, rule in rows:
self._fk_rules[(table, name)] = rule
self._fk_loaded_tables.update(need)
def column_exists(self, table: str, column: str) -> bool:
return column in self._columns.get(table, {})
def fk_ondelete(self, table: str, constraint: str) -> str | None:
return self._fk_rules.get((table, constraint))
def _fk_exists(constraint_name: str, table_name: str) -> bool:
bind = op.get_bind()
result = bind.execute(
sa.text(
"SELECT 1 FROM pg_constraint c "
"JOIN pg_class r ON c.conrelid = r.oid "
"JOIN pg_namespace n ON r.relnamespace = n.oid "
"WHERE c.conname = :name AND r.relname = :table "
" AND n.nspname = current_schema() AND c.contype = 'f'"
),
{"name": constraint_name, "table": table_name},
)
return result.scalar() is not None
def _replace_fk_if_needed(
cache: _SchemaCache,
constraint_name: str,
table_name: str,
ref_table: str,
local_cols: list[str],
remote_cols: list[str],
desired_ondelete: str,
) -> None:
current = cache.fk_ondelete(table_name, constraint_name)
if current and current.upper() == desired_ondelete.upper():
return
if current or _fk_exists(constraint_name, table_name):
op.drop_constraint(constraint_name, table_name, type_="foreignkey")
op.create_foreign_key(
constraint_name,
table_name,
ref_table,
local_cols,
remote_cols,
ondelete=desired_ondelete,
)
# ---------------------------------------------------------------------------
def upgrade() -> None:
c = _SchemaCache()
c.load_columns(_TABLES)
c.load_fk_rules(_TABLES)
# --- request_candidates: add snapshot columns ---
if not c.column_exists("request_candidates", "username"):
op.add_column(
"request_candidates",
sa.Column("username", sa.String(length=100), nullable=True, comment="用户名快照"),
)
if not c.column_exists("request_candidates", "api_key_name"):
op.add_column(
"request_candidates",
sa.Column(
"api_key_name",
sa.String(length=200),
nullable=True,
comment="API Key 名称快照",
),
)
# --- request_candidates: CASCADE -> SET NULL ---
_replace_fk_if_needed(
c,
"request_candidates_user_id_fkey",
"request_candidates",
"users",
["user_id"],
["id"],
"SET NULL",
)
_replace_fk_if_needed(
c,
"request_candidates_api_key_id_fkey",
"request_candidates",
"api_keys",
["api_key_id"],
["id"],
"SET NULL",
)
# --- video_tasks: add snapshot columns ---
if not c.column_exists("video_tasks", "username"):
op.add_column(
"video_tasks",
sa.Column("username", sa.String(length=100), nullable=True, comment="用户名快照"),
)
if not c.column_exists("video_tasks", "api_key_name"):
op.add_column(
"video_tasks",
sa.Column(
"api_key_name",
sa.String(length=200),
nullable=True,
comment="API Key 名称快照",
),
)
# --- video_tasks: CASCADE -> SET NULL, user_id nullable ---
op.alter_column("video_tasks", "user_id", existing_type=sa.String(length=36), nullable=True)
_replace_fk_if_needed(
c,
"video_tasks_user_id_fkey",
"video_tasks",
"users",
["user_id"],
["id"],
"SET NULL",
)
_replace_fk_if_needed(
c,
"video_tasks_api_key_id_fkey",
"video_tasks",
"api_keys",
["api_key_id"],
["id"],
"SET NULL",
)
def downgrade() -> None:
c = _SchemaCache()
c.load_columns(_TABLES)
c.load_fk_rules(_TABLES)
# --- video_tasks: SET NULL -> default (no action), restore NOT NULL ---
_replace_fk_if_needed(
c,
"video_tasks_api_key_id_fkey",
"video_tasks",
"api_keys",
["api_key_id"],
["id"],
"NO ACTION",
)
_replace_fk_if_needed(
c,
"video_tasks_user_id_fkey",
"video_tasks",
"users",
["user_id"],
["id"],
"NO ACTION",
)
op.alter_column("video_tasks", "user_id", existing_type=sa.String(length=36), nullable=False)
if c.column_exists("video_tasks", "api_key_name"):
op.drop_column("video_tasks", "api_key_name")
if c.column_exists("video_tasks", "username"):
op.drop_column("video_tasks", "username")
# --- request_candidates: SET NULL -> CASCADE ---
_replace_fk_if_needed(
c,
"request_candidates_api_key_id_fkey",
"request_candidates",
"api_keys",
["api_key_id"],
["id"],
"CASCADE",
)
_replace_fk_if_needed(
c,
"request_candidates_user_id_fkey",
"request_candidates",
"users",
["user_id"],
["id"],
"CASCADE",
)
if c.column_exists("request_candidates", "api_key_name"):
op.drop_column("request_candidates", "api_key_name")
if c.column_exists("request_candidates", "username"):
op.drop_column("request_candidates", "username")
@@ -1,230 +0,0 @@
"""cost fields: Float -> Numeric(20,8) + provider_api_keys composite index
Revision ID: 2053ab8ed764
Revises: 13a4c8f6d9e0
Create Date: 2026-03-08 15:30:00.000000+00:00
"""
from __future__ import annotations
import re
from collections import defaultdict
from collections.abc import Callable
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = "2053ab8ed764"
down_revision = "13a4c8f6d9e0"
branch_labels = None
depends_on = None
# (table_name, column_name, nullable, server_default)
_COST_COLUMNS: list[tuple[str, str, bool, str | None]] = [
# api_keys
("api_keys", "total_cost_usd", True, "0.0"),
# usage
("usage", "input_cost_usd", True, "0.0"),
("usage", "output_cost_usd", True, "0.0"),
("usage", "cache_cost_usd", True, "0.0"),
("usage", "cache_creation_cost_usd", True, "0.0"),
("usage", "cache_read_cost_usd", True, "0.0"),
("usage", "request_cost_usd", True, "0.0"),
("usage", "total_cost_usd", True, "0.0"),
("usage", "actual_input_cost_usd", True, "0.0"),
("usage", "actual_output_cost_usd", True, "0.0"),
("usage", "actual_cache_creation_cost_usd", True, "0.0"),
("usage", "actual_cache_read_cost_usd", True, "0.0"),
("usage", "actual_request_cost_usd", True, "0.0"),
("usage", "actual_total_cost_usd", True, "0.0"),
("usage", "rate_multiplier", True, "1.0"),
("usage", "input_price_per_1m", True, None),
("usage", "output_price_per_1m", True, None),
("usage", "cache_creation_price_per_1m", True, None),
("usage", "cache_read_price_per_1m", True, None),
("usage", "price_per_request", True, None),
# providers
("providers", "monthly_quota_usd", True, None),
("providers", "monthly_used_usd", True, "0.0"),
# global_models
("global_models", "default_price_per_request", True, None),
# models
("models", "price_per_request", True, None),
# stats_hourly
("stats_hourly", "total_cost", False, "0.0"),
("stats_hourly", "actual_total_cost", False, "0.0"),
# stats_hourly_user
("stats_hourly_user", "total_cost", False, "0.0"),
# stats_hourly_model
("stats_hourly_model", "total_cost", False, "0.0"),
# stats_hourly_provider
("stats_hourly_provider", "total_cost", False, "0.0"),
# stats_daily
("stats_daily", "total_cost", False, "0.0"),
("stats_daily", "actual_total_cost", False, "0.0"),
("stats_daily", "input_cost", False, "0.0"),
("stats_daily", "output_cost", False, "0.0"),
("stats_daily", "cache_creation_cost", False, "0.0"),
("stats_daily", "cache_read_cost", False, "0.0"),
# stats_daily_model
("stats_daily_model", "total_cost", False, "0.0"),
# stats_daily_provider
("stats_daily_provider", "total_cost", False, "0.0"),
# stats_daily_api_key
("stats_daily_api_key", "total_cost", False, "0.0"),
# stats_summary
("stats_summary", "all_time_cost", False, "0.0"),
("stats_summary", "all_time_actual_cost", False, "0.0"),
# stats_user_daily
("stats_user_daily", "total_cost", False, "0.0"),
]
_ALL_TABLES = list({t for t, *_ in _COST_COLUMNS})
# ---------------------------------------------------------------------------
# Inline helpers
# ---------------------------------------------------------------------------
class _SchemaCache:
def __init__(self) -> None:
self._columns: dict[str, dict[str, str]] = {}
def load_columns(self, tables: list[str]) -> None:
need = [t for t in tables if t not in self._columns]
if not need:
return
bind = op.get_bind()
rows = bind.execute(
sa.text(
"SELECT table_name, column_name, data_type "
"FROM information_schema.columns "
"WHERE table_name = ANY(:tables) "
" AND table_schema = current_schema()"
),
{"tables": need},
).fetchall()
for t in need:
self._columns.setdefault(t, {})
for table, col, dtype in rows:
self._columns[table][col] = dtype
def column_exists(self, table: str, column: str) -> bool:
return column in self._columns.get(table, {})
def column_type(self, table: str, column: str) -> str | None:
return self._columns.get(table, {}).get(column)
def is_numeric(self, table: str, column: str) -> bool:
return self.column_type(table, column) == "numeric"
def _index_exists(index_name: str) -> bool:
bind = op.get_bind()
result = bind.execute(
sa.text(
"SELECT 1 FROM pg_indexes "
"WHERE indexname = :name AND schemaname = current_schema()::text"
),
{"name": index_name},
)
return result.scalar() is not None
def _numeric_max(type_spec: str) -> float | None:
m = re.match(r"NUMERIC\((\d+),(\d+)\)", type_spec, re.IGNORECASE)
if not m:
return None
precision, scale = int(m.group(1)), int(m.group(2))
return 10 ** (precision - scale) - 10 ** (-scale)
def _batch_alter_type(
cache: _SchemaCache,
columns: list[tuple[str, str, bool, str | None]],
cast_suffix: str,
type_fn: Callable[[str], str],
) -> None:
by_table: dict[str, list[tuple[str, str]]] = defaultdict(list)
for table, col, _nullable, _default in columns:
if not cache.column_exists(table, col):
continue
by_table[table].append((col, type_fn(col)))
bind = op.get_bind()
for table, col_types in by_table.items():
for col, target in col_types:
cap = _numeric_max(target)
if cap is not None:
bind.execute(
sa.text(
f"UPDATE {table} SET {col} = :cap "
f"WHERE {col} IS NOT NULL AND abs({col}) > :cap"
),
{"cap": cap},
)
parts = [
f"ALTER COLUMN {col} TYPE {target} USING {col}::{cast_suffix}"
for col, target in col_types
]
if parts:
bind.execute(sa.text(f"ALTER TABLE {table} " + ", ".join(parts)))
# ---------------------------------------------------------------------------
def _type_spec(col: str) -> str:
"""Return the SQL type literal for a given column name."""
return "NUMERIC(10,6)" if col == "rate_multiplier" else "NUMERIC(20,8)"
def upgrade() -> None:
c = _SchemaCache()
c.load_columns(_ALL_TABLES)
# -- 1. cost fields: Float -> Numeric (batched per table)
cols_to_convert = [
(t, col, n, d)
for t, col, n, d in _COST_COLUMNS
if c.column_exists(t, col) and not c.is_numeric(t, col)
]
_batch_alter_type(c, cols_to_convert, cast_suffix="numeric", type_fn=_type_spec)
# -- 2. provider_api_keys composite index
if not _index_exists("idx_provider_api_keys_provider_active"):
op.create_index(
"idx_provider_api_keys_provider_active",
"provider_api_keys",
["provider_id", "is_active"],
)
def downgrade() -> None:
# -- 2. drop composite index
if _index_exists("idx_provider_api_keys_provider_active"):
op.drop_index(
"idx_provider_api_keys_provider_active",
table_name="provider_api_keys",
)
# -- 1. Numeric -> Float (batched per table)
c = _SchemaCache()
c.load_columns(_ALL_TABLES)
cols_to_revert = [
(t, col, n, d)
for t, col, n, d in _COST_COLUMNS
if c.column_exists(t, col) and c.is_numeric(t, col)
]
_batch_alter_type(
c,
cols_to_revert,
cast_suffix="double precision",
type_fn=lambda _col: "DOUBLE PRECISION",
)
@@ -1,127 +0,0 @@
"""video_tasks.key_id: add ondelete SET NULL
Revision ID: d7649c1f8e21
Revises: 2053ab8ed764
Create Date: 2026-03-09 01:00:00.000000+00:00
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = "d7649c1f8e21"
down_revision = "2053ab8ed764"
branch_labels = None
depends_on = None
_TABLE = "video_tasks"
_FK_NAME = "video_tasks_key_id_fkey"
# ---------------------------------------------------------------------------
# Inline helpers
# ---------------------------------------------------------------------------
class _SchemaCache:
def __init__(self) -> None:
self._fk_rules: dict[tuple[str, str], str] = {}
self._fk_loaded_tables: set[str] = set()
def load_fk_rules(self, tables: list[str]) -> None:
need = [t for t in tables if t not in self._fk_loaded_tables]
if not need:
return
bind = op.get_bind()
rows = bind.execute(
sa.text(
"SELECT tc.table_name, tc.constraint_name, rc.delete_rule "
"FROM information_schema.referential_constraints rc "
"JOIN information_schema.table_constraints tc "
" ON rc.constraint_name = tc.constraint_name "
" AND rc.constraint_schema = tc.constraint_schema "
"WHERE tc.table_name = ANY(:tables) "
" AND tc.table_schema = current_schema()"
),
{"tables": need},
).fetchall()
for table, name, rule in rows:
self._fk_rules[(table, name)] = rule
self._fk_loaded_tables.update(need)
def fk_ondelete(self, table: str, constraint: str) -> str | None:
return self._fk_rules.get((table, constraint))
def _fk_exists(constraint_name: str, table_name: str) -> bool:
bind = op.get_bind()
result = bind.execute(
sa.text(
"SELECT 1 FROM pg_constraint c "
"JOIN pg_class r ON c.conrelid = r.oid "
"JOIN pg_namespace n ON r.relnamespace = n.oid "
"WHERE c.conname = :name AND r.relname = :table "
" AND n.nspname = current_schema() AND c.contype = 'f'"
),
{"name": constraint_name, "table": table_name},
)
return result.scalar() is not None
def _replace_fk_if_needed(
cache: _SchemaCache,
constraint_name: str,
table_name: str,
ref_table: str,
local_cols: list[str],
remote_cols: list[str],
desired_ondelete: str,
) -> None:
current = cache.fk_ondelete(table_name, constraint_name)
if current and current.upper() == desired_ondelete.upper():
return
if current or _fk_exists(constraint_name, table_name):
op.drop_constraint(constraint_name, table_name, type_="foreignkey")
op.create_foreign_key(
constraint_name,
table_name,
ref_table,
local_cols,
remote_cols,
ondelete=desired_ondelete,
)
# ---------------------------------------------------------------------------
def upgrade() -> None:
c = _SchemaCache()
c.load_fk_rules([_TABLE])
_replace_fk_if_needed(
c,
_FK_NAME,
_TABLE,
"provider_api_keys",
["key_id"],
["id"],
"SET NULL",
)
def downgrade() -> None:
c = _SchemaCache()
c.load_fk_rules([_TABLE])
_replace_fk_if_needed(
c,
_FK_NAME,
_TABLE,
"provider_api_keys",
["key_id"],
["id"],
"NO ACTION",
)
@@ -1,42 +0,0 @@
"""Strip request_results_window from health_by_format JSON.
This data is now maintained in process memory only, no longer persisted to DB.
Revision ID: a3f1b7c9d2e4
Revises: d7649c1f8e21
Create Date: 2026-03-10 12:00:00.000000+00:00
"""
from alembic import op
# revision identifiers, used by Alembic.
revision = "a3f1b7c9d2e4"
down_revision = "d7649c1f8e21"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.execute("""
UPDATE provider_api_keys
SET health_by_format = (
SELECT jsonb_object_agg(
fmt_key,
fmt_value - 'request_results_window'
)
FROM jsonb_each(health_by_format) AS x(fmt_key, fmt_value)
)
WHERE health_by_format IS NOT NULL
AND health_by_format != '{}'::jsonb
AND EXISTS (
SELECT 1
FROM jsonb_each(health_by_format) AS x(fmt_key, fmt_value)
WHERE fmt_value ? 'request_results_window'
)
""")
def downgrade() -> None:
# No-op: window data is rebuilt from scratch on process start
pass
@@ -1,159 +0,0 @@
"""tighten usage billing state machine
Revision ID: 9e4f1a2b3c4d
Revises: a3f1b7c9d2e4
Create Date: 2026-03-11 19:00:00.000000+00:00
This migration does two things:
1. Change new `usage.billing_status` default from `settled` to `pending`.
2. Repair only the clearly-safe inconsistent historical rows for production:
- failed/cancelled zero-cost rows that were marked settled are converted to void
- terminal rows missing finalized_at are backfilled from created_at
Ambiguous positive-cost settled rows are intentionally left untouched for manual audit.
All data updates are batched (10000 rows per iteration) to avoid long-held locks
and excessive WAL generation on large usage tables.
"""
from __future__ import annotations
from collections.abc import Sequence
import sqlalchemy as sa
from sqlalchemy import inspect
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "9e4f1a2b3c4d"
down_revision: str | None = "a3f1b7c9d2e4"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
BATCH_SIZE = 10000
def _table_exists(table_name: str) -> bool:
bind = op.get_bind()
insp = inspect(bind)
insp.clear_cache()
return table_name in insp.get_table_names()
def _column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
insp = inspect(bind)
insp.clear_cache()
return column_name in [col["name"] for col in insp.get_columns(table_name)]
def upgrade() -> None:
if not _table_exists("usage"):
return
if _column_exists("usage", "billing_status"):
op.alter_column(
"usage",
"billing_status",
existing_type=sa.String(length=20),
server_default="pending",
existing_nullable=False,
)
required_columns = {
"billing_status",
"status",
"total_cost_usd",
"request_cost_usd",
"actual_total_cost_usd",
"actual_request_cost_usd",
"wallet_balance_after",
"finalized_at",
"created_at",
}
if not required_columns.issubset(
{col for col in required_columns if _column_exists("usage", col)}
):
return
conn = op.get_bind()
# Step 1: billing_status IS NULL -> 'pending' (batched)
while True:
result = conn.execute(
sa.text("""
WITH batch AS (
SELECT id FROM usage
WHERE billing_status IS NULL
LIMIT :batch_size
FOR UPDATE SKIP LOCKED
)
UPDATE usage
SET billing_status = 'pending'
FROM batch WHERE usage.id = batch.id
"""),
{"batch_size": BATCH_SIZE},
)
if result.rowcount < BATCH_SIZE:
break
# Step 2: failed/cancelled zero-cost settled -> void (batched)
while True:
result = conn.execute(
sa.text("""
WITH batch AS (
SELECT id FROM usage
WHERE billing_status = 'settled'
AND status IN ('failed', 'cancelled')
AND COALESCE(total_cost_usd, 0) = 0
AND wallet_balance_after IS NULL
LIMIT :batch_size
FOR UPDATE SKIP LOCKED
)
UPDATE usage
SET billing_status = 'void',
finalized_at = COALESCE(usage.finalized_at, usage.created_at),
total_cost_usd = 0,
request_cost_usd = 0,
actual_total_cost_usd = 0,
actual_request_cost_usd = 0
FROM batch WHERE usage.id = batch.id
"""),
{"batch_size": BATCH_SIZE},
)
if result.rowcount < BATCH_SIZE:
break
# Step 3: backfill finalized_at for terminal rows (batched)
while True:
result = conn.execute(
sa.text("""
WITH batch AS (
SELECT id FROM usage
WHERE billing_status IN ('settled', 'void')
AND finalized_at IS NULL
LIMIT :batch_size
FOR UPDATE SKIP LOCKED
)
UPDATE usage
SET finalized_at = COALESCE(usage.finalized_at, usage.created_at)
FROM batch WHERE usage.id = batch.id
"""),
{"batch_size": BATCH_SIZE},
)
if result.rowcount < BATCH_SIZE:
break
def downgrade() -> None:
if not _table_exists("usage") or not _column_exists("usage", "billing_status"):
return
op.alter_column(
"usage",
"billing_status",
existing_type=sa.String(length=20),
server_default="settled",
existing_nullable=False,
)
@@ -1,113 +0,0 @@
"""add wallet daily usage ledgers
Revision ID: d4e5f6a7b8c9
Revises: 9e4f1a2b3c4d
Create Date: 2026-03-11 21:00:00.000000+00:00
"""
from __future__ import annotations
from collections.abc import Sequence
import sqlalchemy as sa
from sqlalchemy import inspect
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "d4e5f6a7b8c9"
down_revision: str | None = "9e4f1a2b3c4d"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def _table_exists(table_name: str) -> bool:
bind = op.get_bind()
insp = inspect(bind)
insp.clear_cache()
return table_name in insp.get_table_names()
def _column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
insp = inspect(bind)
insp.clear_cache()
return column_name in [col["name"] for col in insp.get_columns(table_name)]
def _index_exists(table_name: str, index_name: str) -> bool:
bind = op.get_bind()
insp = inspect(bind)
insp.clear_cache()
return any(idx["name"] == index_name for idx in insp.get_indexes(table_name))
def upgrade() -> None:
if not _table_exists("wallet_daily_usage_ledgers"):
op.create_table(
"wallet_daily_usage_ledgers",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("wallet_id", sa.String(length=36), nullable=False),
sa.Column("billing_date", sa.Date(), nullable=False),
sa.Column("billing_timezone", sa.String(length=64), nullable=False),
sa.Column("total_cost_usd", sa.Numeric(20, 8), nullable=False, server_default="0"),
sa.Column("total_requests", sa.Integer(), nullable=False, server_default="0"),
sa.Column("input_tokens", sa.BigInteger(), nullable=False, server_default="0"),
sa.Column("output_tokens", sa.BigInteger(), nullable=False, server_default="0"),
sa.Column("cache_creation_tokens", sa.BigInteger(), nullable=False, server_default="0"),
sa.Column("cache_read_tokens", sa.BigInteger(), nullable=False, server_default="0"),
sa.Column("first_finalized_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("last_finalized_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("aggregated_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(["wallet_id"], ["wallets.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint(
"wallet_id",
"billing_date",
"billing_timezone",
name="uq_wallet_daily_usage_ledgers_wallet_date_tz",
),
)
if not _index_exists("wallet_daily_usage_ledgers", "idx_wallet_daily_usage_wallet_date"):
op.create_index(
"idx_wallet_daily_usage_wallet_date",
"wallet_daily_usage_ledgers",
["wallet_id", "billing_date"],
)
if not _index_exists("wallet_daily_usage_ledgers", "idx_wallet_daily_usage_date"):
op.create_index(
"idx_wallet_daily_usage_date",
"wallet_daily_usage_ledgers",
["billing_date"],
)
if (
_table_exists("usage")
and all(
_column_exists("usage", col) for col in ["billing_status", "finalized_at", "wallet_id"]
)
and not _index_exists("usage", "idx_usage_billing_finalized_wallet")
):
op.create_index(
"idx_usage_billing_finalized_wallet",
"usage",
["billing_status", "finalized_at", "wallet_id"],
)
def downgrade() -> None:
if _table_exists("usage") and _index_exists("usage", "idx_usage_billing_finalized_wallet"):
op.drop_index("idx_usage_billing_finalized_wallet", table_name="usage")
if _table_exists("wallet_daily_usage_ledgers"):
if _index_exists("wallet_daily_usage_ledgers", "idx_wallet_daily_usage_date"):
op.drop_index("idx_wallet_daily_usage_date", table_name="wallet_daily_usage_ledgers")
if _index_exists("wallet_daily_usage_ledgers", "idx_wallet_daily_usage_wallet_date"):
op.drop_index(
"idx_wallet_daily_usage_wallet_date",
table_name="wallet_daily_usage_ledgers",
)
op.drop_table("wallet_daily_usage_ledgers")
@@ -1,59 +0,0 @@
"""add provider_api_keys usage total columns
Revision ID: 9b7c6d5e4f3a
Revises: d4e5f6a7b8c9
Create Date: 2026-03-11 22:00:00.000000+00:00
"""
from __future__ import annotations
from collections.abc import Sequence
import sqlalchemy as sa
from sqlalchemy import inspect
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "9b7c6d5e4f3a"
down_revision: str | None = "d4e5f6a7b8c9"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
columns = [c["name"] for c in inspector.get_columns(table_name)]
return column_name in columns
def upgrade() -> None:
if not column_exists("provider_api_keys", "total_tokens"):
op.add_column(
"provider_api_keys",
sa.Column("total_tokens", sa.BigInteger(), nullable=False, server_default="0"),
)
if column_exists("provider_api_keys", "total_tokens"):
op.alter_column("provider_api_keys", "total_tokens", server_default=None)
if not column_exists("provider_api_keys", "total_cost_usd"):
op.add_column(
"provider_api_keys",
sa.Column(
"total_cost_usd",
sa.Numeric(20, 8),
nullable=False,
server_default="0.0",
),
)
if column_exists("provider_api_keys", "total_cost_usd"):
op.alter_column("provider_api_keys", "total_cost_usd", server_default=None)
def downgrade() -> None:
if column_exists("provider_api_keys", "total_cost_usd"):
op.drop_column("provider_api_keys", "total_cost_usd")
if column_exists("provider_api_keys", "total_tokens"):
op.drop_column("provider_api_keys", "total_tokens")
@@ -1,116 +0,0 @@
"""cleanup stale provider references after provider deletion
Revision ID: c1d2e3f4a5b6
Revises: 9b7c6d5e4f3a
Create Date: 2026-03-11 23:00:00.000000+00:00
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = "c1d2e3f4a5b6"
down_revision = "9b7c6d5e4f3a"
branch_labels = None
depends_on = None
_users = sa.table(
"users",
sa.column("id", sa.String(36)),
sa.column("allowed_providers", sa.JSON()),
)
_api_keys = sa.table(
"api_keys",
sa.column("id", sa.String(36)),
sa.column("allowed_providers", sa.JSON()),
)
_user_preferences = sa.table(
"user_preferences",
sa.column("id", sa.String(36)),
sa.column("default_provider_id", sa.String(36)),
)
_video_tasks = sa.table(
"video_tasks",
sa.column("id", sa.String(36)),
sa.column("provider_id", sa.String(36)),
sa.column("endpoint_id", sa.String(36)),
)
_providers = sa.table("providers", sa.column("id", sa.String(36)))
_provider_endpoints = sa.table("provider_endpoints", sa.column("id", sa.String(36)))
def _load_valid_ids(conn: sa.Connection, table: sa.Table) -> set[str]:
return {str(row[0]) for row in conn.execute(sa.select(table.c.id)).fetchall() if row[0]}
def _cleanup_allowed_providers(
conn: sa.Connection,
table: sa.Table,
valid_provider_ids: set[str],
) -> None:
rows = conn.execute(
sa.select(table.c.id, table.c.allowed_providers).where(
table.c.allowed_providers.isnot(None)
)
).fetchall()
for row_id, allowed_providers in rows:
if not isinstance(allowed_providers, list):
continue
filtered = [
provider_id for provider_id in allowed_providers if provider_id in valid_provider_ids
]
if filtered == allowed_providers:
continue
conn.execute(table.update().where(table.c.id == row_id).values(allowed_providers=filtered))
def _nullify_missing_fk(
conn: sa.Connection,
table: sa.Table,
id_column: sa.ColumnElement[str],
fk_column: sa.ColumnElement[str],
valid_ids: set[str],
) -> None:
rows = conn.execute(sa.select(id_column, fk_column).where(fk_column.isnot(None))).fetchall()
invalid_row_ids = [row_id for row_id, fk_value in rows if fk_value not in valid_ids]
if not invalid_row_ids:
return
conn.execute(table.update().where(id_column.in_(invalid_row_ids)).values({fk_column.key: None}))
def upgrade() -> None:
conn = op.get_bind()
valid_provider_ids = _load_valid_ids(conn, _providers)
valid_endpoint_ids = _load_valid_ids(conn, _provider_endpoints)
_cleanup_allowed_providers(conn, _users, valid_provider_ids)
_cleanup_allowed_providers(conn, _api_keys, valid_provider_ids)
_nullify_missing_fk(
conn,
_user_preferences,
_user_preferences.c.id,
_user_preferences.c.default_provider_id,
valid_provider_ids,
)
_nullify_missing_fk(
conn,
_video_tasks,
_video_tasks.c.id,
_video_tasks.c.provider_id,
valid_provider_ids,
)
_nullify_missing_fk(
conn,
_video_tasks,
_video_tasks.c.id,
_video_tasks.c.endpoint_id,
valid_endpoint_ids,
)
def downgrade() -> None:
pass
@@ -1,64 +0,0 @@
"""decouple request_candidates.key_id foreign key from provider_api_keys lifecycle
Revision ID: b7c8d9e0f1a2
Revises: c1d2e3f4a5b6
Create Date: 2026-03-12 19:15:00.000000+00:00
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = "b7c8d9e0f1a2"
down_revision = "c1d2e3f4a5b6"
branch_labels = None
depends_on = None
def _fk_exists(constraint_name: str, table_name: str) -> bool:
bind = op.get_bind()
result = bind.execute(
sa.text(
"SELECT 1 FROM pg_constraint c "
"JOIN pg_class r ON c.conrelid = r.oid "
"JOIN pg_namespace n ON r.relnamespace = n.oid "
"WHERE c.conname = :name AND r.relname = :table "
" AND n.nspname = current_schema() AND c.contype = 'f'"
),
{"name": constraint_name, "table": table_name},
)
return result.scalar() is not None
def upgrade() -> None:
if _fk_exists("request_candidates_key_id_fkey", "request_candidates"):
op.drop_constraint(
"request_candidates_key_id_fkey", "request_candidates", type_="foreignkey"
)
def downgrade() -> None:
bind = op.get_bind()
bind.execute(
sa.text(
"UPDATE request_candidates rc "
"SET key_id = NULL "
"WHERE key_id IS NOT NULL "
" AND NOT EXISTS ("
" SELECT 1 FROM provider_api_keys pak WHERE pak.id = rc.key_id"
" )"
)
)
if not _fk_exists("request_candidates_key_id_fkey", "request_candidates"):
op.create_foreign_key(
"request_candidates_key_id_fkey",
"request_candidates",
"provider_api_keys",
["key_id"],
["id"],
ondelete="CASCADE",
)
@@ -1,54 +0,0 @@
"""add user rate_limit and backfill normal api key limits
Revision ID: b7e8f9a0c1d2
Revises: b7c8d9e0f1a2
Create Date: 2026-03-13 12:00:00.000000+00:00
"""
from __future__ import annotations
from collections.abc import Sequence
import sqlalchemy as sa
from sqlalchemy import inspect
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "b7e8f9a0c1d2"
down_revision: str | None = "b7c8d9e0f1a2"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
columns = [c["name"] for c in inspector.get_columns(table_name)]
return column_name in columns
def upgrade() -> None:
if not column_exists("users", "rate_limit"):
op.add_column("users", sa.Column("rate_limit", sa.Integer(), nullable=True))
# 普通 Key 新语义不再允许 NULL;存量 NULL 统一回填为 0(不限制)。
op.execute(sa.text("""
UPDATE api_keys
SET rate_limit = 0
WHERE is_standalone = FALSE
AND rate_limit IS NULL
"""))
def downgrade() -> None:
# 恢复普通 Key 的 rate_limit 为 NULL(与 upgrade 中回填 0 对应)
op.execute(sa.text("""
UPDATE api_keys
SET rate_limit = NULL
WHERE is_standalone = FALSE
AND rate_limit = 0
"""))
if column_exists("users", "rate_limit"):
op.drop_column("users", "rate_limit")
@@ -1,87 +0,0 @@
"""add user sessions table for device-level auth
Revision ID: f6e7d8c9b0a1
Revises: b7e8f9a0c1d2
Create Date: 2026-03-15 12:00:00.000000+00:00
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = "f6e7d8c9b0a1"
down_revision = "b7e8f9a0c1d2"
branch_labels = None
depends_on = None
def upgrade() -> None:
bind = op.get_bind()
inspector = sa.inspect(bind)
if "user_sessions" in inspector.get_table_names():
return
op.create_table(
"user_sessions",
sa.Column("id", sa.String(length=36), nullable=False),
sa.Column("user_id", sa.String(length=36), nullable=False),
sa.Column("client_device_id", sa.String(length=128), nullable=False),
sa.Column("device_label", sa.String(length=120), nullable=True),
sa.Column("device_type", sa.String(length=20), nullable=False, server_default="unknown"),
sa.Column("browser_name", sa.String(length=50), nullable=True),
sa.Column("browser_version", sa.String(length=50), nullable=True),
sa.Column("os_name", sa.String(length=50), nullable=True),
sa.Column("os_version", sa.String(length=50), nullable=True),
sa.Column("device_model", sa.String(length=100), nullable=True),
sa.Column("ip_address", sa.String(length=45), nullable=True),
sa.Column("user_agent", sa.String(length=1000), nullable=True),
sa.Column("client_hints", sa.JSON(), nullable=True),
sa.Column("refresh_token_hash", sa.String(length=64), nullable=False),
sa.Column("prev_refresh_token_hash", sa.String(length=64), nullable=True),
sa.Column("rotated_at", sa.DateTime(timezone=True), nullable=True),
sa.Column(
"last_seen_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()
),
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("revoked_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("revoke_reason", sa.String(length=100), nullable=True),
sa.Column(
"created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()
),
sa.ForeignKeyConstraint(["user_id"], ["users.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
)
op.create_index("ix_user_sessions_user_id", "user_sessions", ["user_id"], unique=False)
op.create_index(
"ix_user_sessions_client_device_id",
"user_sessions",
["client_device_id"],
unique=False,
)
op.create_index(
"idx_user_sessions_user_active",
"user_sessions",
["user_id", "revoked_at", "expires_at"],
unique=False,
)
op.create_index(
"idx_user_sessions_user_device",
"user_sessions",
["user_id", "client_device_id"],
unique=False,
)
def downgrade() -> None:
op.drop_index("idx_user_sessions_user_device", table_name="user_sessions")
op.drop_index("idx_user_sessions_user_active", table_name="user_sessions")
op.drop_index("ix_user_sessions_client_device_id", table_name="user_sessions")
op.drop_index("ix_user_sessions_user_id", table_name="user_sessions")
op.drop_table("user_sessions")
@@ -1,41 +0,0 @@
"""add status_snapshot column to provider_api_keys
Revision ID: c9d8e7f6a5b4
Revises: f6e7d8c9b0a1
Create Date: 2026-03-20 12:00:00.000000
"""
from __future__ import annotations
from collections.abc import Sequence
import sqlalchemy as sa
from sqlalchemy import inspect
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "c9d8e7f6a5b4"
down_revision: str | None = "f6e7d8c9b0a1"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def column_exists(table_name: str, column_name: str) -> bool:
bind = op.get_bind()
inspector = inspect(bind)
columns = [c["name"] for c in inspector.get_columns(table_name)]
return column_name in columns
def upgrade() -> None:
if not column_exists("provider_api_keys", "status_snapshot"):
op.add_column(
"provider_api_keys",
sa.Column("status_snapshot", sa.JSON(), nullable=True),
)
def downgrade() -> None:
if column_exists("provider_api_keys", "status_snapshot"):
op.drop_column("provider_api_keys", "status_snapshot")
@@ -1,85 +0,0 @@
# Aether - 数据库迁移说明
## 当前版本
- **Revision ID**: `aether_baseline`
- **创建日期**: 2025-12-06
- **状态**: 全新基线
## 迁移历史
所有历史增量迁移已清理,当前以完整 schema 作为新起点。
## 核心数据库结构
### 用户系统
- **users**: 用户账户管理
- **api_keys**: API 密钥管理
- **wallets**: 统一钱包账户(充值余额/赠款余额/无限制模式)
- **user_preferences**: 用户偏好设置
### Provider 三层架构
- **providers**: LLM 提供商配置
- **provider_endpoints**: Provider 的 API 端点配置
- **provider_api_keys**: Endpoint 的具体 API 密钥
- **api_key_provider_mappings**: 用户 API Key 到 Provider 的映射关系
### 模型系统
- **global_models**: 统一模型定义(GlobalModel)
- **models**: Provider 的模型实现和价格配置
- **model_mappings**: 统一的别名与降级映射表
### 监控和追踪
- **usage**: API 使用记录
- **request_candidates**: 请求候选记录
- **provider_usage_tracking**: Provider 使用统计
- **audit_logs**: 系统审计日志
### 系统功能
- **announcements**: 系统公告
- **announcement_reads**: 公告阅读记录
- **system_configs**: 系统配置
## 从旧数据库迁移
如需从旧数据库迁移数据,请使用迁移脚本:
```bash
# 设置环境变量
export OLD_DATABASE_URL="postgresql://user:pass@old-host:5432/old_db"
export NEW_DATABASE_URL="postgresql://user:pass@new-host:5432/aether"
# 干运行(查看迁移量)
python scripts/migrate_data.py --dry-run
# 执行迁移
python scripts/migrate_data.py
# 只迁移特定表
python scripts/migrate_data.py --tables users,providers,api_keys
# 跳过大表
python scripts/migrate_data.py --skip usage,audit_logs
```
## 新数据库初始化
```bash
# 1. 运行迁移创建表结构
DATABASE_URL="postgresql://user:pass@host:5432/aether" uv run alembic upgrade head
# 2. 初始化管理员账户
python -m src.database.init_db
```
## 未来迁移
基于 `aether_baseline` 创建增量迁移:
```bash
# 修改模型后,生成新的迁移
DATABASE_URL="..." uv run alembic revision --autogenerate -m "描述变更"
# 应用迁移
DATABASE_URL="..." uv run alembic upgrade head
```
View File
-334
View File
@@ -1,334 +0,0 @@
"""Admin API routers.
The admin surface remains Python-only host/control-plane scope. It is not part
of the compatibility frontdoor manifest that Rust is preparing to absorb.
"""
from __future__ import annotations
from fastapi import APIRouter
from starlette.routing import BaseRoute
from .adaptive import router as adaptive_router
from .api_keys import router as api_keys_router
from .billing import router as billing_router
from .endpoints import router as endpoints_router
from .models import router as models_router
from .modules import router as modules_router
from .monitoring import router as monitoring_router
from .payments import router as payments_router
from .pool import router as pool_router
from .provider_oauth import router as provider_oauth_router
from .provider_ops import router as provider_ops_router
from .provider_query import router as provider_query_router
from .provider_strategy import router as provider_strategy_router
from .providers import router as providers_router
from .security import router as security_router
from .stats import router as stats_router
from .system import router as system_router
from .usage import router as usage_router
from .users import router as users_router
from .video_tasks import router as video_tasks_router
from .wallets import router as wallets_router
_RUST_OWNED_ADMIN_ROUTE_SIGNATURES = frozenset(
{
("GET", "/api/admin/modules/status"),
("GET", "/api/admin/modules/status/{module_name}"),
("PUT", "/api/admin/modules/status/{module_name}/enabled"),
("GET", "/api/admin/system/version"),
("GET", "/api/admin/system/check-update"),
("GET", "/api/admin/system/aws-regions"),
("GET", "/api/admin/system/stats"),
("GET", "/api/admin/system/settings"),
("GET", "/api/admin/system/config/export"),
("GET", "/api/admin/system/users/export"),
("POST", "/api/admin/system/config/import"),
("POST", "/api/admin/system/users/import"),
("POST", "/api/admin/system/smtp/test"),
("POST", "/api/admin/system/cleanup"),
("POST", "/api/admin/system/purge/config"),
("POST", "/api/admin/system/purge/users"),
("POST", "/api/admin/system/purge/usage"),
("POST", "/api/admin/system/purge/audit-logs"),
("POST", "/api/admin/system/purge/request-bodies"),
("POST", "/api/admin/system/purge/stats"),
("PUT", "/api/admin/system/settings"),
("GET", "/api/admin/system/configs"),
("GET", "/api/admin/system/configs/{key}"),
("PUT", "/api/admin/system/configs/{key}"),
("DELETE", "/api/admin/system/configs/{key}"),
("GET", "/api/admin/system/api-formats"),
("GET", "/api/admin/system/email/templates"),
("GET", "/api/admin/system/email/templates/{template_type}"),
("PUT", "/api/admin/system/email/templates/{template_type}"),
("POST", "/api/admin/system/email/templates/{template_type}/preview"),
("POST", "/api/admin/system/email/templates/{template_type}/reset"),
("GET", "/api/admin/providers/"),
("POST", "/api/admin/providers/"),
("PATCH", "/api/admin/providers/{provider_id}"),
("DELETE", "/api/admin/providers/{provider_id}"),
("GET", "/api/admin/providers/summary"),
("GET", "/api/admin/providers/{provider_id}/summary"),
("GET", "/api/admin/providers/{provider_id}/health-monitor"),
("GET", "/api/admin/providers/{provider_id}/mapping-preview"),
("GET", "/api/admin/providers/{provider_id}/delete-task/{task_id}"),
("GET", "/api/admin/providers/{provider_id}/pool-status"),
("POST", "/api/admin/providers/{provider_id}/pool/clear-cooldown/{key_id}"),
("POST", "/api/admin/providers/{provider_id}/pool/reset-cost/{key_id}"),
("GET", "/api/admin/providers/{provider_id}/models"),
("POST", "/api/admin/providers/{provider_id}/models"),
("GET", "/api/admin/providers/{provider_id}/models/{model_id}"),
("PATCH", "/api/admin/providers/{provider_id}/models/{model_id}"),
("DELETE", "/api/admin/providers/{provider_id}/models/{model_id}"),
("POST", "/api/admin/providers/{provider_id}/models/batch"),
("GET", "/api/admin/providers/{provider_id}/available-source-models"),
("POST", "/api/admin/providers/{provider_id}/assign-global-models"),
("POST", "/api/admin/providers/{provider_id}/import-from-upstream"),
("GET", "/api/admin/endpoints/providers/{provider_id}/endpoints"),
("POST", "/api/admin/endpoints/providers/{provider_id}/endpoints"),
("GET", "/api/admin/endpoints/defaults/{api_format}/body-rules"),
("GET", "/api/admin/endpoints/{endpoint_id}"),
("PUT", "/api/admin/endpoints/{endpoint_id}"),
("DELETE", "/api/admin/endpoints/{endpoint_id}"),
("PUT", "/api/admin/endpoints/keys/{key_id}"),
("GET", "/api/admin/endpoints/keys/grouped-by-format"),
("GET", "/api/admin/endpoints/keys/{key_id}/reveal"),
("GET", "/api/admin/endpoints/keys/{key_id}/export"),
("DELETE", "/api/admin/endpoints/keys/{key_id}"),
("POST", "/api/admin/endpoints/keys/batch-delete"),
("POST", "/api/admin/endpoints/keys/{key_id}/clear-oauth-invalid"),
("GET", "/api/admin/endpoints/providers/{provider_id}/keys"),
("POST", "/api/admin/endpoints/providers/{provider_id}/keys"),
("POST", "/api/admin/endpoints/providers/{provider_id}/refresh-quota"),
("GET", "/api/admin/endpoints/rpm/key/{key_id}"),
("DELETE", "/api/admin/endpoints/rpm/key/{key_id}"),
("GET", "/api/admin/endpoints/health/summary"),
("GET", "/api/admin/endpoints/health/status"),
("GET", "/api/admin/endpoints/health/api-formats"),
("GET", "/api/admin/endpoints/health/key/{key_id}"),
("PATCH", "/api/admin/endpoints/health/keys/{key_id}"),
("PATCH", "/api/admin/endpoints/health/keys"),
("GET", "/api/admin/provider-oauth/supported-types"),
("POST", "/api/admin/provider-oauth/keys/{key_id}/start"),
("POST", "/api/admin/provider-oauth/keys/{key_id}/complete"),
("POST", "/api/admin/provider-oauth/keys/{key_id}/refresh"),
("POST", "/api/admin/provider-oauth/providers/{provider_id}/start"),
("POST", "/api/admin/provider-oauth/providers/{provider_id}/complete"),
("POST", "/api/admin/provider-oauth/providers/{provider_id}/import-refresh-token"),
("POST", "/api/admin/provider-oauth/providers/{provider_id}/device-authorize"),
("POST", "/api/admin/provider-oauth/providers/{provider_id}/device-poll"),
("POST", "/api/admin/provider-oauth/providers/{provider_id}/batch-import"),
("POST", "/api/admin/provider-oauth/providers/{provider_id}/batch-import/tasks"),
("GET", "/api/admin/provider-oauth/providers/{provider_id}/batch-import/tasks/{task_id}"),
("GET", "/api/admin/adaptive/keys"),
("PATCH", "/api/admin/adaptive/keys/{key_id}/mode"),
("GET", "/api/admin/adaptive/keys/{key_id}/stats"),
("DELETE", "/api/admin/adaptive/keys/{key_id}/learning"),
("PATCH", "/api/admin/adaptive/keys/{key_id}/limit"),
("GET", "/api/admin/adaptive/summary"),
("GET", "/api/admin/provider-ops/architectures"),
("GET", "/api/admin/provider-ops/architectures/{architecture_id}"),
("GET", "/api/admin/provider-ops/providers/{provider_id}/status"),
("GET", "/api/admin/provider-ops/providers/{provider_id}/config"),
("PUT", "/api/admin/provider-ops/providers/{provider_id}/config"),
("DELETE", "/api/admin/provider-ops/providers/{provider_id}/config"),
("POST", "/api/admin/provider-ops/providers/{provider_id}/connect"),
("POST", "/api/admin/provider-ops/providers/{provider_id}/disconnect"),
("POST", "/api/admin/provider-ops/providers/{provider_id}/verify"),
("POST", "/api/admin/provider-ops/providers/{provider_id}/actions/{action_type}"),
("GET", "/api/admin/provider-ops/providers/{provider_id}/balance"),
("POST", "/api/admin/provider-ops/providers/{provider_id}/balance"),
("POST", "/api/admin/provider-ops/providers/{provider_id}/checkin"),
("POST", "/api/admin/provider-ops/batch/balance"),
("GET", "/api/admin/billing/presets"),
("POST", "/api/admin/billing/presets/apply"),
("GET", "/api/admin/billing/rules"),
("GET", "/api/admin/billing/rules/{rule_id}"),
("POST", "/api/admin/billing/rules"),
("PUT", "/api/admin/billing/rules/{rule_id}"),
("GET", "/api/admin/billing/collectors"),
("GET", "/api/admin/billing/collectors/{collector_id}"),
("POST", "/api/admin/billing/collectors"),
("PUT", "/api/admin/billing/collectors/{collector_id}"),
("PUT", "/api/admin/provider-strategy/providers/{provider_id}/billing"),
("GET", "/api/admin/provider-strategy/providers/{provider_id}/stats"),
("GET", "/api/admin/provider-strategy/strategies"),
("DELETE", "/api/admin/provider-strategy/providers/{provider_id}/quota"),
("POST", "/api/admin/provider-query/models"),
("POST", "/api/admin/provider-query/test-model"),
("POST", "/api/admin/provider-query/test-model-failover"),
("GET", "/api/admin/payments/orders"),
("GET", "/api/admin/payments/orders/{order_id}"),
("POST", "/api/admin/payments/orders/{order_id}/expire"),
("POST", "/api/admin/payments/orders/{order_id}/credit"),
("POST", "/api/admin/payments/orders/{order_id}/fail"),
("GET", "/api/admin/payments/callbacks"),
("POST", "/api/admin/security/ip/blacklist"),
("DELETE", "/api/admin/security/ip/blacklist/{ip_address}"),
("GET", "/api/admin/security/ip/blacklist/stats"),
("POST", "/api/admin/security/ip/whitelist"),
("DELETE", "/api/admin/security/ip/whitelist/{ip_address}"),
("GET", "/api/admin/security/ip/whitelist"),
("GET", "/api/admin/stats/providers/quota-usage"),
("GET", "/api/admin/stats/comparison"),
("GET", "/api/admin/stats/errors/distribution"),
("GET", "/api/admin/stats/performance/percentiles"),
("GET", "/api/admin/stats/cost/forecast"),
("GET", "/api/admin/stats/cost/savings"),
("GET", "/api/admin/stats/leaderboard/api-keys"),
("GET", "/api/admin/stats/leaderboard/models"),
("GET", "/api/admin/stats/leaderboard/users"),
("GET", "/api/admin/stats/time-series"),
("GET", "/api/admin/monitoring/audit-logs"),
("GET", "/api/admin/monitoring/system-status"),
("GET", "/api/admin/monitoring/suspicious-activities"),
("GET", "/api/admin/monitoring/user-behavior/{user_id}"),
("GET", "/api/admin/monitoring/resilience-status"),
("GET", "/api/admin/monitoring/resilience/circuit-history"),
("DELETE", "/api/admin/monitoring/resilience/error-stats"),
("GET", "/api/admin/monitoring/trace/{request_id}"),
("GET", "/api/admin/monitoring/trace/stats/provider/{provider_id}"),
("GET", "/api/admin/monitoring/cache/stats"),
("GET", "/api/admin/monitoring/cache/affinity/{user_identifier}"),
("GET", "/api/admin/monitoring/cache/affinities"),
("DELETE", "/api/admin/monitoring/cache/users/{user_identifier}"),
(
"DELETE",
"/api/admin/monitoring/cache/affinity/{affinity_key}/{endpoint_id}/{model_id}/{api_format}",
),
("DELETE", "/api/admin/monitoring/cache"),
("DELETE", "/api/admin/monitoring/cache/providers/{provider_id}"),
("GET", "/api/admin/monitoring/cache/config"),
("GET", "/api/admin/monitoring/cache/metrics"),
("GET", "/api/admin/monitoring/cache/model-mapping/stats"),
("DELETE", "/api/admin/monitoring/cache/model-mapping"),
("DELETE", "/api/admin/monitoring/cache/model-mapping/{model_name}"),
(
"DELETE",
"/api/admin/monitoring/cache/model-mapping/provider/{provider_id}/{global_model_id}",
),
("GET", "/api/admin/monitoring/cache/redis-keys"),
("DELETE", "/api/admin/monitoring/cache/redis-keys/{category}"),
("GET", "/api/admin/usage/aggregation/stats"),
("GET", "/api/admin/usage/stats"),
("GET", "/api/admin/usage/heatmap"),
("GET", "/api/admin/usage/records"),
("GET", "/api/admin/usage/active"),
("GET", "/api/admin/usage/cache-affinity/hit-analysis"),
("GET", "/api/admin/usage/cache-affinity/interval-timeline"),
("GET", "/api/admin/usage/cache-affinity/ttl-analysis"),
("GET", "/api/admin/usage/{usage_id}/curl"),
("GET", "/api/admin/usage/{usage_id}"),
("POST", "/api/admin/usage/{usage_id}/replay"),
("GET", "/api/admin/video-tasks"),
("GET", "/api/admin/video-tasks/stats"),
("GET", "/api/admin/video-tasks/{task_id}"),
("POST", "/api/admin/video-tasks/{task_id}/cancel"),
("GET", "/api/admin/video-tasks/{task_id}/video"),
("GET", "/api/admin/wallets"),
("GET", "/api/admin/wallets/ledger"),
("GET", "/api/admin/wallets/refund-requests"),
("GET", "/api/admin/wallets/{wallet_id}"),
("GET", "/api/admin/wallets/{wallet_id}/transactions"),
("GET", "/api/admin/wallets/{wallet_id}/refunds"),
("POST", "/api/admin/wallets/{wallet_id}/adjust"),
("POST", "/api/admin/wallets/{wallet_id}/recharge"),
("POST", "/api/admin/wallets/{wallet_id}/refunds/{refund_id}/process"),
("POST", "/api/admin/wallets/{wallet_id}/refunds/{refund_id}/complete"),
("POST", "/api/admin/wallets/{wallet_id}/refunds/{refund_id}/fail"),
("GET", "/api/admin/api-keys"),
("POST", "/api/admin/api-keys"),
("GET", "/api/admin/api-keys/{key_id}"),
("PUT", "/api/admin/api-keys/{key_id}"),
("PATCH", "/api/admin/api-keys/{key_id}"),
("DELETE", "/api/admin/api-keys/{key_id}"),
("GET", "/api/admin/users"),
("POST", "/api/admin/users"),
("GET", "/api/admin/users/{user_id}"),
("PUT", "/api/admin/users/{user_id}"),
("DELETE", "/api/admin/users/{user_id}"),
("GET", "/api/admin/users/{user_id}/sessions"),
("DELETE", "/api/admin/users/{user_id}/sessions"),
("DELETE", "/api/admin/users/{user_id}/sessions/{session_id}"),
("GET", "/api/admin/users/{user_id}/api-keys"),
("POST", "/api/admin/users/{user_id}/api-keys"),
("DELETE", "/api/admin/users/{user_id}/api-keys/{key_id}"),
("PUT", "/api/admin/users/{user_id}/api-keys/{key_id}"),
("PATCH", "/api/admin/users/{user_id}/api-keys/{key_id}/lock"),
("GET", "/api/admin/users/{user_id}/api-keys/{key_id}/full-key"),
("GET", "/api/admin/pool/overview"),
("GET", "/api/admin/pool/scheduling-presets"),
("GET", "/api/admin/pool/{provider_id}/keys"),
("GET", "/api/admin/pool/{provider_id}/keys/batch-delete-task/{task_id}"),
("POST", "/api/admin/pool/{provider_id}/keys/batch-action"),
("POST", "/api/admin/pool/{provider_id}/keys/batch-import"),
("POST", "/api/admin/pool/{provider_id}/keys/cleanup-banned"),
("POST", "/api/admin/pool/{provider_id}/keys/resolve-selection"),
("GET", "/api/admin/proxy-nodes"),
("GET", "/api/admin/models/catalog"),
("GET", "/api/admin/models/external"),
("DELETE", "/api/admin/models/external/cache"),
("GET", "/api/admin/models/global"),
("POST", "/api/admin/models/global"),
("GET", "/api/admin/models/global/{global_model_id}"),
("PATCH", "/api/admin/models/global/{global_model_id}"),
("DELETE", "/api/admin/models/global/{global_model_id}"),
("POST", "/api/admin/models/global/batch-delete"),
("POST", "/api/admin/models/global/{global_model_id}/assign-to-providers"),
("GET", "/api/admin/models/global/{global_model_id}/providers"),
("GET", "/api/admin/models/global/{global_model_id}/routing"),
}
)
def _route_is_rust_owned(route: BaseRoute) -> bool:
path = getattr(route, "path", None)
methods = getattr(route, "methods", None)
if not isinstance(path, str) or not methods:
return False
return any(
(method, path) in _RUST_OWNED_ADMIN_ROUTE_SIGNATURES
for method in methods
if method not in {"HEAD", "OPTIONS"}
)
def _build_python_admin_router() -> APIRouter:
"""Admin/control-plane routes that still require the Python host."""
admin_router = APIRouter()
admin_router.include_router(system_router)
admin_router.include_router(users_router)
admin_router.include_router(providers_router)
admin_router.include_router(api_keys_router)
admin_router.include_router(billing_router)
admin_router.include_router(usage_router)
admin_router.include_router(monitoring_router)
admin_router.include_router(payments_router)
admin_router.include_router(endpoints_router)
admin_router.include_router(provider_strategy_router)
admin_router.include_router(provider_oauth_router)
admin_router.include_router(adaptive_router)
admin_router.include_router(models_router)
admin_router.include_router(security_router)
admin_router.include_router(stats_router)
admin_router.include_router(provider_query_router)
admin_router.include_router(modules_router)
admin_router.include_router(pool_router)
admin_router.include_router(provider_ops_router)
admin_router.include_router(video_tasks_router)
admin_router.include_router(wallets_router)
admin_router.routes = [route for route in admin_router.routes if not _route_is_rust_owned(route)]
return admin_router
# Admin/control-plane 在本轮 frontdoor cutover 后仍保留在 Python 宿主。
python_admin_router = _build_python_admin_router()
router = python_admin_router
# 注意:以下路由已迁移到模块系统,由 ModuleRegistry 动态注册
# - ldap_router: 当 LDAP_AVAILABLE=true 时注册
# - management_tokens_router: 当 MANAGEMENT_TOKENS_AVAILABLE=true 时注册
# - proxy_nodes_router: 当 PROXY_NODES_AVAILABLE=true 时注册
__all__ = ["python_admin_router", "router"]
-411
View File
@@ -1,411 +0,0 @@
"""
自适应 RPM 管理 API 端点
设计原则:
- 自适应模式由 rpm_limit 字段决定:
- rpm_limit = NULL:启用自适应模式,系统自动学习并调整 RPM 限制
- rpm_limit = 数字:固定限制模式,使用用户指定的 RPM 限制
- learned_rpm_limit:自适应模式下学习到的 RPM 限制值
- adaptive_mode 是计算字段,基于 rpm_limit 是否为 NULL
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Any
from fastapi import APIRouter, Depends, HTTPException, Query, Request
from pydantic import BaseModel, Field, ValidationError
from sqlalchemy import func
from sqlalchemy.orm import Session, load_only
from src.api.base.admin_adapter import AdminApiAdapter
from src.api.base.context import ApiRequestContext
from src.api.base.pipeline import get_pipeline
from src.core.exceptions import InvalidRequestException, translate_pydantic_error
from src.database import get_db
from src.models.database import ProviderAPIKey
from src.services.rate_limit.adaptive_rpm import get_adaptive_rpm_manager
router = APIRouter(prefix="/api/admin/adaptive", tags=["Adaptive RPM"])
pipeline = get_pipeline()
# ==================== Pydantic Models ====================
class EnableAdaptiveRequest(BaseModel):
"""启用自适应模式请求"""
enabled: bool = Field(..., description="是否启用自适应模式(true=自适应,false=固定限制)")
fixed_limit: int | None = Field(
None, ge=1, le=100, description="固定 RPM 限制(仅当 enabled=false 时生效,1-100)"
)
class AdaptiveStatsResponse(BaseModel):
"""自适应统计响应"""
adaptive_mode: bool = Field(..., description="是否为自适应模式(rpm_limit=NULL)")
rpm_limit: int | None = Field(None, description="用户配置的固定限制(NULL=自适应)")
effective_limit: int | None = Field(
None, description="当前有效限制(自适应使用学习值,固定使用配置值)"
)
learned_limit: int | None = Field(None, description="学习到的 RPM 限制")
concurrent_429_count: int
rpm_429_count: int
last_429_at: str | None
last_429_type: str | None
adjustment_count: int
recent_adjustments: list[dict]
# 置信度相关
learning_confidence: float | None = Field(None, description="学习置信度 (0.0-1.0)")
enforcement_active: bool | None = Field(None, description="是否正在执行本地 RPM 限制")
observation_count: int = Field(0, description="429 观察记录数")
header_observation_count: int = Field(0, description="带 header 的观察记录数")
latest_upstream_limit: int | None = Field(None, description="最近一次上游 header 限制值")
class KeyListItem(BaseModel):
"""Key 列表项"""
id: str
name: str | None
provider_id: str
api_formats: list[str] = Field(default_factory=list)
is_adaptive: bool = Field(..., description="是否为自适应模式(rpm_limit=NULL)")
rpm_limit: int | None = Field(None, description="固定 RPM 限制(NULL=自适应)")
effective_limit: int | None = Field(None, description="当前有效限制")
learned_rpm_limit: int | None = Field(None, description="学习到的 RPM 限制")
concurrent_429_count: int
rpm_429_count: int
# ==================== API Endpoints ====================
@router.get(
"/keys",
response_model=list[KeyListItem],
summary="获取所有启用自适应模式的Key",
)
async def list_adaptive_keys(
request: Request,
provider_id: str | None = Query(None, description="按 Provider 过滤"),
db: Session = Depends(get_db),
) -> Any:
"""
获取所有启用自适应模式的Key列表
可选参数:
- provider_id: 按 Provider 过滤
"""
adapter = ListAdaptiveKeysAdapter(provider_id=provider_id)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.patch(
"/keys/{key_id}/mode",
summary="Toggle key's RPM control mode",
)
async def toggle_adaptive_mode(
key_id: str,
request: Request,
db: Session = Depends(get_db),
) -> Any:
"""
Toggle the RPM control mode for a specific key
Parameters:
- enabled: true=adaptive mode (rpm_limit=NULL), false=fixed limit mode
- fixed_limit: fixed limit value (required when enabled=false)
"""
adapter = ToggleAdaptiveModeAdapter(key_id=key_id)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.get(
"/keys/{key_id}/stats",
response_model=AdaptiveStatsResponse,
summary="获取Key的自适应统计",
)
async def get_adaptive_stats(
key_id: str,
request: Request,
db: Session = Depends(get_db),
) -> Any:
"""
获取指定Key的自适应 RPM 统计信息
包括:
- 当前配置
- 学习到的限制
- 429错误统计
- 调整历史
"""
adapter = GetAdaptiveStatsAdapter(key_id=key_id)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.delete(
"/keys/{key_id}/learning",
summary="Reset key's learning state",
)
async def reset_adaptive_learning(
key_id: str,
request: Request,
db: Session = Depends(get_db),
) -> Any:
"""
Reset the adaptive learning state for a specific key
Clears:
- Learned RPM limit (learned_rpm_limit)
- 429 error counts
- Adjustment history
Does not change:
- rpm_limit config (determines adaptive mode)
"""
adapter = ResetAdaptiveLearningAdapter(key_id=key_id)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.patch(
"/keys/{key_id}/limit",
summary="Set key to fixed RPM limit mode",
)
async def set_rpm_limit(
key_id: str,
request: Request,
limit: int = Query(..., ge=1, le=100, description="RPM limit value (1-100)"),
db: Session = Depends(get_db),
) -> Any:
"""
Set key to fixed RPM limit mode
Note:
- After setting this value, key switches to fixed limit mode and won't auto-adjust
- To restore adaptive mode, use PATCH /keys/{key_id}/mode
"""
adapter = SetRPMLimitAdapter(key_id=key_id, limit=limit)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.get(
"/summary",
summary="获取自适应 RPM 的全局统计",
)
async def get_adaptive_summary(
request: Request,
db: Session = Depends(get_db),
) -> Any:
"""
获取自适应 RPM 的全局统计摘要
包括:
- 启用自适应模式的Key数量
- 总429错误数
- RPM 限制调整次数
"""
adapter = AdaptiveSummaryAdapter()
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
# ==================== Pipeline 适配器 ====================
@dataclass
class ListAdaptiveKeysAdapter(AdminApiAdapter):
provider_id: str | None = None
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
# 自适应模式:rpm_limit = NULL
query = context.db.query(ProviderAPIKey).filter(ProviderAPIKey.rpm_limit.is_(None))
if self.provider_id:
query = query.filter(ProviderAPIKey.provider_id == self.provider_id)
keys = query.all()
adaptive_manager = get_adaptive_rpm_manager()
return [
KeyListItem(
id=key.id,
name=key.name,
provider_id=key.provider_id,
api_formats=key.api_formats or [],
is_adaptive=key.rpm_limit is None,
rpm_limit=key.rpm_limit,
effective_limit=adaptive_manager.get_effective_limit(key),
learned_rpm_limit=key.learned_rpm_limit,
concurrent_429_count=key.concurrent_429_count or 0,
rpm_429_count=key.rpm_429_count or 0,
)
for key in keys
]
@dataclass
class ToggleAdaptiveModeAdapter(AdminApiAdapter):
key_id: str
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
key = context.db.query(ProviderAPIKey).filter(ProviderAPIKey.id == self.key_id).first()
if not key:
raise HTTPException(status_code=404, detail="Key not found")
payload = context.ensure_json_body()
try:
body = EnableAdaptiveRequest.model_validate(payload)
except ValidationError as e:
errors = e.errors()
if errors:
raise InvalidRequestException(translate_pydantic_error(errors[0]))
raise InvalidRequestException("请求数据验证失败")
if body.enabled:
# 启用自适应模式:将 rpm_limit 设为 NULL
key.rpm_limit = None
message = "已切换为自适应模式,系统将自动学习并调整 RPM 限制"
else:
# 禁用自适应模式:设置固定限制
if body.fixed_limit is None:
raise HTTPException(
status_code=400, detail="禁用自适应模式时必须提供 fixed_limit 参数"
)
key.rpm_limit = body.fixed_limit
message = f"已切换为固定限制模式,RPM 限制设为 {body.fixed_limit}"
context.db.commit()
context.db.refresh(key)
is_adaptive = key.rpm_limit is None
adaptive_manager = get_adaptive_rpm_manager()
return {
"message": message,
"key_id": key.id,
"is_adaptive": is_adaptive,
"rpm_limit": key.rpm_limit,
"effective_limit": adaptive_manager.get_effective_limit(key),
}
@dataclass
class GetAdaptiveStatsAdapter(AdminApiAdapter):
key_id: str
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
key = context.db.query(ProviderAPIKey).filter(ProviderAPIKey.id == self.key_id).first()
if not key:
raise HTTPException(status_code=404, detail="Key not found")
adaptive_manager = get_adaptive_rpm_manager()
stats = adaptive_manager.get_adjustment_stats(key)
# 转换字段名以匹配响应模型
return AdaptiveStatsResponse(
adaptive_mode=stats["adaptive_mode"],
rpm_limit=stats["rpm_limit"],
effective_limit=stats["effective_limit"],
learned_limit=stats["learned_limit"],
concurrent_429_count=stats["concurrent_429_count"],
rpm_429_count=stats["rpm_429_count"],
last_429_at=stats["last_429_at"],
last_429_type=stats["last_429_type"],
adjustment_count=stats["adjustment_count"],
recent_adjustments=stats["recent_adjustments"],
learning_confidence=stats.get("learning_confidence"),
enforcement_active=stats.get("enforcement_active"),
observation_count=stats.get("observation_count", 0),
header_observation_count=stats.get("header_observation_count", 0),
latest_upstream_limit=stats.get("latest_upstream_limit"),
)
@dataclass
class ResetAdaptiveLearningAdapter(AdminApiAdapter):
key_id: str
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
key = context.db.query(ProviderAPIKey).filter(ProviderAPIKey.id == self.key_id).first()
if not key:
raise HTTPException(status_code=404, detail="Key not found")
adaptive_manager = get_adaptive_rpm_manager()
adaptive_manager.reset_learning(context.db, key)
return {"message": "学习状态已重置", "key_id": key.id}
@dataclass
class SetRPMLimitAdapter(AdminApiAdapter):
key_id: str
limit: int
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
key = context.db.query(ProviderAPIKey).filter(ProviderAPIKey.id == self.key_id).first()
if not key:
raise HTTPException(status_code=404, detail="Key not found")
was_adaptive = key.rpm_limit is None
key.rpm_limit = self.limit
context.db.commit()
context.db.refresh(key)
return {
"message": f"已设置为固定限制模式,RPM 限制为 {self.limit}",
"key_id": key.id,
"is_adaptive": False,
"rpm_limit": key.rpm_limit,
"previous_mode": "adaptive" if was_adaptive else "fixed",
}
class AdaptiveSummaryAdapter(AdminApiAdapter):
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
db = context.db
is_adaptive = ProviderAPIKey.rpm_limit.is_(None)
# SQL 聚合获取 count / sum,避免全表 ORM 加载
total_keys, total_concurrent_429, total_rpm_429 = (
db.query(
func.count(ProviderAPIKey.id),
func.coalesce(func.sum(ProviderAPIKey.concurrent_429_count), 0),
func.coalesce(func.sum(ProviderAPIKey.rpm_429_count), 0),
)
.filter(is_adaptive)
.one()
)
# adjustment_history 是 JSON 列,长度只能在 Python 侧统计;
# 只加载有历史记录的 key 的必要列
keys_with_history = (
db.query(ProviderAPIKey)
.options(
load_only(ProviderAPIKey.id, ProviderAPIKey.name, ProviderAPIKey.adjustment_history)
)
.filter(is_adaptive, ProviderAPIKey.adjustment_history.isnot(None))
.all()
)
total_adjustments = sum(len(key.adjustment_history or []) for key in keys_with_history)
recent_adjustments = []
for key in keys_with_history:
if key.adjustment_history:
for adj in key.adjustment_history[-3:]:
recent_adjustments.append(
{
"key_id": key.id,
"key_name": key.name,
**adj,
}
)
recent_adjustments.sort(key=lambda item: item.get("timestamp", ""), reverse=True)
return {
"total_adaptive_keys": total_keys,
"total_concurrent_429_errors": int(total_concurrent_429),
"total_rpm_429_errors": int(total_rpm_429),
"total_adjustments": total_adjustments,
"recent_adjustments": recent_adjustments[:10],
}
@@ -1,5 +0,0 @@
"""API key admin routes export."""
from .routes import router
__all__ = ["router"]
@@ -1,726 +0,0 @@
"""管理员独立余额 API Key 管理路由。
独立余额Key:不关联用户配额,可配置独立余额限制或无限额度,用于给非注册用户使用。
"""
from __future__ import annotations
from datetime import datetime, timedelta, timezone
from typing import Any, Literal
from zoneinfo import ZoneInfo
from fastapi import APIRouter, Depends, HTTPException, Query, Request
from fastapi.concurrency import run_in_threadpool
from sqlalchemy import func
from sqlalchemy.orm import Session
from src.api.base.admin_adapter import AdminApiAdapter
from src.api.base.context import ApiRequestContext
from src.api.base.pipeline import get_pipeline
from src.config import config
from src.core.exceptions import InvalidRequestException, NotFoundException
from src.core.logger import logger
from src.database import get_db, get_db_context
from src.models.api import CreateApiKeyRequest
from src.models.database import ApiKey, Usage, Wallet
from src.services.user.apikey import ApiKeyService
from src.services.user.bulk_cleanup import pre_clean_api_key
from src.services.wallet import WalletService
APP_TIMEZONE = ZoneInfo(config.app_timezone)
def parse_expiry_date(date_str: str | None) -> datetime | None:
"""解析过期日期字符串为 datetime 对象。
Args:
date_str: 日期字符串,支持 "YYYY-MM-DD" 或 ISO 格式
Returns:
datetime 对象(当天 23:59:59.999999,应用时区),或 None 如果输入为空
Raises:
BadRequestException: 日期格式无效
"""
if not date_str or not date_str.strip():
return None
date_str = date_str.strip()
# 尝试 YYYY-MM-DD 格式
try:
parsed_date = datetime.strptime(date_str, "%Y-%m-%d")
# 设置为当天结束时间 (23:59:59.999999,应用时区)
return parsed_date.replace(
hour=23, minute=59, second=59, microsecond=999999, tzinfo=APP_TIMEZONE
)
except ValueError:
pass
# 尝试完整 ISO 格式
try:
return datetime.fromisoformat(date_str.replace("Z", "+00:00"))
except ValueError:
pass
raise InvalidRequestException(f"无效的日期格式: {date_str},请使用 YYYY-MM-DD 格式")
router = APIRouter(prefix="/api/admin/api-keys", tags=["Admin - API Keys (Standalone)"])
pipeline = get_pipeline()
def _serialize_standalone_key_item(
api_key: ApiKey, *, total_tokens: int | None = None
) -> dict[str, Any]:
return {
"id": api_key.id,
"user_id": api_key.user_id,
"name": api_key.name,
"key_display": api_key.get_display_key(),
"is_active": api_key.is_active,
"is_standalone": api_key.is_standalone,
"total_requests": api_key.total_requests,
"total_tokens": int(total_tokens or 0),
"total_cost_usd": float(api_key.total_cost_usd or 0),
"rate_limit": api_key.rate_limit,
"allowed_providers": api_key.allowed_providers,
"allowed_api_formats": api_key.allowed_api_formats,
"allowed_models": api_key.allowed_models,
"last_used_at": api_key.last_used_at.isoformat() if api_key.last_used_at else None,
"expires_at": api_key.expires_at.isoformat() if api_key.expires_at else None,
"created_at": api_key.created_at.isoformat(),
"updated_at": api_key.updated_at.isoformat() if api_key.updated_at else None,
"auto_delete_on_expiry": api_key.auto_delete_on_expiry,
}
def _list_standalone_api_keys_sync(
skip: int,
limit: int,
is_active: bool | None,
) -> dict[str, Any]:
with get_db_context() as db:
query = db.query(ApiKey).filter(ApiKey.is_standalone == True)
if is_active is not None:
query = query.filter(ApiKey.is_active == is_active)
total = int(query.with_entities(func.count(ApiKey.id)).scalar() or 0)
api_keys = query.order_by(ApiKey.created_at.desc()).offset(skip).limit(limit).all()
wallet_initialized = False
for api_key in api_keys:
wallet = WalletService.get_wallet(db, api_key_id=api_key.id)
if wallet is None:
_ensure_standalone_wallet(db, api_key)
wallet_initialized = True
if wallet_initialized:
db.commit()
for api_key in api_keys:
db.refresh(api_key)
token_map: dict[str, int] = {}
if api_keys:
stats_rows = (
db.query(
Usage.api_key_id,
func.sum(Usage.total_tokens).label("total_tokens"),
)
.filter(Usage.api_key_id.in_([api_key.id for api_key in api_keys]))
.group_by(Usage.api_key_id)
.all()
)
token_map = {row.api_key_id: int(row.total_tokens or 0) for row in stats_rows}
return {
"api_keys": [
_serialize_standalone_key_item(
api_key,
total_tokens=token_map.get(api_key.id, 0),
)
for api_key in api_keys
],
"total": total,
"limit": limit,
"skip": skip,
}
def _create_standalone_api_key_sync(
admin_user_id: str,
key_data: CreateApiKeyRequest,
) -> tuple[dict[str, Any], dict[str, Any]]:
with get_db_context() as db:
if key_data.initial_balance_usd is not None and key_data.initial_balance_usd <= 0:
raise HTTPException(
status_code=400,
detail="创建独立余额Key时,初始余额必须大于 0(或设置为 null 表示无限制)",
)
expires_at_dt = parse_expiry_date(key_data.expires_at)
api_key, plain_key = ApiKeyService.create_api_key(
db=db,
user_id=admin_user_id,
name=key_data.name,
allowed_providers=key_data.allowed_providers,
allowed_api_formats=key_data.allowed_api_formats,
allowed_models=key_data.allowed_models,
rate_limit=key_data.rate_limit,
expire_days=key_data.expire_days,
expires_at=expires_at_dt,
is_standalone=True,
auto_delete_on_expiry=key_data.auto_delete_on_expiry,
)
wallet = WalletService.initialize_api_key_wallet(
db,
api_key=api_key,
initial_balance_usd=key_data.initial_balance_usd,
unlimited=key_data.initial_balance_usd is None,
operator_id=admin_user_id,
description="独立密钥初始调账",
)
if wallet is None:
raise InvalidRequestException("独立密钥钱包初始化失败")
db.commit()
db.refresh(api_key)
wallet_summary = WalletService.serialize_wallet_summary(wallet)
return (
{
"id": api_key.id,
"key": plain_key,
"name": api_key.name,
"key_display": api_key.get_display_key(),
"is_standalone": True,
"rate_limit": api_key.rate_limit,
"expires_at": api_key.expires_at.isoformat() if api_key.expires_at else None,
"created_at": api_key.created_at.isoformat(),
"wallet": wallet_summary,
"message": "独立余额Key创建成功,请妥善保存完整密钥,后续将无法查看",
},
{
"action": "create_standalone_api_key",
"key_id": api_key.id,
"initial_balance_usd": key_data.initial_balance_usd,
},
)
def _update_standalone_api_key_sync(
key_id: str,
key_data: CreateApiKeyRequest,
) -> tuple[dict[str, Any], list[str]]:
with get_db_context() as db:
api_key = db.query(ApiKey).filter(ApiKey.id == key_id).first()
if not api_key:
raise NotFoundException("API密钥不存在", "api_key")
if not api_key.is_standalone:
raise InvalidRequestException("仅支持更新独立密钥")
update_data: dict[str, Any] = {}
if key_data.name is not None:
update_data["name"] = key_data.name
if "rate_limit" in key_data.model_fields_set:
update_data["rate_limit"] = key_data.rate_limit
if (
hasattr(key_data, "auto_delete_on_expiry")
and key_data.auto_delete_on_expiry is not None
):
update_data["auto_delete_on_expiry"] = key_data.auto_delete_on_expiry
if hasattr(key_data, "allowed_providers"):
update_data["allowed_providers"] = key_data.allowed_providers
if hasattr(key_data, "allowed_api_formats"):
update_data["allowed_api_formats"] = key_data.allowed_api_formats
if hasattr(key_data, "allowed_models"):
update_data["allowed_models"] = key_data.allowed_models
if key_data.expires_at and key_data.expires_at.strip():
update_data["expires_at"] = parse_expiry_date(key_data.expires_at)
elif "expires_at" in key_data.model_fields_set:
update_data["expires_at"] = None
elif "expire_days" in key_data.model_fields_set:
if key_data.expire_days is not None and key_data.expire_days > 0:
update_data["expires_at"] = datetime.now(timezone.utc) + timedelta(
days=key_data.expire_days
)
else:
update_data["expires_at"] = None
changed_fields = list(update_data.keys())
if "initial_balance_usd" in key_data.model_fields_set:
raise InvalidRequestException("编辑独立密钥不支持修改余额,请使用钱包操作")
if (
"unlimited_balance" in key_data.model_fields_set
and key_data.unlimited_balance is not None
):
wallet = _ensure_standalone_wallet(db, api_key)
desired_mode: Literal["finite", "unlimited"] = (
"unlimited" if key_data.unlimited_balance else "finite"
)
if wallet.limit_mode != desired_mode:
WalletService.set_wallet_limit_mode(db, wallet=wallet, limit_mode=desired_mode)
changed_fields.append("unlimited_balance")
updated_key = ApiKeyService.update_api_key(db, key_id, **update_data)
if not updated_key:
raise NotFoundException("更新失败", "api_key")
wallet = _ensure_standalone_wallet(db, updated_key)
wallet_summary = WalletService.serialize_wallet_summary(wallet)
return (
{
"id": updated_key.id,
"name": updated_key.name,
"key_display": updated_key.get_display_key(),
"is_active": updated_key.is_active,
"rate_limit": updated_key.rate_limit,
"expires_at": (
updated_key.expires_at.isoformat() if updated_key.expires_at else None
),
"updated_at": (
updated_key.updated_at.isoformat() if updated_key.updated_at else None
),
"wallet": wallet_summary,
"message": "API密钥已更新",
},
changed_fields,
)
def _toggle_standalone_api_key_sync(key_id: str) -> tuple[dict[str, Any], dict[str, Any]]:
with get_db_context() as db:
api_key = db.query(ApiKey).filter(ApiKey.id == key_id).first()
if not api_key:
raise NotFoundException("API密钥不存在", "api_key")
if not api_key.is_standalone:
raise InvalidRequestException("仅支持操作独立密钥")
api_key.is_active = not api_key.is_active
api_key.updated_at = datetime.now(timezone.utc)
db.commit()
db.refresh(api_key)
return (
{
"id": api_key.id,
"is_active": api_key.is_active,
"message": f"API密钥已{'启用' if api_key.is_active else '禁用'}",
},
{
"action": "toggle_api_key",
"target_key_id": api_key.id,
"user_id": api_key.user_id,
"new_status": "enabled" if api_key.is_active else "disabled",
},
)
def _delete_standalone_api_key_sync(
key_id: str,
) -> tuple[dict[str, str], dict[str, Any], str | None]:
with get_db_context() as db:
api_key = db.query(ApiKey).filter(ApiKey.id == key_id).first()
if not api_key:
raise HTTPException(status_code=404, detail="API密钥不存在")
if not api_key.is_standalone:
raise InvalidRequestException("仅支持删除独立密钥")
user = api_key.user
pre_clean_api_key(db, api_key.id)
db.delete(api_key)
return (
{"message": "API密钥已删除"},
{
"action": "delete_api_key",
"target_key_id": key_id,
"user_id": user.id if user else None,
"user_email": user.email if user else None,
},
user.email if user else None,
)
def _ensure_standalone_wallet(
db: Session,
api_key: ApiKey,
*,
limit_mode: Literal["finite", "unlimited"] | None = None,
) -> Wallet:
"""确保独立 Key 已绑定钱包,并可选同步额度模式。"""
wallet = WalletService.get_or_create_wallet(db, api_key=api_key)
if wallet is None:
raise InvalidRequestException("独立密钥钱包初始化失败")
if limit_mode is not None and wallet.limit_mode != limit_mode:
wallet = WalletService.set_wallet_limit_mode(db, wallet=wallet, limit_mode=limit_mode)
return wallet
@router.get("")
async def list_standalone_api_keys(
request: Request,
skip: int = Query(0, ge=0),
limit: int = Query(100, ge=1, le=500),
is_active: bool | None = None,
db: Session = Depends(get_db),
) -> Any:
"""
列出所有独立余额 API Keys
获取系统中所有独立余额 API Key 的列表。独立余额 Key 不关联用户配额,
有独立的余额限制,主要用于给非注册用户使用。
**查询参数**:
- `skip`: 跳过的记录数(分页偏移量),默认 0
- `limit`: 返回的记录数(分页限制),默认 100,最大 500
- `is_active`: 可选,根据启用状态筛选(true/false)
**返回字段**:
- `api_keys`: API Key 列表,包含 id, name, key_display, is_active, is_standalone,
total_requests, total_cost_usd, rate_limit, allowed_providers, allowed_api_formats,
allowed_models, last_used_at, expires_at, created_at, updated_at, auto_delete_on_expiry 等字段
- `total`: 符合条件的总记录数
- `limit`: 当前分页限制
- `skip`: 当前分页偏移量
"""
adapter = AdminListStandaloneKeysAdapter(skip=skip, limit=limit, is_active=is_active)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.post("")
async def create_standalone_api_key(
request: Request,
key_data: CreateApiKeyRequest,
db: Session = Depends(get_db),
) -> Any:
"""
创建独立余额 API Key
创建一个新的独立余额 API Key。独立余额 Key 可设置初始余额,或使用无限额度。
**请求体字段**:
- `name`: API Key 的名称
- `initial_balance_usd`: 可选,初始余额(美元),null 表示无限制额度
- `allowed_providers`: 可选,允许使用的提供商列表
- `allowed_api_formats`: 可选,允许使用的 API 格式列表
- `allowed_models`: 可选,允许使用的模型列表
- `rate_limit`: 可选,每分钟请求限制(null 表示跟随系统默认,0 表示不限制)
- `expire_days`: 可选,过期天数(与 expires_at 二选一)
- `expires_at`: 可选,过期时间(ISO 格式或 YYYY-MM-DD 格式,优先级高于 expire_days)
- `auto_delete_on_expiry`: 可选,过期后是否自动删除
**返回字段**:
- `id`: API Key ID
- `key`: 完整的 API Key(仅在创建时返回一次)
- `name`: API Key 名称
- `key_display`: 脱敏显示的 Key
- `is_standalone`: 是否为独立余额 Key(始终为 true)
- `wallet`: 钱包摘要(总余额、充值余额、赠款余额、额度模式等)
- `rate_limit`: 速率限制配置
- `expires_at`: 过期时间
- `created_at`: 创建时间
- `message`: 提示信息
"""
adapter = AdminCreateStandaloneKeyAdapter(key_data=key_data)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.put("/{key_id}")
async def update_api_key(
key_id: str, request: Request, key_data: CreateApiKeyRequest, db: Session = Depends(get_db)
) -> Any:
"""
更新独立余额 API Key
更新指定 ID 的独立余额 API Key 的配置信息。
**路径参数**:
- `key_id`: API Key ID
**请求体字段**:
- `name`: 可选,API Key 的名称
- `unlimited_balance`: 可选,是否无限余额(true=无限,false=有限,不修改余额数值)
- `rate_limit`: 可选,每分钟请求限制(null 表示跟随系统默认,0 表示不限制)
- `allowed_providers`: 可选,允许使用的提供商列表
- `allowed_api_formats`: 可选,允许使用的 API 格式列表
- `allowed_models`: 可选,允许使用的模型列表
- `expire_days`: 可选,过期天数(与 expires_at 二选一)
- `expires_at`: 可选,过期时间(ISO 格式或 YYYY-MM-DD 格式,优先级高于 expire_days,null 或空字符串表示永不过期)
- `auto_delete_on_expiry`: 可选,过期后是否自动删除
**返回字段**:
- `id`: API Key ID
- `name`: API Key 名称
- `key_display`: 脱敏显示的 Key
- `is_active`: 是否启用
- `wallet`: 钱包摘要(总余额、充值余额、赠款余额、额度模式等)
- `rate_limit`: 速率限制配置
- `expires_at`: 过期时间
- `updated_at`: 更新时间
- `message`: 提示信息
"""
adapter = AdminUpdateApiKeyAdapter(key_id=key_id, key_data=key_data)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.patch("/{key_id}")
async def toggle_api_key(key_id: str, request: Request, db: Session = Depends(get_db)) -> Any:
"""
切换 API Key 启用状态
切换指定 API Key 的启用/禁用状态。
**路径参数**:
- `key_id`: API Key ID
**返回字段**:
- `id`: API Key ID
- `is_active`: 新的启用状态
- `message`: 提示信息
"""
adapter = AdminToggleApiKeyAdapter(key_id=key_id)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.delete("/{key_id}")
async def delete_api_key(key_id: str, request: Request, db: Session = Depends(get_db)) -> None:
"""
删除 API Key
删除指定的 API Key。此操作不可逆。
**路径参数**:
- `key_id`: API Key ID
**返回字段**:
- `message`: 提示信息
"""
adapter = AdminDeleteApiKeyAdapter(key_id=key_id)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.get("/{key_id}")
async def get_api_key_detail(
key_id: str,
request: Request,
include_key: bool = Query(False, description="Include full decrypted key in response"),
db: Session = Depends(get_db),
) -> Any:
"""
获取 API Key 详情
获取指定 API Key 的详细信息。可选择是否返回完整的解密密钥。
**路径参数**:
- `key_id`: API Key ID
**查询参数**:
- `include_key`: 是否包含完整的解密密钥,默认 false
**返回字段**:
- 当 include_key=false 时,返回基本信息:id, user_id, name, key_display, is_active,
is_standalone, total_requests, total_cost_usd, rate_limit, allowed_providers,
allowed_api_formats, allowed_models, last_used_at, expires_at, created_at, updated_at,
wallet
- 当 include_key=true 时,返回完整密钥:key
"""
if include_key:
adapter = AdminGetFullKeyAdapter(key_id=key_id)
else:
# Return basic key info without full key
adapter = AdminGetKeyDetailAdapter(key_id=key_id)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
class AdminListStandaloneKeysAdapter(AdminApiAdapter):
"""列出独立余额Keys"""
def __init__(
self,
skip: int,
limit: int,
is_active: bool | None,
):
self.skip = skip
self.limit = limit
self.is_active = is_active
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
result = await run_in_threadpool(
_list_standalone_api_keys_sync,
self.skip,
self.limit,
self.is_active,
)
context.add_audit_metadata(
action="list_standalone_api_keys",
filter_is_active=self.is_active,
limit=self.limit,
skip=self.skip,
total=result["total"],
)
return result
class AdminCreateStandaloneKeyAdapter(AdminApiAdapter):
"""创建独立余额Key"""
def __init__(self, key_data: CreateApiKeyRequest):
self.key_data = key_data
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
result, audit_meta = await run_in_threadpool(
_create_standalone_api_key_sync,
context.user.id,
self.key_data,
)
logger.info(
f"管理员创建独立余额Key: ID {result['id']}, 初始余额 ${self.key_data.initial_balance_usd}"
)
context.add_audit_metadata(**audit_meta)
return result
class AdminUpdateApiKeyAdapter(AdminApiAdapter):
"""更新独立余额Key"""
def __init__(self, key_id: str, key_data: CreateApiKeyRequest):
self.key_id = key_id
self.key_data = key_data
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
result, changed_fields = await run_in_threadpool(
_update_standalone_api_key_sync,
self.key_id,
self.key_data,
)
logger.info(f"管理员更新独立余额Key: ID {self.key_id}, 更新字段 {changed_fields}")
context.add_audit_metadata(
action="update_standalone_api_key",
key_id=self.key_id,
updated_fields=changed_fields,
)
return result
class AdminToggleApiKeyAdapter(AdminApiAdapter):
def __init__(self, key_id: str):
self.key_id = key_id
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
result, audit_meta = await run_in_threadpool(_toggle_standalone_api_key_sync, self.key_id)
logger.info(
f"管理员切换API密钥状态: Key ID {self.key_id}, 新状态 {'启用' if result['is_active'] else '禁用'}"
)
context.add_audit_metadata(**audit_meta)
return result
class AdminDeleteApiKeyAdapter(AdminApiAdapter):
def __init__(self, key_id: str):
self.key_id = key_id
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
result, audit_meta, user_email = await run_in_threadpool(
_delete_standalone_api_key_sync,
self.key_id,
)
logger.info(f"管理员删除API密钥: Key ID {self.key_id}, 用户 {user_email or '未知'}")
context.add_audit_metadata(**audit_meta)
return result
class AdminGetFullKeyAdapter(AdminApiAdapter):
"""获取完整的API密钥"""
def __init__(self, key_id: str):
self.key_id = key_id
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
from src.core.crypto import crypto_service
db = context.db
# 查找API密钥
api_key = db.query(ApiKey).filter(ApiKey.id == self.key_id).first()
if not api_key:
raise NotFoundException("API密钥不存在", "api_key")
if not api_key.is_standalone:
raise InvalidRequestException("仅支持查看独立密钥")
# 解密完整密钥
if not api_key.key_encrypted:
raise HTTPException(status_code=400, detail="该密钥没有存储完整密钥信息")
try:
full_key = crypto_service.decrypt(api_key.key_encrypted)
except Exception as e:
logger.error(f"解密API密钥失败: Key ID {self.key_id}, 错误: {e}")
raise HTTPException(status_code=500, detail="解密密钥失败")
logger.info(f"管理员查看完整API密钥: Key ID {self.key_id}")
context.add_audit_metadata(
action="view_full_api_key",
key_id=self.key_id,
key_name=api_key.name,
)
return {
"key": full_key,
}
class AdminGetKeyDetailAdapter(AdminApiAdapter):
"""Get API key detail without full key"""
def __init__(self, key_id: str):
self.key_id = key_id
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
db = context.db
api_key = db.query(ApiKey).filter(ApiKey.id == self.key_id).first()
if not api_key:
raise NotFoundException("API密钥不存在", "api_key")
if not api_key.is_standalone:
raise InvalidRequestException("仅支持查看独立密钥")
wallet = WalletService.get_wallet(db, api_key_id=api_key.id)
wallet_summary = WalletService.serialize_wallet_summary(wallet)
context.add_audit_metadata(
action="get_api_key_detail",
key_id=self.key_id,
)
return {
"id": api_key.id,
"user_id": api_key.user_id,
"name": api_key.name,
"key_display": api_key.get_display_key(),
"is_active": api_key.is_active,
"is_standalone": api_key.is_standalone,
"total_requests": api_key.total_requests,
"total_tokens": int(
(
db.query(func.sum(Usage.total_tokens))
.filter(Usage.api_key_id == api_key.id)
.scalar()
)
or 0
),
"total_cost_usd": float(api_key.total_cost_usd or 0),
"rate_limit": api_key.rate_limit,
"allowed_providers": api_key.allowed_providers,
"allowed_api_formats": api_key.allowed_api_formats,
"allowed_models": api_key.allowed_models,
"last_used_at": api_key.last_used_at.isoformat() if api_key.last_used_at else None,
"expires_at": api_key.expires_at.isoformat() if api_key.expires_at else None,
"created_at": api_key.created_at.isoformat(),
"updated_at": api_key.updated_at.isoformat() if api_key.updated_at else None,
"wallet": wallet_summary,
}
@@ -1,5 +0,0 @@
"""Billing 配置管理 API 模块(billing_rules / dimension_collectors)。"""
from .routes import router
__all__ = ["router"]
@@ -1,588 +0,0 @@
"""Billing 配置管理 API 路由。
包含:
- billing_rules: 计费规则(公式/变量/维度映射)
- dimension_collectors: 维度采集器(request/response/metadata/computed)
"""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime
from typing import Any, Literal
from fastapi import APIRouter, Depends, Query, Request
from pydantic import BaseModel, Field
from sqlalchemy import func
from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm import Session
from src.api.base.admin_adapter import AdminApiAdapter
from src.api.base.context import ApiRequestContext
from src.api.base.pipeline import get_pipeline
from src.core.exceptions import InvalidRequestException, NotFoundException
from src.database import get_db
from src.models.database import BillingRule, DimensionCollector
from src.services.billing.formula_engine import SafeExpressionEvaluator, UnsafeExpressionError
from src.services.billing.presets import BillingPresetService, PresetApplyMode, list_preset_packs
router = APIRouter(prefix="/api/admin/billing", tags=["Admin - Billing"])
pipeline = get_pipeline()
_expr_validator = SafeExpressionEvaluator()
AllowedTaskType = Literal["chat", "video", "image", "audio"]
AllowedCollectorSourceType = Literal["request", "response", "metadata", "computed"]
AllowedValueType = Literal["float", "int", "string"]
class BillingRuleUpsertRequest(BaseModel):
name: str = Field(..., min_length=1, max_length=100)
task_type: AllowedTaskType = "chat"
global_model_id: str | None = None
model_id: str | None = None
expression: str = Field(..., min_length=1)
variables: dict[str, Any] = Field(default_factory=dict)
dimension_mappings: dict[str, Any] = Field(default_factory=dict)
is_enabled: bool = True
class BillingRuleResponse(BaseModel):
id: str
name: str
task_type: str
global_model_id: str | None
model_id: str | None
expression: str
variables: dict[str, Any]
dimension_mappings: dict[str, Any]
is_enabled: bool
created_at: datetime
updated_at: datetime
@classmethod
def from_orm_obj(cls, rule: BillingRule) -> "BillingRuleResponse":
return cls(
id=rule.id,
name=rule.name,
task_type=rule.task_type,
global_model_id=rule.global_model_id,
model_id=rule.model_id,
expression=rule.expression,
variables=rule.variables or {},
dimension_mappings=rule.dimension_mappings or {},
is_enabled=bool(rule.is_enabled),
created_at=rule.created_at,
updated_at=rule.updated_at,
)
class DimensionCollectorUpsertRequest(BaseModel):
api_format: str = Field(..., min_length=1, max_length=50)
task_type: str = Field(..., min_length=1, max_length=20)
dimension_name: str = Field(..., min_length=1, max_length=100)
source_type: AllowedCollectorSourceType
source_path: str | None = None
value_type: AllowedValueType = "float"
transform_expression: str | None = None
default_value: str | None = None
priority: int = 0
is_enabled: bool = True
class DimensionCollectorResponse(BaseModel):
id: str
api_format: str
task_type: str
dimension_name: str
source_type: str
source_path: str | None
value_type: str
transform_expression: str | None
default_value: str | None
priority: int
is_enabled: bool
created_at: datetime
updated_at: datetime
@classmethod
def from_orm_obj(cls, c: DimensionCollector) -> "DimensionCollectorResponse":
return cls(
id=c.id,
api_format=c.api_format,
task_type=c.task_type,
dimension_name=c.dimension_name,
source_type=c.source_type,
source_path=c.source_path,
value_type=c.value_type,
transform_expression=c.transform_expression,
default_value=c.default_value,
priority=int(c.priority or 0),
is_enabled=bool(c.is_enabled),
created_at=c.created_at,
updated_at=c.updated_at,
)
class BillingPresetInfoResponse(BaseModel):
name: str
version: str
description: str
collector_count: int
class ApplyBillingPresetRequest(BaseModel):
preset: str = Field(..., min_length=1, max_length=100)
mode: PresetApplyMode = "merge"
@router.get("/presets")
async def list_billing_presets(request: Request, db: Session = Depends(get_db)) -> Any:
adapter = BillingPresetListAdapter()
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.post("/presets/apply")
async def apply_billing_preset(request: Request, db: Session = Depends(get_db)) -> Any:
adapter = BillingPresetApplyAdapter()
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.get("/rules")
async def list_billing_rules(
request: Request,
task_type: str | None = Query(None),
is_enabled: bool | None = Query(None),
page: int = Query(1, ge=1),
page_size: int = Query(50, ge=1, le=200),
db: Session = Depends(get_db),
) -> Any:
adapter = BillingRuleListAdapter(
task_type=task_type,
is_enabled=is_enabled,
page=page,
page_size=page_size,
)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.get("/rules/{rule_id}")
async def get_billing_rule(rule_id: str, request: Request, db: Session = Depends(get_db)) -> Any:
adapter = BillingRuleDetailAdapter(rule_id=rule_id)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.post("/rules")
async def create_billing_rule(request: Request, db: Session = Depends(get_db)) -> Any:
adapter = BillingRuleCreateAdapter()
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.put("/rules/{rule_id}")
async def update_billing_rule(rule_id: str, request: Request, db: Session = Depends(get_db)) -> Any:
adapter = BillingRuleUpdateAdapter(rule_id=rule_id)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.get("/collectors")
async def list_dimension_collectors(
request: Request,
api_format: str | None = Query(None),
task_type: str | None = Query(None),
dimension_name: str | None = Query(None),
is_enabled: bool | None = Query(None),
page: int = Query(1, ge=1),
page_size: int = Query(50, ge=1, le=200),
db: Session = Depends(get_db),
) -> Any:
adapter = DimensionCollectorListAdapter(
api_format=api_format,
task_type=task_type,
dimension_name=dimension_name,
is_enabled=is_enabled,
page=page,
page_size=page_size,
)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.get("/collectors/{collector_id}")
async def get_dimension_collector(
collector_id: str, request: Request, db: Session = Depends(get_db)
) -> Any:
adapter = DimensionCollectorDetailAdapter(collector_id=collector_id)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.post("/collectors")
async def create_dimension_collector(request: Request, db: Session = Depends(get_db)) -> Any:
adapter = DimensionCollectorCreateAdapter()
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.put("/collectors/{collector_id}")
async def update_dimension_collector(
collector_id: str, request: Request, db: Session = Depends(get_db)
) -> Any:
adapter = DimensionCollectorUpdateAdapter(collector_id=collector_id)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
# ---------------------------------------------------------------------------
# Adapters
# ---------------------------------------------------------------------------
@dataclass
class BillingRuleListAdapter(AdminApiAdapter):
page: int
page_size: int
task_type: str | None = None
is_enabled: bool | None = None
async def handle(self, context: ApiRequestContext) -> dict[str, Any]:
q = context.db.query(BillingRule)
if self.task_type:
q = q.filter(BillingRule.task_type == self.task_type.lower())
if self.is_enabled is not None:
q = q.filter(BillingRule.is_enabled == self.is_enabled)
total = int(q.with_entities(func.count(BillingRule.id)).scalar() or 0)
items = (
q.order_by(BillingRule.updated_at.desc())
.offset((self.page - 1) * self.page_size)
.limit(self.page_size)
.all()
)
return {
"items": [BillingRuleResponse.from_orm_obj(r).model_dump() for r in items],
"total": total,
"page": self.page,
"page_size": self.page_size,
"pages": (total + self.page_size - 1) // self.page_size,
}
@dataclass
class BillingRuleDetailAdapter(AdminApiAdapter):
rule_id: str
async def handle(self, context: ApiRequestContext) -> dict[str, Any]:
rule = context.db.query(BillingRule).filter(BillingRule.id == self.rule_id).first()
if not rule:
raise NotFoundException("Billing rule not found")
return BillingRuleResponse.from_orm_obj(rule).model_dump()
class BillingRuleCreateAdapter(AdminApiAdapter):
async def handle(self, context: ApiRequestContext) -> dict[str, Any]:
payload = context.ensure_json_body()
try:
req = BillingRuleUpsertRequest.model_validate(payload)
except Exception as exc:
raise InvalidRequestException(f"Invalid request body: {exc}")
_validate_billing_rule_request(req)
rule = BillingRule(
name=req.name,
task_type=req.task_type,
global_model_id=req.global_model_id,
model_id=req.model_id,
expression=req.expression,
variables=req.variables,
dimension_mappings=req.dimension_mappings,
is_enabled=req.is_enabled,
)
context.db.add(rule)
try:
context.db.commit()
except IntegrityError as exc:
context.db.rollback()
raise InvalidRequestException(f"Integrity error: {exc}")
context.db.refresh(rule)
return BillingRuleResponse.from_orm_obj(rule).model_dump()
@dataclass
class BillingRuleUpdateAdapter(AdminApiAdapter):
rule_id: str
async def handle(self, context: ApiRequestContext) -> dict[str, Any]:
rule = context.db.query(BillingRule).filter(BillingRule.id == self.rule_id).first()
if not rule:
raise NotFoundException("Billing rule not found")
payload = context.ensure_json_body()
try:
req = BillingRuleUpsertRequest.model_validate(payload)
except Exception as exc:
raise InvalidRequestException(f"Invalid request body: {exc}")
_validate_billing_rule_request(req)
rule.name = req.name
rule.task_type = req.task_type
rule.global_model_id = req.global_model_id
rule.model_id = req.model_id
rule.expression = req.expression
rule.variables = req.variables
rule.dimension_mappings = req.dimension_mappings
rule.is_enabled = req.is_enabled
try:
context.db.commit()
except IntegrityError as exc:
context.db.rollback()
raise InvalidRequestException(f"Integrity error: {exc}")
context.db.refresh(rule)
return BillingRuleResponse.from_orm_obj(rule).model_dump()
@dataclass
class DimensionCollectorListAdapter(AdminApiAdapter):
page: int
page_size: int
api_format: str | None = None
task_type: str | None = None
dimension_name: str | None = None
is_enabled: bool | None = None
async def handle(self, context: ApiRequestContext) -> dict[str, Any]:
q = context.db.query(DimensionCollector)
if self.api_format:
q = q.filter(DimensionCollector.api_format == self.api_format.upper())
if self.task_type:
q = q.filter(DimensionCollector.task_type == self.task_type.lower())
if self.dimension_name:
q = q.filter(DimensionCollector.dimension_name == self.dimension_name)
if self.is_enabled is not None:
q = q.filter(DimensionCollector.is_enabled == self.is_enabled)
total = int(q.with_entities(func.count(DimensionCollector.id)).scalar() or 0)
items = (
q.order_by(DimensionCollector.updated_at.desc())
.offset((self.page - 1) * self.page_size)
.limit(self.page_size)
.all()
)
return {
"items": [DimensionCollectorResponse.from_orm_obj(c).model_dump() for c in items],
"total": total,
"page": self.page,
"page_size": self.page_size,
"pages": (total + self.page_size - 1) // self.page_size,
}
@dataclass
class DimensionCollectorDetailAdapter(AdminApiAdapter):
collector_id: str
async def handle(self, context: ApiRequestContext) -> dict[str, Any]:
c = (
context.db.query(DimensionCollector)
.filter(DimensionCollector.id == self.collector_id)
.first()
)
if not c:
raise NotFoundException("Dimension collector not found")
return DimensionCollectorResponse.from_orm_obj(c).model_dump()
class DimensionCollectorCreateAdapter(AdminApiAdapter):
async def handle(self, context: ApiRequestContext) -> dict[str, Any]:
payload = context.ensure_json_body()
try:
req = DimensionCollectorUpsertRequest.model_validate(payload)
except Exception as exc:
raise InvalidRequestException(f"Invalid request body: {exc}")
_validate_dimension_collector_request(context.db, req, existing_id=None)
c = DimensionCollector(
api_format=req.api_format.upper(),
task_type=req.task_type.lower(),
dimension_name=req.dimension_name,
source_type=req.source_type,
source_path=req.source_path,
value_type=req.value_type,
transform_expression=req.transform_expression,
default_value=req.default_value,
priority=req.priority,
is_enabled=req.is_enabled,
)
context.db.add(c)
try:
context.db.commit()
except IntegrityError as exc:
context.db.rollback()
raise InvalidRequestException(f"Integrity error: {exc}")
context.db.refresh(c)
return DimensionCollectorResponse.from_orm_obj(c).model_dump()
@dataclass
class DimensionCollectorUpdateAdapter(AdminApiAdapter):
collector_id: str
async def handle(self, context: ApiRequestContext) -> dict[str, Any]:
c = (
context.db.query(DimensionCollector)
.filter(DimensionCollector.id == self.collector_id)
.first()
)
if not c:
raise NotFoundException("Dimension collector not found")
payload = context.ensure_json_body()
try:
req = DimensionCollectorUpsertRequest.model_validate(payload)
except Exception as exc:
raise InvalidRequestException(f"Invalid request body: {exc}")
_validate_dimension_collector_request(context.db, req, existing_id=self.collector_id)
c.api_format = req.api_format.upper()
c.task_type = req.task_type.lower()
c.dimension_name = req.dimension_name
c.source_type = req.source_type
c.source_path = req.source_path
c.value_type = req.value_type
c.transform_expression = req.transform_expression
c.default_value = req.default_value
c.priority = req.priority
c.is_enabled = req.is_enabled
try:
context.db.commit()
except IntegrityError as exc:
context.db.rollback()
raise InvalidRequestException(f"Integrity error: {exc}")
context.db.refresh(c)
return DimensionCollectorResponse.from_orm_obj(c).model_dump()
# ---------------------------------------------------------------------------
# Validation helpers
# ---------------------------------------------------------------------------
def _validate_billing_rule_request(req: BillingRuleUpsertRequest) -> None:
# model/global_model 二选一
if bool(req.global_model_id) == bool(req.model_id):
raise InvalidRequestException("Exactly one of global_model_id or model_id must be provided")
# task_type 校验:Pydantic Literal 已限制为 "chat", "video", "image", "audio"
# 注:CLI 在计费域等同于 chat,billing_rules 不存储 "cli"
# expression 安全校验
try:
_expr_validator.validate(req.expression)
except UnsafeExpressionError as exc:
raise InvalidRequestException(f"Invalid expression: {exc}")
# variables 必须为数值(JSON 可包含 int/float)
if not isinstance(req.variables, dict):
raise InvalidRequestException("variables must be a JSON object")
for k, v in req.variables.items():
if not isinstance(k, str) or not k:
raise InvalidRequestException("variables keys must be non-empty strings")
if isinstance(v, bool) or not isinstance(v, (int, float)):
raise InvalidRequestException(f"variables['{k}'] must be a number")
# dimension_mappings 结构做轻量校验(详细 schema 由业务侧保障)
if not isinstance(req.dimension_mappings, dict):
raise InvalidRequestException("dimension_mappings must be a JSON object")
for var_name, mapping in req.dimension_mappings.items():
if not isinstance(var_name, str) or not var_name:
raise InvalidRequestException("dimension_mappings keys must be non-empty strings")
if not isinstance(mapping, dict):
raise InvalidRequestException(f"dimension_mappings['{var_name}'] must be an object")
if "source" not in mapping:
raise InvalidRequestException(f"dimension_mappings['{var_name}'].source is required")
def _validate_dimension_collector_request(
db: Session,
req: DimensionCollectorUpsertRequest,
*,
existing_id: str | None,
) -> None:
src = req.source_type
if src == "computed":
if req.source_path is not None:
raise InvalidRequestException("computed collector must have source_path=null")
if not req.transform_expression:
raise InvalidRequestException("computed collector must have transform_expression")
else:
if not req.source_path:
raise InvalidRequestException("non-computed collector must have source_path")
# transform_expression 安全校验(如配置)
if req.transform_expression:
try:
_expr_validator.validate(req.transform_expression)
except UnsafeExpressionError as exc:
raise InvalidRequestException(f"Invalid transform_expression: {exc}")
# default_value 仅允许同一维度一条(enabled=true)
if req.default_value is not None and req.is_enabled:
q = db.query(DimensionCollector).filter(
DimensionCollector.api_format == req.api_format.upper(),
DimensionCollector.task_type == req.task_type.lower(),
DimensionCollector.dimension_name == req.dimension_name,
DimensionCollector.is_enabled.is_(True),
DimensionCollector.default_value.isnot(None),
)
if existing_id:
q = q.filter(DimensionCollector.id != existing_id)
exists = db.query(q.exists()).scalar()
if exists:
raise InvalidRequestException(
"default_value already exists for this (api_format, task_type, dimension_name)"
)
@dataclass
class BillingPresetListAdapter(AdminApiAdapter):
async def handle(self, context: ApiRequestContext) -> dict[str, Any]:
items = []
for p in list_preset_packs():
items.append(
BillingPresetInfoResponse(
name=p.name,
version=p.version,
description=p.description,
collector_count=len(p.collectors or []),
).model_dump()
)
return {"items": items}
class BillingPresetApplyAdapter(AdminApiAdapter):
async def handle(self, context: ApiRequestContext) -> dict[str, Any]:
payload = context.ensure_json_body()
try:
req = ApplyBillingPresetRequest.model_validate(payload)
except Exception as exc:
raise InvalidRequestException(f"Invalid request body: {exc}")
result = BillingPresetService.apply_preset(
context.db,
preset_name=req.preset,
mode=req.mode,
)
if result.errors:
# still return counts; caller can display partial results
return {"ok": False, **result.to_dict()}
return {"ok": True, **result.to_dict()}
@@ -1,24 +0,0 @@
"""Endpoint management API routers."""
from fastapi import APIRouter
from .concurrency import router as concurrency_router
from .health import router as health_router
from .keys import router as keys_router
from .routes import router as routes_router
router = APIRouter(prefix="/api/admin/endpoints", tags=["Admin - Endpoints"])
# Endpoint CRUD
router.include_router(routes_router)
# Endpoint Keys management
router.include_router(keys_router)
# Health monitoring
router.include_router(health_router)
# Concurrency control
router.include_router(concurrency_router)
__all__ = ["router"]
@@ -1,99 +0,0 @@
"""
Key RPM 限制管理 API
"""
from dataclasses import dataclass
from typing import Any
from fastapi import APIRouter, Depends, Request
from sqlalchemy.orm import Session
from src.api.base.admin_adapter import AdminApiAdapter
from src.api.base.context import ApiRequestContext
from src.api.base.pipeline import get_pipeline
from src.core.exceptions import NotFoundException
from src.database import get_db
from src.models.database import ProviderAPIKey
from src.models.endpoint_models import KeyRpmStatusResponse
from src.services.rate_limit.concurrency_manager import get_concurrency_manager
router = APIRouter(tags=["RPM Control"])
pipeline = get_pipeline()
@router.get("/rpm/key/{key_id}", response_model=KeyRpmStatusResponse)
async def get_key_rpm(
key_id: str,
request: Request,
db: Session = Depends(get_db),
) -> KeyRpmStatusResponse:
"""
获取 Key 当前 RPM 状态
查询指定 API Key 的实时 RPM 使用情况,包括当前 RPM 计数和最大 RPM 限制。
**路径参数**:
- `key_id`: API Key ID
**返回字段**:
- `key_id`: API Key ID
- `current_rpm`: 当前 RPM 计数
- `rpm_limit`: RPM 限制
"""
adapter = AdminKeyRpmAdapter(key_id=key_id)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.delete("/rpm/key/{key_id}")
async def reset_key_rpm(
key_id: str,
http_request: Request,
db: Session = Depends(get_db),
) -> dict:
"""
重置 Key RPM 计数器
重置指定 API Key 的 RPM 计数器,用于解决计数不准确的问题。
管理员功能,请谨慎使用。
**路径参数**:
- `key_id`: API Key ID
**返回字段**:
- `message`: 操作结果消息
"""
adapter = AdminResetKeyRpmAdapter(key_id=key_id)
return await pipeline.run(adapter=adapter, http_request=http_request, db=db, mode=adapter.mode)
# -------- Adapters --------
@dataclass
class AdminKeyRpmAdapter(AdminApiAdapter):
key_id: str
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
db = context.db
key = db.query(ProviderAPIKey).filter(ProviderAPIKey.id == self.key_id).first()
if not key:
raise NotFoundException(f"Key {self.key_id} 不存在")
concurrency_manager = await get_concurrency_manager()
key_count = await concurrency_manager.get_key_rpm_count(key_id=self.key_id)
return KeyRpmStatusResponse(
key_id=self.key_id,
current_rpm=key_count,
rpm_limit=key.rpm_limit,
)
@dataclass
class AdminResetKeyRpmAdapter(AdminApiAdapter):
key_id: str
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
concurrency_manager = await get_concurrency_manager()
await concurrency_manager.reset_key_rpm(key_id=self.key_id)
return {"message": "RPM 计数已重置"}
@@ -1,601 +0,0 @@
"""
Endpoint 健康监控 API
"""
from __future__ import annotations
import asyncio
from collections import defaultdict
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from typing import Any
from fastapi import APIRouter, Depends, Query, Request
from sqlalchemy import func
from sqlalchemy.orm import Session
from src.api.base.admin_adapter import AdminApiAdapter
from src.api.base.context import ApiRequestContext
from src.api.base.pipeline import get_pipeline
from src.core.exceptions import NotFoundException
from src.core.logger import logger
from src.database import get_db
from src.models.database import Provider, ProviderAPIKey, ProviderEndpoint, RequestCandidate
from src.models.endpoint_models import (
ApiFormatHealthMonitor,
ApiFormatHealthMonitorResponse,
EndpointHealthEvent,
HealthStatusResponse,
HealthSummaryResponse,
)
from src.services.health.endpoint import EndpointHealthService
from src.services.health.monitor import HealthMonitor, get_health_monitor
router = APIRouter(tags=["Endpoint Health"])
def _recover_key_health_sync(db: Session, key_id: str, api_format: str | None) -> dict[str, Any]:
key = db.query(ProviderAPIKey).filter(ProviderAPIKey.id == key_id).first()
if not key:
raise NotFoundException(f"Key {key_id} 不存在")
success = get_health_monitor().reset_health(db, key_id=key_id, api_format=api_format)
if not success:
raise Exception("重置健康度失败")
if not key.is_active:
key.is_active = True # type: ignore[assignment]
db.commit()
return {
"is_active": bool(key.is_active),
"api_format": api_format,
}
def _recover_all_keys_health_sync(db: Session) -> list[dict[str, Any]]:
candidates = (
db.query(ProviderAPIKey)
.filter(
ProviderAPIKey.circuit_breaker_by_format.isnot(None),
ProviderAPIKey.circuit_breaker_by_format != "{}",
)
.all()
)
circuit_open_keys = [
key
for key in candidates
if any(cb.get("open") for cb in (key.circuit_breaker_by_format or {}).values())
]
recovered_keys: list[dict[str, Any]] = []
for key in circuit_open_keys:
key.health_by_format = {} # type: ignore[assignment]
key.circuit_breaker_by_format = {} # type: ignore[assignment]
recovered_keys.append(
{
"key_id": key.id,
"key_name": key.name,
"provider_id": key.provider_id,
"api_formats": key.api_formats,
}
)
if recovered_keys:
db.commit()
return recovered_keys
def _format_str(api_format_enum: Any) -> str:
"""将 DB 查询返回的 api_format(可能是 enum 或 str)统一转为 str。"""
return api_format_enum.value if hasattr(api_format_enum, "value") else str(api_format_enum)
def _fetch_recent_attempts_for_api_format(
db: Session,
*,
api_format: str,
since: datetime,
per_format_limit: int,
) -> list[RequestCandidate]:
"""获取单个 API 格式最近的最终态请求,用于事件展示。"""
final_statuses = ["success", "failed", "skipped"]
return (
db.query(RequestCandidate)
.join(ProviderEndpoint, RequestCandidate.endpoint_id == ProviderEndpoint.id)
.join(Provider, ProviderEndpoint.provider_id == Provider.id)
.filter(
ProviderEndpoint.is_active.is_(True),
Provider.is_active.is_(True),
ProviderEndpoint.api_format == api_format,
RequestCandidate.created_at >= since,
RequestCandidate.status.in_(final_statuses),
)
.order_by(RequestCandidate.created_at.desc())
.limit(per_format_limit)
.all()
)
pipeline = get_pipeline()
@router.get("/health/summary", response_model=HealthSummaryResponse)
async def get_health_summary(
request: Request,
db: Session = Depends(get_db),
) -> HealthSummaryResponse:
"""
获取健康状态摘要
获取系统整体健康状态摘要,包括所有 Provider、Endpoint 和 Key 的健康状态统计。
**返回字段**:
- `total_providers`: Provider 总数
- `active_providers`: 活跃 Provider 数量
- `total_endpoints`: Endpoint 总数
- `active_endpoints`: 活跃 Endpoint 数量
- `total_keys`: Key 总数
- `active_keys`: 活跃 Key 数量
- `circuit_breaker_open_keys`: 熔断的 Key 数量
"""
adapter = AdminHealthSummaryAdapter()
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.get("/health/status")
async def get_endpoint_health_status(
request: Request,
lookback_hours: int = Query(6, ge=1, le=72, description="回溯的小时数"),
db: Session = Depends(get_db),
) -> Any:
"""
获取端点健康状态(简化视图,与用户端点统一)
获取按 API 格式聚合的端点健康状态时间线,基于 Usage 表统计,
返回 50 个时间段的聚合状态,适用于快速查看整体健康趋势。
与 /health/api-formats 的区别:
- /health/status: 返回聚合的时间线状态(50个时间段),基于 Usage 表
- /health/api-formats: 返回详细的事件列表,基于 RequestCandidate 表
**查询参数**:
- `lookback_hours`: 回溯的小时数(1-72),默认 6
**返回字段**:
- `api_format`: API 格式名称
- `timeline`: 时间线数据(50个时间段)
- `time_range_start`: 时间范围起始
- `time_range_end`: 时间范围结束
"""
adapter = AdminEndpointHealthStatusAdapter(lookback_hours=lookback_hours)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.get("/health/api-formats", response_model=ApiFormatHealthMonitorResponse)
async def get_api_format_health_monitor(
request: Request,
lookback_hours: int = Query(6, ge=1, le=72, description="回溯的小时数"),
per_format_limit: int = Query(60, ge=10, le=200, description="每个 API 格式的事件数量"),
db: Session = Depends(get_db),
) -> ApiFormatHealthMonitorResponse:
"""
获取按 API 格式聚合的健康监控时间线(详细事件列表)
获取每个 API 格式的详细健康监控数据,包括请求事件列表、成功率统计、
时间线数据等,基于 RequestCandidate 表查询,适用于详细分析。
**查询参数**:
- `lookback_hours`: 回溯的小时数(1-72),默认 6
- `per_format_limit`: 每个 API 格式返回的事件数量(10-200),默认 60
**返回字段**:
- `generated_at`: 数据生成时间
- `formats`: API 格式健康监控数据列表
- `api_format`: API 格式名称
- `total_attempts`: 总请求数
- `success_count`: 成功请求数
- `failed_count`: 失败请求数
- `skipped_count`: 跳过请求数
- `success_rate`: 成功率
- `provider_count`: Provider 数量
- `key_count`: Key 数量
- `last_event_at`: 最后事件时间
- `events`: 事件列表
- `timeline`: 时间线数据
- `time_range_start`: 时间范围起始
- `time_range_end`: 时间范围结束
"""
adapter = AdminApiFormatHealthMonitorAdapter(
lookback_hours=lookback_hours,
per_format_limit=per_format_limit,
)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.get("/health/key/{key_id}", response_model=HealthStatusResponse)
async def get_key_health(
key_id: str,
request: Request,
api_format: str | None = Query(None, description="API 格式(可选,如 CLAUDE、OPENAI)"),
db: Session = Depends(get_db),
) -> HealthStatusResponse:
"""
获取 Key 健康状态
获取指定 API Key 的健康状态详情,包括健康分数、连续失败次数、
熔断器状态等信息。支持按 API 格式查询。
**路径参数**:
- `key_id`: API Key ID
**查询参数**:
- `api_format`: 可选,指定 API 格式(如 CLAUDE、OPENAI)。
- 指定时返回该格式的健康度详情
- 不指定时返回所有格式的健康度摘要
**返回字段**:
- `key_id`: API Key ID
- `key_health_score`: 健康分数(0.0-1.0)
- `key_is_active`: 是否活跃
- `key_statistics`: 统计信息
- `health_by_format`: 按格式的健康度数据(无 api_format 参数时)
- `circuit_breaker_open`: 熔断器是否打开(有 api_format 参数时)
"""
adapter = AdminKeyHealthAdapter(key_id=key_id, api_format=api_format)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.patch("/health/keys/{key_id}")
async def recover_key_health(
key_id: str,
request: Request,
api_format: str | None = Query(None, description="API 格式(可选,不指定则恢复所有格式)"),
db: Session = Depends(get_db),
) -> dict:
"""
恢复 Key 健康状态
手动恢复指定 Key 的健康状态,将健康分数重置为 1.0,关闭熔断器,
取消自动禁用,并重置所有失败计数。支持按 API 格式恢复。
**路径参数**:
- `key_id`: API Key ID
**查询参数**:
- `api_format`: 可选,指定 API 格式(如 CLAUDE、OPENAI)
- 指定时仅恢复该格式的健康度
- 不指定时恢复所有格式
**返回字段**:
- `message`: 操作结果消息
- `details`: 详细信息
- `health_score`: 健康分数
- `circuit_breaker_open`: 熔断器状态
- `is_active`: 是否活跃
"""
adapter = AdminRecoverKeyHealthAdapter(key_id=key_id, api_format=api_format)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.patch("/health/keys")
async def recover_all_keys_health(
request: Request,
db: Session = Depends(get_db),
) -> dict:
"""
批量恢复所有熔断 Key 的健康状态
查找所有处于熔断状态的 Key(circuit_breaker_open=True),
并批量执行以下操作:
1. 将健康分数重置为 1.0
2. 关闭熔断器
3. 重置失败计数
**返回字段**:
- `message`: 操作结果消息
- `recovered_count`: 恢复的 Key 数量
- `recovered_keys`: 恢复的 Key 列表
- `key_id`: Key ID
- `key_name`: Key 名称
- `endpoint_id`: Endpoint ID
"""
adapter = AdminRecoverAllKeysHealthAdapter()
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
# -------- Adapters --------
class AdminHealthSummaryAdapter(AdminApiAdapter):
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
summary = get_health_monitor().get_all_health_status(context.db)
return HealthSummaryResponse(**summary)
@dataclass
class AdminEndpointHealthStatusAdapter(AdminApiAdapter):
"""管理员端点健康状态适配器(与用户端点统一,但包含管理员字段)"""
lookback_hours: int
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
db = context.db
# 使用共享服务获取健康状态(管理员视图)
result = EndpointHealthService.get_endpoint_health_by_format(
db=db,
lookback_hours=self.lookback_hours,
include_admin_fields=True, # 包含管理员字段
use_cache=False, # 管理员不使用缓存,确保实时性
)
context.add_audit_metadata(
action="endpoint_health_status",
format_count=len(result),
lookback_hours=self.lookback_hours,
)
return result
@dataclass
class AdminApiFormatHealthMonitorAdapter(AdminApiAdapter):
lookback_hours: int
per_format_limit: int
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
db = context.db
now = datetime.now(timezone.utc)
since = now - timedelta(hours=self.lookback_hours)
# 1. 单次查询获取所有活跃 endpoint 行,在内存中聚合 provider_count / endpoint_map
endpoint_rows = (
db.query(ProviderEndpoint.api_format, ProviderEndpoint.id, ProviderEndpoint.provider_id)
.join(Provider, ProviderEndpoint.provider_id == Provider.id)
.filter(
ProviderEndpoint.is_active.is_(True),
Provider.is_active.is_(True),
)
.all()
)
all_formats: dict[str, int] = {} # api_format -> distinct provider count
endpoint_map: dict[str, list[str]] = defaultdict(list)
active_provider_formats: set[tuple[str, str]] = set()
_provider_sets: dict[str, set[str]] = defaultdict(set)
for api_format_enum, endpoint_id, provider_id in endpoint_rows:
fmt = _format_str(api_format_enum)
endpoint_map[fmt].append(endpoint_id)
_provider_sets[fmt].add(str(provider_id))
active_provider_formats.add((str(provider_id), fmt))
for fmt, pids in _provider_sets.items():
all_formats[fmt] = len(pids)
# 1.2 统计每个 API 格式可用的活跃 Key 数量(Key 属于 Provider,通过 api_formats 关联格式)
key_counts: dict[str, int] = {}
if active_provider_formats:
active_provider_keys = (
db.query(ProviderAPIKey.provider_id, ProviderAPIKey.api_formats)
.join(Provider, ProviderAPIKey.provider_id == Provider.id)
.filter(
Provider.is_active.is_(True),
ProviderAPIKey.is_active.is_(True),
)
.all()
)
for provider_id, api_formats in active_provider_keys:
pid = str(provider_id)
for fmt in api_formats or []:
if (pid, fmt) not in active_provider_formats:
continue
key_counts[fmt] = key_counts.get(fmt, 0) + 1
# 2. 统计窗口内每个 API 格式的请求状态分布(真实统计)
# 只统计最终状态:success, failed, skipped
final_statuses = ["success", "failed", "skipped"]
status_counts_query = (
db.query(
ProviderEndpoint.api_format,
RequestCandidate.status,
func.count(RequestCandidate.id).label("count"),
)
.join(RequestCandidate, ProviderEndpoint.id == RequestCandidate.endpoint_id)
.join(Provider, ProviderEndpoint.provider_id == Provider.id)
.filter(
ProviderEndpoint.is_active.is_(True),
Provider.is_active.is_(True),
RequestCandidate.created_at >= since,
RequestCandidate.status.in_(final_statuses),
)
.group_by(ProviderEndpoint.api_format, RequestCandidate.status)
.all()
)
# 构建每个格式的状态统计
status_counts: dict[str, dict[str, int]] = {}
for api_format_enum, status, count in status_counts_query:
fmt = _format_str(api_format_enum)
if fmt not in status_counts:
status_counts[fmt] = {"success": 0, "failed": 0, "skipped": 0}
status_counts[fmt][status] = count
# 3. 为所有活跃格式生成监控数据(包括没有请求记录的)
monitors: list[ApiFormatHealthMonitor] = []
for api_format in all_formats:
attempts = _fetch_recent_attempts_for_api_format(
db=db,
api_format=api_format,
since=since,
per_format_limit=self.per_format_limit,
)
# 获取窗口内的真实统计数据
# 只统计最终状态:success, failed, skipped
# 中间状态(available, pending, used, started)不计入统计
format_stats = status_counts.get(api_format, {"success": 0, "failed": 0, "skipped": 0})
real_success_count = format_stats.get("success", 0)
real_failed_count = format_stats.get("failed", 0)
real_skipped_count = format_stats.get("skipped", 0)
# total_attempts 只包含最终状态的请求数
total_attempts = real_success_count + real_failed_count + real_skipped_count
# 时间线按时间正序
attempts_sorted = list(reversed(attempts))
events: list[EndpointHealthEvent] = []
for attempt in attempts_sorted:
event_timestamp = attempt.finished_at or attempt.started_at or attempt.created_at
events.append(
EndpointHealthEvent(
timestamp=event_timestamp,
status=attempt.status,
status_code=attempt.status_code,
latency_ms=attempt.latency_ms,
error_type=attempt.error_type,
error_message=attempt.error_message,
)
)
# 成功率 = success / (success + failed)
# skipped 不算失败,不计入成功率分母
# 无实际完成请求时成功率为 1.0(灰色状态)
actual_completed = real_success_count + real_failed_count
success_rate = real_success_count / actual_completed if actual_completed > 0 else 1.0
last_event_at = events[-1].timestamp if events else None
# 生成 Usage 基于时间窗口的健康时间线
timeline_data = EndpointHealthService._generate_timeline_from_usage(
db=db,
endpoint_ids=endpoint_map.get(api_format, []),
now=now,
lookback_hours=self.lookback_hours,
)
monitors.append(
ApiFormatHealthMonitor(
api_format=api_format,
total_attempts=total_attempts, # 真实总请求数
success_count=real_success_count, # 真实成功数
failed_count=real_failed_count, # 真实失败数
skipped_count=real_skipped_count, # 真实跳过数
success_rate=success_rate, # 基于真实统计的成功率
provider_count=all_formats[api_format],
key_count=key_counts.get(api_format, 0),
last_event_at=last_event_at,
events=events, # 限制为 per_format_limit 条(用于时间线显示)
timeline=timeline_data.get("timeline", []),
time_range_start=timeline_data.get("time_range_start"),
time_range_end=timeline_data.get("time_range_end"),
)
)
response = ApiFormatHealthMonitorResponse(
generated_at=now,
formats=monitors,
)
context.add_audit_metadata(
action="api_format_health_monitor",
format_count=len(monitors),
lookback_hours=self.lookback_hours,
per_format_limit=self.per_format_limit,
)
return response
@dataclass
class AdminKeyHealthAdapter(AdminApiAdapter):
key_id: str
api_format: str | None = None
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
health_data = get_health_monitor().get_key_health(context.db, self.key_id, self.api_format)
if not health_data:
raise NotFoundException(f"Key {self.key_id} 不存在")
# 构建响应
response_data = {
"key_id": health_data["key_id"],
"key_is_active": health_data["is_active"],
"key_statistics": health_data.get("statistics"),
"key_health_score": health_data.get("health_score", 1.0),
}
if self.api_format:
# 单格式查询
response_data["api_format"] = self.api_format
response_data["key_consecutive_failures"] = health_data.get("consecutive_failures")
response_data["key_last_failure_at"] = health_data.get("last_failure_at")
circuit = health_data.get("circuit_breaker", {})
response_data["circuit_breaker_open"] = circuit.get("open", False)
response_data["circuit_breaker_open_at"] = circuit.get("open_at")
response_data["next_probe_at"] = circuit.get("next_probe_at")
response_data["half_open_until"] = circuit.get("half_open_until")
response_data["half_open_successes"] = circuit.get("half_open_successes", 0)
response_data["half_open_failures"] = circuit.get("half_open_failures", 0)
else:
# 全格式查询
response_data["any_circuit_open"] = health_data.get("any_circuit_open", False)
response_data["health_by_format"] = health_data.get("health_by_format")
return HealthStatusResponse(**response_data)
@dataclass
class AdminRecoverKeyHealthAdapter(AdminApiAdapter):
key_id: str
api_format: str | None = None
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
db = context.db
await asyncio.to_thread(_recover_key_health_sync, db, self.key_id, self.api_format)
if self.api_format:
logger.info(f"管理员恢复Key健康状态: {self.key_id}/{self.api_format}")
return {
"message": f"Key 的 {self.api_format} 格式已恢复",
"details": {
"api_format": self.api_format,
"health_score": 1.0,
"circuit_breaker_open": False,
"is_active": True,
},
}
else:
logger.info(f"管理员恢复Key健康状态: {self.key_id} (所有格式)")
return {
"message": "Key 所有格式已恢复",
"details": {
"health_score": 1.0,
"circuit_breaker_open": False,
"is_active": True,
},
}
class AdminRecoverAllKeysHealthAdapter(AdminApiAdapter):
"""批量恢复所有熔断 Key 的健康状态"""
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
db = context.db
recovered_keys = await asyncio.to_thread(_recover_all_keys_health_sync, db)
if not recovered_keys:
return {
"message": "没有需要恢复的 Key",
"recovered_count": 0,
"recovered_keys": [],
}
# 重置健康监控器的熔断计数
HealthMonitor.reset_open_circuit_count()
logger.info(f"管理员批量恢复 {len(recovered_keys)} 个 Key 的健康状态")
return {
"message": f"已恢复 {len(recovered_keys)} 个 Key",
"recovered_count": len(recovered_keys),
"recovered_keys": recovered_keys,
}
@@ -1,437 +0,0 @@
"""
Provider API Keys 管理
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Any
from fastapi import APIRouter, Body, Depends, Query, Request
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from src.api.base.admin_adapter import AdminApiAdapter
from src.api.base.context import ApiRequestContext
from src.api.base.pipeline import get_pipeline
from src.database import get_db
from src.models.database import User
from src.models.endpoint_models import (
EndpointAPIKeyCreate,
EndpointAPIKeyResponse,
EndpointAPIKeyUpdate,
)
from src.services.provider_keys import (
batch_delete_endpoint_keys_response,
clear_oauth_invalid_response,
create_provider_key_response,
delete_endpoint_key_response,
export_oauth_key_data,
)
from src.services.provider_keys import get_keys_grouped_by_format as query_keys_grouped_by_format
from src.services.provider_keys import (
list_provider_keys_responses,
refresh_provider_quota_for_provider,
reveal_endpoint_key_payload,
update_endpoint_key_response,
)
from src.services.provider_keys.key_quota_service import (
CODEX_WHAM_USAGE_URL as _CODEX_WHAM_USAGE_URL,
)
from src.utils.auth_utils import require_admin
router = APIRouter(tags=["Provider Keys"])
pipeline = get_pipeline()
@router.put("/keys/{key_id}", response_model=EndpointAPIKeyResponse)
async def update_endpoint_key(
key_id: str,
key_data: EndpointAPIKeyUpdate,
request: Request,
db: Session = Depends(get_db),
) -> EndpointAPIKeyResponse:
"""
更新 Provider Key
更新指定 Key 的配置,支持修改并发限制、速率倍数、优先级、
配额限制、能力限制等。支持部分更新。
**路径参数**:
- `key_id`: Key ID
**请求体字段**(均为可选):
- `api_key`: 新的 API Key 原文
- `name`: Key 名称
- `note`: 备注
- `rate_multipliers`: 按 API 格式的成本倍率
- `internal_priority`: 内部优先级
- `rpm_limit`: RPM 限制(设置为 null 可切换到自适应模式)
- `allowed_models`: 允许的模型列表
- `capabilities`: 能力配置
- `is_active`: 是否活跃
**返回字段**:
- 包含更新后的完整 Key 信息
"""
adapter = AdminUpdateEndpointKeyAdapter(key_id=key_id, key_data=key_data)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.get("/keys/grouped-by-format")
async def get_keys_grouped_by_format(
request: Request,
db: Session = Depends(get_db),
) -> dict:
"""
获取按 API 格式分组的所有 Keys
获取所有活跃的 Key,按 API 格式分组返回,用于全局优先级管理。
每个 Key 包含基本信息、健康度指标、能力标签等。
**返回字段**:
- 返回一个字典,键为 API 格式,值为该格式下的 Key 列表
- 每个 Key 包含:
- `id`: Key ID
- `name`: Key 名称
- `api_key_masked`: 脱敏后的 API Key
- `internal_priority`: 内部优先级
- `global_priority_by_format`: 按 API 格式的全局优先级
- `format_priority`: 当前格式的优先级
- `rate_multipliers`: 按 API 格式的成本倍率
- `is_active`: 是否活跃
- `circuit_breaker_open`: 熔断器状态
- `provider_name`: Provider 名称
- `endpoint_base_url`: Endpoint 基础 URL
- `api_format`: API 格式
- `capabilities`: 能力简称列表
- `success_rate`: 成功率
- `avg_response_time_ms`: 平均响应时间
- `request_count`: 请求总数
"""
adapter = AdminGetKeysGroupedByFormatAdapter()
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.get("/keys/{key_id}/reveal")
async def reveal_endpoint_key(
key_id: str,
request: Request,
db: Session = Depends(get_db),
) -> dict:
"""
获取完整的 API Key
解密并返回指定 Key 的完整原文,用于查看和复制。
此操作会被记录到审计日志。
**路径参数**:
- `key_id`: Key ID
**返回字段**:
- `api_key`: 完整的 API Key 原文
"""
adapter = AdminRevealEndpointKeyAdapter(key_id=key_id)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.get("/keys/{key_id}/export")
async def export_key(
key_id: str,
request: Request,
db: Session = Depends(get_db),
_: User = Depends(require_admin),
) -> dict:
"""
导出 OAuth Key 凭据(用于跨实例迁移)
解密 auth_config,返回精简的扁平 JSON,去掉 null 和临时字段。
所有 OAuth Provider 格式统一。
**路径参数**:
- `key_id`: Key ID
"""
adapter = AdminExportKeyAdapter(key_id=key_id)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.delete("/keys/{key_id}")
async def delete_endpoint_key(
key_id: str,
request: Request,
db: Session = Depends(get_db),
) -> dict:
"""
删除 Provider Key
删除指定的 API Key。此操作不可逆,请谨慎使用。
**路径参数**:
- `key_id`: Key ID
**返回字段**:
- `message`: 操作结果消息
"""
adapter = AdminDeleteEndpointKeyAdapter(key_id=key_id)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.post("/keys/batch-delete")
async def batch_delete_endpoint_keys(
request: Request,
ids: list[str] = Body(..., embed=True, max_length=100),
db: Session = Depends(get_db),
) -> dict:
"""
批量删除 Provider Keys
一次性删除多个 Key,按 Provider 聚合执行副作用(缓存失效、模型关联检查),
避免逐个删除导致的重复 Redis 操作和性能问题。
**请求体字段**:
- `ids`: Key ID 列表(最多 100 个)
**返回字段**:
- `success_count`: 成功删除的数量
- `failed_count`: 失败的数量
- `failed`: 失败的详情列表
"""
adapter = AdminBatchDeleteEndpointKeysAdapter(ids=ids)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.post("/keys/{key_id}/clear-oauth-invalid")
async def clear_oauth_invalid(
key_id: str,
request: Request,
db: Session = Depends(get_db),
_: User = Depends(require_admin),
) -> dict:
"""
清除 Key 的 OAuth 失效标记
手动清除指定 Key 的 oauth_invalid_at / oauth_invalid_reason 状态,
通常在管理员确认账号已完成验证后使用。
这是 admin/status 维修入口,不是 AI 运行时请求恢复路径。
Rust 热路径迁移完成后,这里仍负责人工解除 OAuth invalid 标记。
**路径参数**:
- `key_id`: Key ID
**返回字段**:
- `message`: 操作结果消息
"""
adapter = AdminClearOAuthInvalidAdapter(key_id=key_id)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
# ========== Provider Keys API ==========
@router.get("/providers/{provider_id}/keys", response_model=list[EndpointAPIKeyResponse])
async def list_provider_keys(
provider_id: str,
request: Request,
skip: int = Query(0, ge=0, description="跳过的记录数"),
limit: int = Query(100, ge=1, le=1000, description="返回的最大记录数"),
db: Session = Depends(get_db),
) -> list[EndpointAPIKeyResponse]:
"""
获取 Provider 的所有 Keys
获取指定 Provider 下的所有 API Key 列表,支持多 API 格式。
结果按优先级和创建时间排序。
**路径参数**:
- `provider_id`: Provider ID
**查询参数**:
- `skip`: 跳过的记录数,用于分页(默认 0)
- `limit`: 返回的最大记录数(1-1000,默认 100)
"""
adapter = AdminListProviderKeysAdapter(
provider_id=provider_id,
skip=skip,
limit=limit,
)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.post("/providers/{provider_id}/keys", response_model=EndpointAPIKeyResponse)
async def add_provider_key(
provider_id: str,
key_data: EndpointAPIKeyCreate,
request: Request,
db: Session = Depends(get_db),
) -> EndpointAPIKeyResponse:
"""
为 Provider 添加 Key
为指定 Provider 添加新的 API Key,支持配置多个 API 格式。
**路径参数**:
- `provider_id`: Provider ID
**请求体字段**:
- `api_formats`: 支持的 API 格式列表(必填)
- `api_key`: API Key 原文(将被加密存储)
- `name`: Key 名称
- 其他配置字段同 Key
"""
adapter = AdminCreateProviderKeyAdapter(provider_id=provider_id, key_data=key_data)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
# -------- Adapters --------
@dataclass
class AdminUpdateEndpointKeyAdapter(AdminApiAdapter):
key_id: str
key_data: EndpointAPIKeyUpdate
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
return await update_endpoint_key_response(
db=context.db,
key_id=self.key_id,
key_data=self.key_data,
)
@dataclass
class AdminRevealEndpointKeyAdapter(AdminApiAdapter):
"""获取完整的 API Key 或 Auth Config(用于查看和复制)"""
key_id: str
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
return reveal_endpoint_key_payload(context.db, self.key_id)
@dataclass
class AdminExportKeyAdapter(AdminApiAdapter):
"""导出 OAuth Key 凭据:解密 auth_config,委托 provider-specific builder 构建导出数据。"""
key_id: str
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
return export_oauth_key_data(context.db, self.key_id)
@dataclass
class AdminDeleteEndpointKeyAdapter(AdminApiAdapter):
key_id: str
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
return await delete_endpoint_key_response(db=context.db, key_id=self.key_id)
@dataclass
class AdminBatchDeleteEndpointKeysAdapter(AdminApiAdapter):
"""批量删除多个 Provider Key"""
ids: list[str]
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
return await batch_delete_endpoint_keys_response(db=context.db, key_ids=self.ids)
@dataclass
class AdminClearOAuthInvalidAdapter(AdminApiAdapter):
"""清除 Key 的 OAuth 失效标记。"""
key_id: str
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
return clear_oauth_invalid_response(context.db, self.key_id)
class AdminGetKeysGroupedByFormatAdapter(AdminApiAdapter):
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
return query_keys_grouped_by_format(context.db)
# ========== Adapters ==========
@dataclass
class AdminListProviderKeysAdapter(AdminApiAdapter):
"""获取 Provider 的所有 Keys"""
provider_id: str
skip: int
limit: int
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
return list_provider_keys_responses(context.db, self.provider_id, self.skip, self.limit)
@dataclass
class AdminCreateProviderKeyAdapter(AdminApiAdapter):
"""为 Provider 添加 Key"""
provider_id: str
key_data: EndpointAPIKeyCreate
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
return await create_provider_key_response(
db=context.db,
provider_id=self.provider_id,
key_data=self.key_data,
)
# ========== Quota Refresh API ==========
class RefreshProviderQuotaRequest(BaseModel):
key_ids: list[str] | None = Field(default=None, description="仅刷新指定 Key 列表(可选)")
@router.post("/providers/{provider_id}/refresh-quota")
async def refresh_provider_quota(
provider_id: str,
request: Request,
payload: RefreshProviderQuotaRequest | None = None,
db: Session = Depends(get_db),
) -> dict:
"""
刷新 Provider 所有 Keys 的限额信息
支持的 Provider 类型:
- Codex: 调用 wham/usage API 获取限额
- Antigravity: 调用 fetchAvailableModels 获取配额
- Kiro: 调用 getUsageLimits API 获取使用额度
**路径参数**:
- `provider_id`: Provider ID
**请求体**(可选):
- `key_ids`: 仅刷新指定 Key 列表,不传时刷新所有活跃 Key
**返回字段**:
- `success`: 成功刷新的 Key 数量
- `failed`: 失败的 Key 数量
- `results`: 每个 Key 的刷新结果
"""
adapter = AdminRefreshProviderQuotaAdapter(
provider_id=provider_id,
key_ids=payload.key_ids if payload else None,
)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@dataclass
class AdminRefreshProviderQuotaAdapter(AdminApiAdapter):
"""刷新 Provider 所有 Keys 的限额信息"""
provider_id: str
key_ids: list[str] | None = None
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
return await refresh_provider_quota_for_provider(
db=context.db,
provider_id=self.provider_id,
codex_wham_usage_url=_CODEX_WHAM_USAGE_URL,
key_ids=self.key_ids,
)
@@ -1,637 +0,0 @@
"""
ProviderEndpoint CRUD 管理 API
"""
from __future__ import annotations
import uuid
from dataclasses import dataclass
from datetime import datetime, timezone
from typing import Any
from fastapi import APIRouter, Depends, Query, Request
from sqlalchemy import and_
from sqlalchemy.orm import Session
from sqlalchemy.orm.attributes import flag_modified
from src.api.base.admin_adapter import AdminApiAdapter
from src.api.base.context import ApiRequestContext
from src.api.base.models_service import invalidate_models_list_cache
from src.api.base.pipeline import get_pipeline
from src.core.api_format.metadata import get_default_body_rules_for_endpoint
from src.core.api_format.signature import parse_signature_key
from src.core.exceptions import InvalidRequestException, NotFoundException
from src.core.logger import logger
from src.core.provider_templates.fixed_providers import FIXED_PROVIDERS
from src.core.provider_types import ProviderType
from src.database import get_db
from src.models.database import Provider, ProviderAPIKey, ProviderEndpoint
from src.models.endpoint_models import (
ProviderEndpointCreate,
ProviderEndpointResponse,
ProviderEndpointUpdate,
)
from src.services.provider.stream_policy import UpstreamStreamPolicy, parse_upstream_stream_policy
router = APIRouter(tags=["Endpoint Management"])
pipeline = get_pipeline()
def mask_proxy_password(proxy_config: dict | None) -> dict | None:
"""对代理配置中的密码进行脱敏处理"""
if not proxy_config:
return None
masked = dict(proxy_config)
if masked.get("password"):
masked["password"] = "***"
return masked
def _is_fixed_provider(provider_type: str | None) -> bool:
"""Whether this provider_type is managed by fixed-provider templates."""
normalized = (provider_type or "custom").strip().lower()
if normalized == ProviderType.CUSTOM.value:
return False
try:
return ProviderType(normalized) in FIXED_PROVIDERS
except Exception:
return False
@router.get("/providers/{provider_id}/endpoints", response_model=list[ProviderEndpointResponse])
async def list_provider_endpoints(
provider_id: str,
request: Request,
skip: int = Query(0, ge=0, description="跳过的记录数"),
limit: int = Query(100, ge=1, le=1000, description="返回的最大记录数"),
db: Session = Depends(get_db),
) -> list[ProviderEndpointResponse]:
"""
获取指定 Provider 的所有 Endpoints
获取指定 Provider 下的所有 Endpoint 列表,包括配置、统计信息等。
结果按创建时间倒序排列。
**路径参数**:
- `provider_id`: Provider ID
**查询参数**:
- `skip`: 跳过的记录数,用于分页(默认 0)
- `limit`: 返回的最大记录数(1-1000,默认 100)
**返回字段**:
- `id`: Endpoint ID
- `provider_id`: Provider ID
- `provider_name`: Provider 名称
- `api_format`: API 格式
- `base_url`: 基础 URL
- `custom_path`: 自定义路径
- `max_retries`: 最大重试次数
- `is_active`: 是否活跃
- `total_keys`: Key 总数
- `active_keys`: 活跃 Key 数量
- `proxy`: 代理配置(密码已脱敏)
- 其他配置字段
"""
adapter = AdminListProviderEndpointsAdapter(
provider_id=provider_id,
skip=skip,
limit=limit,
)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.post("/providers/{provider_id}/endpoints", response_model=ProviderEndpointResponse)
async def create_provider_endpoint(
provider_id: str,
endpoint_data: ProviderEndpointCreate,
request: Request,
db: Session = Depends(get_db),
) -> ProviderEndpointResponse:
"""
为 Provider 创建新的 Endpoint
为指定 Provider 创建新的 Endpoint,每个 Provider 的每种 API 格式
只能创建一个 Endpoint。
**路径参数**:
- `provider_id`: Provider ID
**请求体字段**:
- `provider_id`: Provider ID(必须与路径参数一致)
- `api_format`: API 格式(如 claude、openai、gemini 等)
- `base_url`: 基础 URL
- `custom_path`: 自定义路径(可选)
- `header_rules`: 请求头规则列表(可选,支持 set/drop/rename 操作)
- `max_retries`: 最大重试次数(默认 2)
- `config`: 额外配置(可选)
- `proxy`: 代理配置(可选)
**返回字段**:
- 包含完整的 Endpoint 信息
"""
adapter = AdminCreateProviderEndpointAdapter(
provider_id=provider_id,
endpoint_data=endpoint_data,
)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.get("/defaults/{api_format}/body-rules")
async def get_default_endpoint_body_rules(
api_format: str,
request: Request,
provider_type: str | None = None,
db: Session = Depends(get_db),
) -> dict[str, Any]:
"""获取指定 endpoint signature 的默认 body_rules。"""
adapter = AdminGetDefaultBodyRulesAdapter(
api_format=api_format, provider_type=provider_type or None
)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.get("/{endpoint_id}", response_model=ProviderEndpointResponse)
async def get_endpoint(
endpoint_id: str,
request: Request,
db: Session = Depends(get_db),
) -> ProviderEndpointResponse:
"""
获取 Endpoint 详情
获取指定 Endpoint 的详细信息,包括配置、统计信息等。
**路径参数**:
- `endpoint_id`: Endpoint ID
**返回字段**:
- `id`: Endpoint ID
- `provider_id`: Provider ID
- `provider_name`: Provider 名称
- `api_format`: API 格式
- `base_url`: 基础 URL
- `custom_path`: 自定义路径
- `max_retries`: 最大重试次数
- `is_active`: 是否活跃
- `total_keys`: Key 总数
- `active_keys`: 活跃 Key 数量
- `proxy`: 代理配置(密码已脱敏)
- 其他配置字段
"""
adapter = AdminGetProviderEndpointAdapter(endpoint_id=endpoint_id)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.put("/{endpoint_id}", response_model=ProviderEndpointResponse)
async def update_endpoint(
endpoint_id: str,
endpoint_data: ProviderEndpointUpdate,
request: Request,
db: Session = Depends(get_db),
) -> ProviderEndpointResponse:
"""
更新 Endpoint
更新指定 Endpoint 的配置。支持部分更新。
**路径参数**:
- `endpoint_id`: Endpoint ID
**请求体字段**(均为可选):
- `base_url`: 基础 URL
- `custom_path`: 自定义路径
- `header_rules`: 请求头规则列表
- `max_retries`: 最大重试次数
- `is_active`: 是否活跃
- `config`: 额外配置
- `proxy`: 代理配置(设置为 null 可清除代理)
**返回字段**:
- 包含更新后的完整 Endpoint 信息
"""
adapter = AdminUpdateProviderEndpointAdapter(
endpoint_id=endpoint_id,
endpoint_data=endpoint_data,
)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.delete("/{endpoint_id}")
async def delete_endpoint(
endpoint_id: str,
request: Request,
db: Session = Depends(get_db),
) -> dict:
"""
删除 Endpoint
删除指定的 Endpoint,会影响该 Provider 在该 API 格式下的路由能力。
Key 不会被删除,但包含该 API 格式的 Key 将无法被调度使用(直到重新创建该格式的 Endpoint)。
**路径参数**:
- `endpoint_id`: Endpoint ID
**返回字段**:
- `message`: 操作结果消息
- `affected_keys_count`: 受影响的 Key 数量(包含该 API 格式)
"""
adapter = AdminDeleteProviderEndpointAdapter(endpoint_id=endpoint_id)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
# -------- Adapters --------
@dataclass
class AdminListProviderEndpointsAdapter(AdminApiAdapter):
provider_id: str
skip: int
limit: int
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
db = context.db
provider = db.query(Provider).filter(Provider.id == self.provider_id).first()
if not provider:
raise NotFoundException(f"Provider {self.provider_id} 不存在")
endpoints = (
db.query(ProviderEndpoint)
.filter(ProviderEndpoint.provider_id == self.provider_id)
.order_by(ProviderEndpoint.created_at.desc())
.offset(self.skip)
.limit(self.limit)
.all()
)
# Key 是 Provider 级别资源:按 key.api_formats 归类到各 Endpoint.api_format 下
keys = (
db.query(ProviderAPIKey.api_formats, ProviderAPIKey.is_active)
.filter(ProviderAPIKey.provider_id == self.provider_id)
.all()
)
total_keys_map: dict[str, int] = {}
active_keys_map: dict[str, int] = {}
for api_formats, is_active in keys:
for fmt in api_formats or []:
total_keys_map[fmt] = total_keys_map.get(fmt, 0) + 1
if is_active:
active_keys_map[fmt] = active_keys_map.get(fmt, 0) + 1
result: list[ProviderEndpointResponse] = []
for endpoint in endpoints:
endpoint_format = (
endpoint.api_format
if isinstance(endpoint.api_format, str)
else endpoint.api_format.value
)
endpoint_dict = {
**endpoint.__dict__,
"provider_name": provider.name,
"api_format": endpoint.api_format,
"total_keys": total_keys_map.get(endpoint_format, 0),
"active_keys": active_keys_map.get(endpoint_format, 0),
"proxy": mask_proxy_password(endpoint.proxy),
}
endpoint_dict.pop("_sa_instance_state", None)
result.append(ProviderEndpointResponse(**endpoint_dict))
return result
@dataclass
class AdminCreateProviderEndpointAdapter(AdminApiAdapter):
provider_id: str
endpoint_data: ProviderEndpointCreate
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
db = context.db
provider = db.query(Provider).filter(Provider.id == self.provider_id).first()
if not provider:
raise NotFoundException(f"Provider {self.provider_id} 不存在")
# 固定类型 Provider:禁止通过该接口新增 Endpoints(端点由模板自动创建并锁定)
provider_type = getattr(provider, "provider_type", None) or "custom"
if _is_fixed_provider(provider_type):
raise InvalidRequestException("固定类型 Provider 不允许手动新增 Endpoint")
if self.endpoint_data.provider_id != self.provider_id:
raise InvalidRequestException("provider_id 不匹配")
existing = (
db.query(ProviderEndpoint)
.filter(
and_(
ProviderEndpoint.provider_id == self.provider_id,
ProviderEndpoint.api_format == self.endpoint_data.api_format,
)
)
.first()
)
if existing:
raise InvalidRequestException(
f"Provider {provider.name} 已存在 {self.endpoint_data.api_format} 格式的 Endpoint"
)
now = datetime.now(timezone.utc)
sig = parse_signature_key(self.endpoint_data.api_format)
api_family = sig.api_family.value
endpoint_kind = sig.endpoint_kind.value
# 使用归一化后的 signature key,确保格式一致性
normalized_api_format = sig.key
body_rules = self.endpoint_data.body_rules
if body_rules is None:
body_rules = (
get_default_body_rules_for_endpoint(
normalized_api_format, provider_type=provider_type
)
or None
)
new_endpoint = ProviderEndpoint(
id=str(uuid.uuid4()),
provider_id=self.provider_id,
api_format=normalized_api_format,
api_family=api_family,
endpoint_kind=endpoint_kind,
base_url=self.endpoint_data.base_url,
custom_path=self.endpoint_data.custom_path,
header_rules=self.endpoint_data.header_rules,
body_rules=body_rules,
max_retries=self.endpoint_data.max_retries,
is_active=True,
config=self.endpoint_data.config,
proxy=self.endpoint_data.proxy.model_dump() if self.endpoint_data.proxy else None,
format_acceptance_config=self.endpoint_data.format_acceptance_config,
created_at=now,
updated_at=now,
)
db.add(new_endpoint)
db.commit()
db.refresh(new_endpoint)
# 清除 /v1/models 列表缓存
await invalidate_models_list_cache()
logger.info(
f"[OK] 创建 Endpoint: Provider={provider.name}, Format={self.endpoint_data.api_format}, ID={new_endpoint.id}"
)
endpoint_dict = {
k: v
for k, v in new_endpoint.__dict__.items()
if k not in {"api_format", "_sa_instance_state", "proxy"}
}
return ProviderEndpointResponse(
**endpoint_dict,
provider_name=provider.name,
api_format=new_endpoint.api_format,
proxy=mask_proxy_password(new_endpoint.proxy),
total_keys=0,
active_keys=0,
)
@dataclass
class AdminGetProviderEndpointAdapter(AdminApiAdapter):
endpoint_id: str
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
db = context.db
endpoint = (
db.query(ProviderEndpoint, Provider)
.join(Provider, ProviderEndpoint.provider_id == Provider.id)
.filter(ProviderEndpoint.id == self.endpoint_id)
.first()
)
if not endpoint:
raise NotFoundException(f"Endpoint {self.endpoint_id} 不存在")
endpoint_obj, provider = endpoint
endpoint_format = (
endpoint_obj.api_format
if isinstance(endpoint_obj.api_format, str)
else endpoint_obj.api_format.value
)
keys = (
db.query(ProviderAPIKey.api_formats, ProviderAPIKey.is_active)
.filter(ProviderAPIKey.provider_id == endpoint_obj.provider_id)
.all()
)
total_keys = 0
active_keys = 0
for api_formats, is_active in keys:
if endpoint_format in (api_formats or []):
total_keys += 1
if is_active:
active_keys += 1
endpoint_dict = {
k: v
for k, v in endpoint_obj.__dict__.items()
if k not in {"api_format", "_sa_instance_state", "proxy"}
}
return ProviderEndpointResponse(
**endpoint_dict,
provider_name=provider.name,
api_format=endpoint_obj.api_format,
proxy=mask_proxy_password(endpoint_obj.proxy),
total_keys=total_keys,
active_keys=active_keys,
)
@dataclass
class AdminUpdateProviderEndpointAdapter(AdminApiAdapter):
endpoint_id: str
endpoint_data: ProviderEndpointUpdate
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
db = context.db
endpoint = (
db.query(ProviderEndpoint).filter(ProviderEndpoint.id == self.endpoint_id).first()
)
if not endpoint:
raise NotFoundException(f"Endpoint {self.endpoint_id} 不存在")
update_data = self.endpoint_data.model_dump(exclude_unset=True)
# 固定类型 Provider 的 endpoint:锁定 base_url/custom_path(前端禁用仅是 UX,后端必须强校验)
provider = db.query(Provider).filter(Provider.id == endpoint.provider_id).first()
if provider:
provider_type = getattr(provider, "provider_type", "custom")
if _is_fixed_provider(provider_type):
if "base_url" in update_data or "custom_path" in update_data:
raise InvalidRequestException(
"固定类型 Provider 的 Endpoint 不允许修改 base_url/custom_path"
)
normalized_provider_type = str(provider_type or "custom").strip().lower()
endpoint_sig = str(getattr(endpoint, "api_format", "") or "").strip().lower()
if (
normalized_provider_type == ProviderType.CODEX.value
and endpoint_sig == "openai:cli"
):
has_config_in_payload = "config" in update_data
cfg_payload = (
update_data.get("config")
if has_config_in_payload
else getattr(endpoint, "config", None)
)
cfg = dict(cfg_payload) if isinstance(cfg_payload, dict) else {}
requested = (
cfg.get("upstream_stream_policy")
or cfg.get("upstreamStreamPolicy")
or cfg.get("upstream_stream")
)
if (
has_config_in_payload
and requested is not None
and parse_upstream_stream_policy(requested)
!= UpstreamStreamPolicy.FORCE_STREAM
):
raise InvalidRequestException(
"Codex OpenAI CLI 端点固定为强制流式,不允许修改"
)
cfg.pop("upstreamStreamPolicy", None)
cfg.pop("upstream_stream", None)
cfg["upstream_stream_policy"] = "force_stream"
update_data["config"] = cfg
# 把 proxy 转换为 dict 存储,支持显式设置为 None 清除代理
if "proxy" in update_data:
if update_data["proxy"] is not None:
new_proxy = dict(update_data["proxy"])
# 只有当密码字段未提供时才保留原密码(空字符串视为显式清除)
if "password" not in new_proxy and endpoint.proxy:
old_password = endpoint.proxy.get("password")
if old_password:
new_proxy["password"] = old_password
update_data["proxy"] = new_proxy
# proxy 为 None 时保留,用于清除代理配置
# JSON 列需要 flag_modified 以确保 SQLAlchemy 检测到变更
json_fields = {"header_rules", "body_rules", "config", "proxy", "format_acceptance_config"}
for field, value in update_data.items():
setattr(endpoint, field, value)
if field in json_fields:
flag_modified(endpoint, field)
# Phase 3/4: 自动维护新架构字段,确保新增/历史数据都能被调度器按 family/kind 查询
sig = parse_signature_key(endpoint.api_format)
endpoint.api_family = sig.api_family.value
endpoint.endpoint_kind = sig.endpoint_kind.value
endpoint.updated_at = datetime.now(timezone.utc)
db.commit()
db.refresh(endpoint)
# 清除 /v1/models 列表缓存(is_active 变更会影响模型可用性)
await invalidate_models_list_cache()
provider = db.query(Provider).filter(Provider.id == endpoint.provider_id).first()
logger.info(
f"[OK] 更新 Endpoint: ID={self.endpoint_id}, Updates={list(update_data.keys())}"
)
endpoint_format = (
endpoint.api_format
if isinstance(endpoint.api_format, str)
else endpoint.api_format.value
)
keys = (
db.query(ProviderAPIKey.api_formats, ProviderAPIKey.is_active)
.filter(ProviderAPIKey.provider_id == endpoint.provider_id)
.all()
)
total_keys = 0
active_keys = 0
for api_formats, is_active in keys:
if endpoint_format in (api_formats or []):
total_keys += 1
if is_active:
active_keys += 1
endpoint_dict = {
k: v
for k, v in endpoint.__dict__.items()
if k not in {"api_format", "_sa_instance_state", "proxy"}
}
return ProviderEndpointResponse(
**endpoint_dict,
provider_name=provider.name if provider else "Unknown",
api_format=endpoint.api_format,
proxy=mask_proxy_password(endpoint.proxy),
total_keys=total_keys,
active_keys=active_keys,
)
@dataclass
class AdminDeleteProviderEndpointAdapter(AdminApiAdapter):
endpoint_id: str
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
db = context.db
endpoint = (
db.query(ProviderEndpoint).filter(ProviderEndpoint.id == self.endpoint_id).first()
)
if not endpoint:
raise NotFoundException(f"Endpoint {self.endpoint_id} 不存在")
endpoint_format = (
endpoint.api_format
if isinstance(endpoint.api_format, str)
else endpoint.api_format.value
)
# 查询包含该格式的所有 Key,并从 api_formats 中移除该格式
keys = (
db.query(ProviderAPIKey)
.filter(ProviderAPIKey.provider_id == endpoint.provider_id)
.all()
)
affected_keys_count = 0
for key in keys:
if key.api_formats and endpoint_format in key.api_formats:
affected_keys_count += 1
# 移除该格式
new_formats = [f for f in key.api_formats if f != endpoint_format]
key.api_formats = new_formats if new_formats else []
flag_modified(key, "api_formats")
db.delete(endpoint)
db.commit()
# 清除 /v1/models 列表缓存
await invalidate_models_list_cache()
logger.warning(
f"[DELETE] 删除 Endpoint: ID={self.endpoint_id}, Format={endpoint_format}, "
f"AffectedKeys={affected_keys_count}"
)
return {
"message": f"Endpoint {self.endpoint_id} 已删除",
"affected_keys_count": affected_keys_count,
}
@dataclass
class AdminGetDefaultBodyRulesAdapter(AdminApiAdapter):
api_format: str
provider_type: str | None = None
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
try:
normalized_api_format = parse_signature_key(self.api_format).key
except Exception as exc:
raise InvalidRequestException(f"无效的 api_format: {self.api_format}") from exc
return {
"api_format": normalized_api_format,
"body_rules": get_default_body_rules_for_endpoint(
normalized_api_format, provider_type=self.provider_type
),
}
@@ -1,383 +0,0 @@
"""
Gemini Files 管理 API
提供文件映射管理与能力查询;上传入口已收成 Rust-only 兼容壳。
"""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timezone
from typing import Any
from fastapi import APIRouter, Depends, File, HTTPException, Query, Request, UploadFile
from pydantic import BaseModel
from sqlalchemy import delete, func
from sqlalchemy.orm import Session, load_only
from src.api.base.admin_adapter import AdminApiAdapter
from src.api.base.context import ApiRequestContext
from src.api.base.pipeline import get_pipeline
from src.database import get_db
from src.models.database import GeminiFileMapping, ProviderAPIKey, User
from src.services.gemini_files_mapping import delete_file_key_mapping
router = APIRouter(prefix="/api/admin/gemini-files", tags=["Gemini Files Management"])
pipeline = get_pipeline()
_RUST_UPLOADER_DETAIL = "Admin Gemini file upload requires Rust uploader"
class FileMappingResponse(BaseModel):
id: str
file_name: str
key_id: str
key_name: str | None = None
user_id: str | None = None
username: str | None = None
display_name: str | None = None
mime_type: str | None = None
created_at: datetime
expires_at: datetime
is_expired: bool
class FileMappingListResponse(BaseModel):
items: list[FileMappingResponse]
total: int
page: int
page_size: int
class FileMappingStatsResponse(BaseModel):
total_mappings: int
active_mappings: int
expired_mappings: int
by_mime_type: dict[str, int]
capable_keys_count: int
class CapableKeyResponse(BaseModel):
id: str
name: str
provider_name: str | None = None
class UploadResultItem(BaseModel):
key_id: str
key_name: str | None = None
success: bool
file_name: str | None = None
error: str | None = None
class UploadResponse(BaseModel):
display_name: str
mime_type: str
size_bytes: int
results: list[UploadResultItem]
success_count: int
fail_count: int
async def _list_file_mappings_response(
*,
db: Session,
page: int,
page_size: int,
include_expired: bool,
search: str | None,
) -> FileMappingListResponse:
now = datetime.now(timezone.utc)
query = db.query(GeminiFileMapping)
count_query = db.query(func.count(GeminiFileMapping.id))
if not include_expired:
active_filter = GeminiFileMapping.expires_at > now
query = query.filter(active_filter)
count_query = count_query.filter(active_filter)
if search:
search_pattern = f"%{search}%"
search_filter = (GeminiFileMapping.file_name.ilike(search_pattern)) | (
GeminiFileMapping.display_name.ilike(search_pattern)
)
query = query.filter(search_filter)
count_query = count_query.filter(search_filter)
total = int(count_query.scalar() or 0)
offset = (page - 1) * page_size
mappings = (
query.options(
load_only(
GeminiFileMapping.id,
GeminiFileMapping.file_name,
GeminiFileMapping.key_id,
GeminiFileMapping.user_id,
GeminiFileMapping.display_name,
GeminiFileMapping.mime_type,
GeminiFileMapping.created_at,
GeminiFileMapping.expires_at,
)
)
.order_by(GeminiFileMapping.created_at.desc())
.offset(offset)
.limit(page_size)
.all()
)
key_ids = {m.key_id for m in mappings}
user_ids = {m.user_id for m in mappings if m.user_id}
keys_map: dict[str, str | None] = {}
if key_ids:
keys = (
db.query(ProviderAPIKey)
.options(load_only(ProviderAPIKey.id, ProviderAPIKey.name))
.filter(ProviderAPIKey.id.in_(key_ids))
.all()
)
keys_map = {str(k.id): k.name for k in keys}
users_map: dict[str, str | None] = {}
if user_ids:
users = (
db.query(User)
.options(load_only(User.id, User.username))
.filter(User.id.in_(user_ids))
.all()
)
users_map = {str(u.id): u.username for u in users}
return FileMappingListResponse(
items=[
FileMappingResponse(
id=str(m.id),
file_name=m.file_name,
key_id=str(m.key_id),
key_name=keys_map.get(str(m.key_id)),
user_id=str(m.user_id) if m.user_id else None,
username=users_map.get(str(m.user_id)) if m.user_id else None,
display_name=m.display_name,
mime_type=m.mime_type,
created_at=m.created_at,
expires_at=m.expires_at,
is_expired=m.expires_at <= now,
)
for m in mappings
],
total=total,
page=page,
page_size=page_size,
)
async def _get_file_mapping_stats_response(*, db: Session) -> FileMappingStatsResponse:
now = datetime.now(timezone.utc)
total_mappings = db.query(func.count(GeminiFileMapping.id)).scalar() or 0
active_mappings = (
db.query(func.count(GeminiFileMapping.id))
.filter(GeminiFileMapping.expires_at > now)
.scalar()
or 0
)
expired_mappings = total_mappings - active_mappings
mime_stats = (
db.query(GeminiFileMapping.mime_type, func.count(GeminiFileMapping.id))
.filter(GeminiFileMapping.expires_at > now)
.group_by(GeminiFileMapping.mime_type)
.all()
)
by_mime_type = {(mime_type or "unknown"): count for mime_type, count in mime_stats}
keys = db.query(ProviderAPIKey.capabilities).filter(ProviderAPIKey.is_active.is_(True)).all()
capable_keys_count = sum(
1
for (capabilities,) in keys
if isinstance(capabilities, dict) and capabilities.get("gemini_files", False)
)
return FileMappingStatsResponse(
total_mappings=total_mappings,
active_mappings=active_mappings,
expired_mappings=expired_mappings,
by_mime_type=by_mime_type,
capable_keys_count=capable_keys_count,
)
async def _delete_mapping_response(*, db: Session, mapping_id: str) -> dict[str, Any]:
mapping = db.query(GeminiFileMapping).filter(GeminiFileMapping.id == mapping_id).first()
if not mapping:
raise HTTPException(status_code=404, detail="Mapping not found")
file_name = mapping.file_name
db.delete(mapping)
db.commit()
await delete_file_key_mapping(file_name)
return {"message": "Mapping deleted successfully", "file_name": file_name}
async def _cleanup_expired_mappings_response(*, db: Session) -> dict[str, Any]:
now = datetime.now(timezone.utc)
result = db.execute(delete(GeminiFileMapping).where(GeminiFileMapping.expires_at <= now))
db.commit()
deleted_count = result.rowcount
return {
"message": f"Cleaned up {deleted_count} expired mappings",
"deleted_count": deleted_count,
}
async def _list_capable_keys_response(*, db: Session) -> list[CapableKeyResponse]:
from src.models.database import Provider
key_rows = (
db.query(
ProviderAPIKey.id,
ProviderAPIKey.name,
ProviderAPIKey.provider_id,
ProviderAPIKey.capabilities,
)
.filter(ProviderAPIKey.is_active.is_(True))
.all()
)
capable_keys = [
key
for key in key_rows
if isinstance(key.capabilities, dict) and key.capabilities.get("gemini_files", False)
]
provider_ids = {key.provider_id for key in capable_keys if key.provider_id}
provider_map: dict[str, str] = {}
if provider_ids:
providers = db.query(Provider.id, Provider.name).filter(Provider.id.in_(provider_ids)).all()
provider_map = {str(provider_id): provider_name for provider_id, provider_name in providers}
return [
CapableKeyResponse(
id=str(key.id),
name=key.name,
provider_name=provider_map.get(str(key.provider_id)),
)
for key in capable_keys
]
async def _upload_file_response(*, file: UploadFile, key_ids: str) -> Any:
del file, key_ids
raise HTTPException(status_code=503, detail=_RUST_UPLOADER_DETAIL)
@dataclass
class AdminGeminiFilesListMappingsAdapter(AdminApiAdapter):
page: int
page_size: int
include_expired: bool
search: str | None
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
return await _list_file_mappings_response(
db=context.db,
page=self.page,
page_size=self.page_size,
include_expired=self.include_expired,
search=self.search,
)
class AdminGeminiFilesStatsAdapter(AdminApiAdapter):
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
return await _get_file_mapping_stats_response(db=context.db)
@dataclass
class AdminGeminiFilesDeleteMappingAdapter(AdminApiAdapter):
mapping_id: str
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
return await _delete_mapping_response(db=context.db, mapping_id=self.mapping_id)
class AdminGeminiFilesCleanupMappingsAdapter(AdminApiAdapter):
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
return await _cleanup_expired_mappings_response(db=context.db)
class AdminGeminiFilesCapableKeysAdapter(AdminApiAdapter):
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
return await _list_capable_keys_response(db=context.db)
@dataclass
class AdminGeminiFilesUploadAdapter(AdminApiAdapter):
file: UploadFile
key_ids: str
async def handle(self, context: ApiRequestContext) -> Any: # type: ignore[override]
del context
return await _upload_file_response(file=self.file, key_ids=self.key_ids)
@router.get("/mappings", response_model=FileMappingListResponse)
async def list_file_mappings(
request: Request,
db: Session = Depends(get_db),
page: int = Query(1, ge=1),
page_size: int = Query(20, ge=1, le=100),
include_expired: bool = Query(False),
search: str | None = Query(None),
) -> Any:
adapter = AdminGeminiFilesListMappingsAdapter(
page=page,
page_size=page_size,
include_expired=include_expired,
search=search,
)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.get("/stats", response_model=FileMappingStatsResponse)
async def get_file_mapping_stats(
request: Request,
db: Session = Depends(get_db),
) -> Any:
adapter = AdminGeminiFilesStatsAdapter()
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.delete("/mappings/{mapping_id}")
async def delete_mapping(
mapping_id: str,
request: Request,
db: Session = Depends(get_db),
) -> Any:
adapter = AdminGeminiFilesDeleteMappingAdapter(mapping_id=mapping_id)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.delete("/mappings")
async def cleanup_expired_mappings(
request: Request,
db: Session = Depends(get_db),
) -> Any:
adapter = AdminGeminiFilesCleanupMappingsAdapter()
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.get("/capable-keys", response_model=list[CapableKeyResponse])
async def list_capable_keys(
request: Request,
db: Session = Depends(get_db),
) -> Any:
adapter = AdminGeminiFilesCapableKeysAdapter()
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.post("/upload", response_model=UploadResponse)
async def upload_file(
request: Request,
file: UploadFile = File(...),
key_ids: str = Query(..., description="逗号分隔的 Key ID 列表"),
db: Session = Depends(get_db),
) -> Any:
adapter = AdminGeminiFilesUploadAdapter(file=file, key_ids=key_ids)
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
-504
View File
@@ -1,504 +0,0 @@
"""LDAP配置管理API端点。"""
from __future__ import annotations
import re
from typing import Any
from fastapi import APIRouter, Depends, Request
from pydantic import BaseModel, Field, ValidationError, field_validator
from sqlalchemy.orm import Session
from src.api.base.admin_adapter import AdminApiAdapter
from src.api.base.context import ApiRequestContext
from src.api.base.pipeline import get_pipeline
from src.core.crypto import crypto_service
from src.core.enums import AuthSource
from src.core.exceptions import InvalidRequestException, translate_pydantic_error
from src.core.logger import logger
from src.database import get_db
from src.models.database import AuditEventType, LDAPConfig, User, UserRole
from src.services.system.audit import AuditService
router = APIRouter(prefix="/api/admin/ldap", tags=["Admin - LDAP"])
pipeline = get_pipeline()
# bcrypt 哈希格式正则:$2a$, $2b$, $2y$ + 2位cost + $ + 53字符(22位salt + 31位hash)
BCRYPT_HASH_PATTERN = re.compile(r"^\$2[aby]\$\d{2}\$.{53}$")
# ========== Request/Response Models ==========
class LDAPConfigResponse(BaseModel):
"""LDAP配置响应(不返回密码)"""
server_url: str | None = None
bind_dn: str | None = None
base_dn: str | None = None
has_bind_password: bool = False
user_search_filter: str
username_attr: str
email_attr: str
display_name_attr: str
is_enabled: bool
is_exclusive: bool
use_starttls: bool
connect_timeout: int
class LDAPConfigUpdate(BaseModel):
"""LDAP配置更新请求"""
server_url: str = Field(..., min_length=1, max_length=255)
bind_dn: str = Field(..., min_length=1, max_length=255)
# 允许空字符串表示"清除密码";非空时自动 strip 并校验不能为空
bind_password: str | None = Field(None, max_length=1024)
base_dn: str = Field(..., min_length=1, max_length=255)
user_search_filter: str = Field(default="(uid={username})", max_length=500)
username_attr: str = Field(default="uid", max_length=50)
email_attr: str = Field(default="mail", max_length=50)
display_name_attr: str = Field(default="cn", max_length=50)
is_enabled: bool = False
is_exclusive: bool = False
use_starttls: bool = False
connect_timeout: int = Field(default=10, ge=1, le=60) # 单次操作超时,跨国网络建议 15-30 秒
@field_validator("bind_password")
@classmethod
def validate_bind_password(cls, v: str | None) -> str | None:
if v is None or v == "":
return v
v = v.strip()
if not v:
raise ValueError("绑定密码不能为空")
return v
@field_validator("user_search_filter")
@classmethod
def validate_search_filter(cls, v: str) -> str:
if "{username}" not in v:
raise ValueError("搜索过滤器必须包含 {username} 占位符")
# 验证括号匹配和嵌套正确性
depth = 0
for char in v:
if char == "(":
depth += 1
elif char == ")":
depth -= 1
if depth < 0:
raise ValueError("搜索过滤器括号不匹配")
if depth != 0:
raise ValueError("搜索过滤器括号不匹配")
# 限制过滤器复杂度,防止构造复杂查询
# 检查嵌套层数而非括号总数
depth = 0
max_depth = 0
for char in v:
if char == "(":
depth += 1
max_depth = max(max_depth, depth)
elif char == ")":
depth -= 1
if max_depth > 5:
raise ValueError("搜索过滤器嵌套层数过深(最多5层)")
if len(v) > 200:
raise ValueError("搜索过滤器过长(最多200字符)")
return v
class LDAPTestResponse(BaseModel):
"""LDAP连接测试响应"""
success: bool
message: str
class LDAPConfigTest(BaseModel):
"""LDAP配置测试请求(全部可选,用于临时覆盖)"""
server_url: str | None = Field(None, min_length=1, max_length=255)
bind_dn: str | None = Field(None, min_length=1, max_length=255)
bind_password: str | None = Field(None, min_length=1)
base_dn: str | None = Field(None, min_length=1, max_length=255)
user_search_filter: str | None = Field(None, max_length=500)
username_attr: str | None = Field(None, max_length=50)
email_attr: str | None = Field(None, max_length=50)
display_name_attr: str | None = Field(None, max_length=50)
is_enabled: bool | None = None
is_exclusive: bool | None = None
use_starttls: bool | None = None
connect_timeout: int | None = Field(None, ge=1, le=60)
@field_validator("user_search_filter")
@classmethod
def validate_search_filter(cls, v: str | None) -> str | None:
if v is None:
return v
if "{username}" not in v:
raise ValueError("搜索过滤器必须包含 {username} 占位符")
# 验证括号匹配和嵌套正确性
depth = 0
for char in v:
if char == "(":
depth += 1
elif char == ")":
depth -= 1
if depth < 0:
raise ValueError("搜索过滤器括号不匹配")
if depth != 0:
raise ValueError("搜索过滤器括号不匹配")
# 限制过滤器复杂度(检查嵌套层数而非括号总数)
depth = 0
max_depth = 0
for char in v:
if char == "(":
depth += 1
max_depth = max(max_depth, depth)
elif char == ")":
depth -= 1
if max_depth > 5:
raise ValueError("搜索过滤器嵌套层数过深(最多5层)")
if len(v) > 200:
raise ValueError("搜索过滤器过长(最多200字符)")
return v
# ========== API Endpoints ==========
@router.get("/config")
async def get_ldap_config(request: Request, db: Session = Depends(get_db)) -> Any:
"""
获取 LDAP 配置
获取系统当前的 LDAP 认证配置信息,用于管理界面显示和编辑。
密码字段不会返回原文,仅返回是否已设置的标志。
**返回字段**:
- `server_url`: LDAP 服务器地址(如:ldap://ldap.example.com:389)
- `bind_dn`: 绑定 DN(如:cn=admin,dc=example,dc=com)
- `base_dn`: 搜索基准 DN(如:ou=users,dc=example,dc=com)
- `has_bind_password`: 是否已设置绑定密码(布尔值)
- `user_search_filter`: 用户搜索过滤器(默认:(uid={username}))
- `username_attr`: 用户名属性(默认:uid)
- `email_attr`: 邮箱属性(默认:mail)
- `display_name_attr`: 显示名称属性(默认:cn)
- `is_enabled`: 是否启用 LDAP 认证
- `is_exclusive`: 是否仅允许 LDAP 登录(独占模式)
- `use_starttls`: 是否使用 STARTTLS 加密连接
- `connect_timeout`: 连接超时时间(秒,1-60)
"""
adapter = AdminGetLDAPConfigAdapter()
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.put("/config")
async def update_ldap_config(request: Request, db: Session = Depends(get_db)) -> Any:
"""
更新 LDAP 配置
更新系统的 LDAP 认证配置。支持完整配置更新,包括连接参数、
搜索过滤器、属性映射等。提供多重安全校验,防止误锁定管理员。
**请求体字段**:
- `server_url`: LDAP 服务器地址(必填,1-255字符)
- `bind_dn`: 绑定 DN(必填,1-255字符)
- `bind_password`: 绑定密码(可选,设为空字符串可清除密码)
- `base_dn`: 搜索基准 DN(必填,1-255字符)
- `user_search_filter`: 用户搜索过滤器(必须包含 {username} 占位符,默认:(uid={username}))
- `username_attr`: 用户名属性(默认:uid)
- `email_attr`: 邮箱属性(默认:mail)
- `display_name_attr`: 显示名称属性(默认:cn)
- `is_enabled`: 是否启用 LDAP 认证
- `is_exclusive`: 是否仅允许 LDAP 登录(需先启用 LDAP)
- `use_starttls`: 是否使用 STARTTLS 加密连接
- `connect_timeout`: 连接超时时间(秒,1-60,默认 10)
**安全校验**:
- 启用 LDAP 时必须设置有效的绑定密码
- 启用独占模式前会检查是否有至少 1 个有效的本地管理员账户
- 独占模式要求先启用 LDAP 认证
- 搜索过滤器必须包含 {username} 占位符且括号匹配
- 搜索过滤器嵌套层数不超过 5 层,长度不超过 200 字符
**返回字段**:
- `message`: 操作结果消息
"""
adapter = AdminUpdateLDAPConfigAdapter()
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
@router.post("/test")
async def test_ldap_connection(request: Request, db: Session = Depends(get_db)) -> Any:
"""
测试 LDAP 连接
在保存配置前测试 LDAP 服务器连接是否正常。支持使用已保存的配置,
也支持通过请求体覆盖任意配置项进行临时测试,而不影响已保存的配置。
**请求体字段**(均为可选,用于临时覆盖):
- `server_url`: LDAP 服务器地址(覆盖已保存的配置)
- `bind_dn`: 绑定 DN(覆盖已保存的配置)
- `bind_password`: 绑定密码(覆盖已保存的密码)
- `base_dn`: 搜索基准 DN(覆盖已保存的配置)
- `user_search_filter`: 用户搜索过滤器(覆盖已保存的配置)
- `username_attr`: 用户名属性(覆盖已保存的配置)
- `email_attr`: 邮箱属性(覆盖已保存的配置)
- `display_name_attr`: 显示名称属性(覆盖已保存的配置)
- `use_starttls`: 是否使用 STARTTLS(覆盖已保存的配置)
- `connect_timeout`: 连接超时时间(覆盖已保存的配置)
**测试逻辑**:
- 未提供的字段使用已保存的配置值
- `bind_password` 优先使用请求体中的值,否则使用已保存的加密密码
- 测试时会尝试连接 LDAP 服务器并验证绑定 DN
**返回字段**:
- `success`: 测试是否成功(布尔值)
- `message`: 测试结果消息(成功或失败原因)
"""
adapter = AdminTestLDAPConnectionAdapter()
return await pipeline.run(adapter=adapter, http_request=request, db=db, mode=adapter.mode)
# ========== Adapters ==========
class AdminGetLDAPConfigAdapter(AdminApiAdapter):
async def handle(self, context: ApiRequestContext) -> dict[str, Any]: # type: ignore[override]
db = context.db
config = db.query(LDAPConfig).first()
if not config:
return LDAPConfigResponse(
server_url=None,
bind_dn=None,
base_dn=None,
has_bind_password=False,
user_search_filter="(uid={username})",
username_attr="uid",
email_attr="mail",
display_name_attr="cn",
is_enabled=False,
is_exclusive=False,
use_starttls=False,
connect_timeout=10,
).model_dump()
return LDAPConfigResponse(
server_url=config.server_url,
bind_dn=config.bind_dn,
base_dn=config.base_dn,
has_bind_password=bool(config.bind_password_encrypted),
user_search_filter=config.user_search_filter,
username_attr=config.username_attr,
email_attr=config.email_attr,
display_name_attr=config.display_name_attr,
is_enabled=config.is_enabled,
is_exclusive=config.is_exclusive,
use_starttls=config.use_starttls,
connect_timeout=config.connect_timeout,
).model_dump()
class AdminUpdateLDAPConfigAdapter(AdminApiAdapter):
async def handle(self, context: ApiRequestContext) -> dict[str, str]: # type: ignore[override]
db = context.db
payload = context.ensure_json_body()
try:
config_update = LDAPConfigUpdate.model_validate(payload)
except ValidationError as e:
errors = e.errors()
if errors:
raise InvalidRequestException(translate_pydantic_error(errors[0]))
raise InvalidRequestException("请求数据验证失败")
# 使用行级锁防止并发修改导致的竞态条件
config = db.query(LDAPConfig).with_for_update().first()
is_new_config = config is None
if is_new_config:
# 首次创建配置时必须提供密码
if not config_update.bind_password:
raise InvalidRequestException("首次配置 LDAP 时必须设置绑定密码")
config = LDAPConfig()
db.add(config)
# 需要启用 LDAP 且未提交新密码时,验证已保存密码可解密(避免开启后不可用)
if config_update.is_enabled and config_update.bind_password is None:
try:
if not config.get_bind_password():
raise InvalidRequestException("启用 LDAP 认证 需要先设置绑定密码")
except InvalidRequestException:
raise
except Exception:
raise InvalidRequestException("绑定密码解密失败,请重新设置绑定密码")
# 计算更新后的密码状态(用于校验是否可启用/独占)
if config_update.bind_password is None:
will_have_password = bool(config.bind_password_encrypted)
elif config_update.bind_password == "":
will_have_password = False
else:
will_have_password = True
# 独占模式必须启用 LDAP 且必须有绑定密码(防止误锁定)
if config_update.is_exclusive and not config_update.is_enabled:
raise InvalidRequestException("仅允许 LDAP 登录 需要先启用 LDAP 认证")
if config_update.is_enabled and not will_have_password:
raise InvalidRequestException("启用 LDAP 认证 需要先设置绑定密码")
if config_update.is_exclusive and not will_have_password:
raise InvalidRequestException("仅允许 LDAP 登录 需要先设置绑定密码")
config.server_url = config_update.server_url
config.bind_dn = config_update.bind_dn
config.base_dn = config_update.base_dn
config.user_search_filter = config_update.user_search_filter
config.username_attr = config_update.username_attr
config.email_attr = config_update.email_attr
config.display_name_attr = config_update.display_name_attr
config.is_enabled = config_update.is_enabled
config.is_exclusive = config_update.is_exclusive
config.use_starttls = config_update.use_starttls
config.connect_timeout = config_update.connect_timeout
# 启用独占模式前检查是否有足够的本地管理员(防止锁定)
# 使用 with_for_update() 阻塞锁防止竞态条件(移除 nowait 确保并发安全)
if config_update.is_enabled and config_update.is_exclusive:
local_admins = (
db.query(User)
.filter(
User.role == UserRole.ADMIN,
User.auth_source == AuthSource.LOCAL,
User.is_active.is_(True),
User.is_deleted.is_(False),
)
.with_for_update()
.all()
)
# 验证至少有一个管理员有有效的密码哈希(可以登录)
# 使用严格的 bcrypt 格式校验:$2a$/$2b$/$2y$ + 2位cost + $ + 53字符
valid_admin_count = sum(
1
for admin in local_admins
if admin.password_hash
and isinstance(admin.password_hash, str)
and BCRYPT_HASH_PATTERN.match(admin.password_hash)
)
if valid_admin_count < 1:
raise InvalidRequestException(
"启用 LDAP 独占模式前,必须至少保留 1 个有效的本地管理员账户(含有效密码)作为紧急恢复通道"
)
if config_update.bind_password is not None:
if config_update.bind_password == "":
# 显式清除密码(设置为 NULL)
config.bind_password_encrypted = None
password_changed = "cleared"
else:
config.bind_password_encrypted = crypto_service.encrypt(config_update.bind_password)
password_changed = "updated"
else:
password_changed = None
db.commit()
# 记录审计日志
AuditService.log_event(
db=db,
event_type=AuditEventType.CONFIG_CHANGED,
description=f"LDAP 配置已更新 (enabled={config_update.is_enabled}, exclusive={config_update.is_exclusive})",
user_id=str(context.user.id) if context.user else None,
metadata={
"server_url": config_update.server_url,
"is_enabled": config_update.is_enabled,
"is_exclusive": config_update.is_exclusive,
"password_changed": password_changed,
"is_new_config": is_new_config,
},
)
return {"message": "LDAP配置更新成功"}
class AdminTestLDAPConnectionAdapter(AdminApiAdapter):
async def handle(self, context: ApiRequestContext) -> dict[str, Any]: # type: ignore[override]
from src.services.auth.ldap import LDAPService
db = context.db
if context.json_body is not None:
payload = context.json_body
elif not context.raw_body:
payload = {}
else:
payload = context.ensure_json_body()
saved_config = db.query(LDAPConfig).first()
try:
overrides = LDAPConfigTest.model_validate(payload)
except ValidationError as e:
errors = e.errors()
if errors:
raise InvalidRequestException(translate_pydantic_error(errors[0]))
raise InvalidRequestException("请求数据验证失败")
config_data: dict[str, Any] = {}
if saved_config:
config_data = {
"server_url": saved_config.server_url,
"bind_dn": saved_config.bind_dn,
"base_dn": saved_config.base_dn,
"user_search_filter": saved_config.user_search_filter,
"username_attr": saved_config.username_attr,
"email_attr": saved_config.email_attr,
"display_name_attr": saved_config.display_name_attr,
"use_starttls": saved_config.use_starttls,
"connect_timeout": saved_config.connect_timeout,
}
# 应用前端传入的覆盖值
for field in [
"server_url",
"bind_dn",
"base_dn",
"user_search_filter",
"username_attr",
"email_attr",
"display_name_attr",
"use_starttls",
"is_enabled",
"is_exclusive",
"connect_timeout",
]:
value = getattr(overrides, field)
if value is not None:
config_data[field] = value
# bind_password 优先使用 overrides;否则使用已保存的密码(允许保存密码无法解密时依然用 overrides 测试)
if overrides.bind_password is not None:
config_data["bind_password"] = overrides.bind_password
elif saved_config and saved_config.bind_password_encrypted:
try:
config_data["bind_password"] = crypto_service.decrypt(
saved_config.bind_password_encrypted
)
except Exception as e:
logger.error(f"绑定密码解密失败: {type(e).__name__}: {e}")
return LDAPTestResponse(
success=False, message="绑定密码解密失败,请检查配置或重新设置密码"
).model_dump()
# 必填字段检查
required_fields = ["server_url", "bind_dn", "base_dn", "bind_password"]
missing = [f for f in required_fields if not config_data.get(f)]
if missing:
return LDAPTestResponse(
success=False, message=f"缺少必要字段: {', '.join(missing)}"
).model_dump()
success, message = LDAPService.test_connection_with_config(config_data)
return LDAPTestResponse(success=success, message=message).model_dump()
@@ -1,10 +0,0 @@
"""Management Token 管理员路由模块"""
from fastapi import APIRouter
from .routes import router as management_tokens_router
router = APIRouter()
router.include_router(management_tokens_router)
__all__ = ["router"]

Some files were not shown because too many files have changed in this diff Show More