Commit Graph
408 Commits
Author SHA1 Message Date
elky 84f0d5ce70 Merge origin/main into main 2026-10-05 12:10:23 +08:00
ZheFox 8ffd8188e4 Merge pull request #879 from Kayphoon/feat/gemini-usage-reasoning-badges
feat(usage): surface Gemini thinkingConfig as reasoning effort
2026-10-05 12:03:02 +08:00
elky cb7b9c9ecd feat: unify user analytics and optimize overview aggregation
Merge user accounts and usage reporting into one page with a combined ranking and account table, shared precise time ranges, and simpler range labels.

Parse overview metadata once through a schema-only view migration and disable JIT locally for bucket rebuilds. Preserve automatic backfills.

Add redacted OAuth refresh diagnostics, bucket failure context, and regression coverage. Resolve strict Clippy warnings.
2026-10-05 00:28:31 +08:00
elky 2075cd95de chore: pass fmt clippy and nextest checks
Format merged Rust sources, satisfy strict clippy lints, and restore documentation fixtures required by format registry tests.

Validation: cargo fmt --all --check; CI-scoped clippy with -D warnings; cargo nextest workspace excluding integration tests, 9747 passed and 47 skipped; targeted PostgreSQL analytics regression passed.
2026-10-02 17:35:08 +08:00
elky 625456ff24 Merge origin/main into main
Integrate upstream updates while preserving the local analytics dashboards and schema-only migration changes.

Combine user account analysis with upstream user/group usage statistics in separate tabs, retain all migration versions, and keep the deleted audit document removed.

Validation: gateway all-target cargo check, frontend type check and 57 focused tests, 48 migration tests, schema composition checks, and diff whitespace checks.
2026-10-02 11:57:18 +08:00
elky 066ea87d72 feat: revamp analytics dashboards and harden database migrations
Add dashboard and overview analytics, health monitoring, provider expense tracking, and announcement updates across the gateway and frontend.

Keep schema migrations free of historical backfills while preserving automatic backfill execution. Bound migration deadlines, run schema preparation before Compose replacement, and anonymize deleted dashboard users.

Include the current documentation cleanup and regression coverage.
2026-10-01 11:48:17 +08:00
ZheFox fb7e3fc224 Merge pull request #838 from dalamudx/feat/user-group-provider-stats
fix(stats): scope group usage by providers and add ungrouped view
2026-09-29 12:23:27 +08:00
ZheFox 73f1d79637 Merge pull request #856 from Kayphoon/fix/manual-cleanup-buffered-body
fix(admin): buffer request body for manual cleanup, smtp test, and system update routes
2026-09-29 10:05:33 +08:00
RWDai 74072e5007 test(postgres): decode ledger amount as float8 2026-09-28 18:23:20 +08:00
RWDai 4c07d9fcfb fix(admin): floor batch deductions at zero 2026-09-28 17:56:27 +08:00
Kayphoon 3541ccfe29 perf(usage): truncate usage_body_blobs in before-now cleanup to reclaim physical disk space 2026-09-25 06:02:14 +00:00
RWDai 390b73d4b6 test(data): include new migration in version expectation 2026-09-23 20:34:25 +08:00
RWDai bcd121d447 fix(admin): make bulk wallet batches idempotent 2026-09-23 19:43:18 +08:00
RWDai e25e240d16 feat(admin): batch adjust user wallet balances 2026-09-23 11:30:33 +08:00
AAEE86 07cb401fd4 fix: extract nested provider response models 2026-09-22 11:54:58 +08:00
wangpengxiang 593327c803 refactor(stats): pass query object to raw usage summary 2026-09-21 13:07:25 +08:00
wangpengxiang a9a7c64e5d fix(stats): scope group usage by providers and add ungrouped view 2026-09-21 12:49:48 +08:00
AAEE86 f960bbd2c8 feat: expose upstream response model in usage records 2026-09-21 09:51:33 +08:00
wangpengxiang a95f0d2488 feat(stats): add user group usage views 2026-09-18 16:24:03 +08:00
Kayphoon b296d46e97 feat(usage): surface Gemini thinkingConfig as reasoning effort
Usage records show a reasoning badge next to the model name for OpenAI and
Claude requests, but Gemini requests never got one. The extraction only read
the OpenAI/Claude shapes (`reasoning_effort`, `reasoning.effort`,
`output_config.effort`), while Gemini states its reasoning depth inside
`generationConfig.thinkingConfig` — so nothing was written to the usage
metadata and the list and detail views had no badge to render.

Read the Gemini shape too, as a fallback after the existing three so the
OpenAI and Claude paths are untouched:

- `thinkingLevel` / `thinking_level` wins when present, trimmed and
  lowercased, with the protobuf enum prefix stripped so `THINKING_LEVEL_HIGH`
  resolves like `high`.
- Otherwise `thinkingBudget` / `thinking_budget` goes through the existing
  shared budget ladder, yielding the same `low|medium|high|xhigh` vocabulary
  the badge already understands.
- Both camelCase and snake_case spellings are read, so a captured client body
  and a converted provider body resolve to the same label.
- `includeThoughts` alone is a visibility flag, not a depth, and produces no
  badge.

Two cases are handled explicitly rather than through the shared ladder:

- `thinkingBudget: 0` disables reasoning outright. The shared ladder maps
  `0..=1664` to `low`, which would report an explicitly disabled request as a
  shallow one, so it reports `none` instead.
- `THINKING_LEVEL_UNSPECIFIED` is the enum's "no explicit level" member, not a
  depth; it is rejected rather than surfaced as an `unspecified` badge.

The frontend needs no change: `UsageModelDisplay` already renders the badge
whenever the fields are present, and keeps the `high -> xhigh` mapping format
when the requested and upstream efforts differ.
2026-09-18 07:15:59 +00:00
ZheFox 5842c7232e fix(usage): preserve full bodies before queue truncation 2026-09-16 13:10:40 +08:00
stabeyandClaude Opus 5 04c4a97766 feat(xai): add native image and video endpoints
Expose the xAI Imagine image and video surfaces on top of the `xai`
provider, and make the shared OpenAI video-task layer survive the
production configuration they need.

Native video requests live under /v1 (generations, edits, extensions,
with /v1/videos as a creation alias that only selects xAI candidates);
the OpenAI-compatible adapter stays under /openai/v1/videos and maps
`seconds` / `size` onto numeric duration, aspect ratio and resolution.
Clients receive an opaque Aether task ID scoped to the owning user;
polling uses the upstream task ID and the original credential, and
completed downloads fetch the returned media URL without forwarding
provider authorization to the media host.

Three fixes to the shared video layer are required for this to work
outside tests:

- OpenAI/xAI task persistence now supplies a stable 16-character
  short_id, which the PostgreSQL schema requires. Existing rows keep
  their original value across reconstruction, so no schema change or
  historical rewrite is needed.
- Task retrieval and content downloads are admitted by the production
  GET execution gate, and reconstructed tasks resolve proxy nodes,
  system proxy defaults, tunnel affinity and transport profiles through
  the same deployment resolver used for creation. A configured proxy
  route no longer silently becomes a direct request after restart.
- When the gateway also serves the frontend, /openai/v1/videos and its
  subpaths bypass the static SPA handler. Otherwise a video query
  returns HTTP 200 with text/html instead of the task JSON.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-14 21:17:21 +08:00
stabeyandClaude Opus 5 e83399db2f feat(providers): add xAI provider with device code OAuth
Add a separate `xai` provider type for xAI Grok CLI subscription accounts.
It is independent of the existing `grok` provider, which reverse-proxies
grok.com with browser cookies; behavior of `grok` is unchanged.

Account binding uses the xAI device code flow, so no local callback
listener is needed and headless deployments can bind accounts. Refresh
tokens can also be imported individually or in batches, and are rotated
on refresh.

OAuth requests default to the cli-chat-proxy Responses API; API keys and
compact stay on api.x.ai. Explicit custom gateways are preserved. Only
`openai:responses` and `openai:responses:compact` are exposed; Chat,
Claude and Gemini clients reach the provider through Aether's existing
cross-format conversion rather than new native endpoints.

Upstream Responses payloads are sanitized for what xAI actually rejects:
`previous_response_id` and `metadata.user_id` are dropped, hosted
`tool_choice` is rewritten, `web_search` is restored for converted
clients, `image_generation` is stripped on older Grok conversation
models, unsupported reasoning effort is removed, and requested
`reasoning.encrypted_content` is preserved with a replay policy keyed on
the configured provider type rather than the model name.

Quota refresh reads /user and /billing?format=credits and stores a
structured usage snapshot; a prepaid balance keeps an account selectable
after the weekly allowance is exhausted. API-key accounts skip the
subscription billing surface. The admin UI shows remaining weekly quota
as a labeled bar in the provider drawer and the pool list.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-14 21:09:03 +08:00
elky 60b89cc840 fix(payment): restore recharge crediting and balance refresh 2026-09-11 13:04:54 +08:00
elky ecc16673eb fix: harden concurrency limits and high-RPM runtime paths
Bound request, stream, queue, and shutdown resource lifetimes. Reduce scheduler and Redis hot-path work and isolate database maintenance. Include regression coverage, load probes, and concurrency audit results.
2026-09-10 08:14:58 +08:00
elky 6630856061 fix: harden routing failover, model testing, and wallet queries 2026-09-09 10:38:25 +08:00
elky a893bd0557 refactor(data): reuse payment order query 2026-09-09 09:21:09 +08:00
elky e58570d79d feat(routing): make client disconnect behavior strategy-scoped 2026-09-08 23:11:37 +08:00
elky 99f6499b2b fix(conversion): improve stream failures and diagnostic exports 2026-09-08 21:04:22 +08:00
elky c7e403b410 fix: restore container logging compatibility and normalize legacy policies 2026-09-08 11:43:51 +08:00
elky 7113d04f8a fix(usage): preserve original captured HTTP headers 2026-09-08 08:49:35 +08:00
elky 099b810a2f feat: optimize usage body viewing and provider card layout 2026-09-08 02:49:06 +08:00
elky a90d564931 fix: restore security hardening compatibility and validation
Restore authorized rule reveal, explicit full HTTP capture and retention, video task business fields, and valid payment URLs. Add opt-in credential preservation for trusted recovery, fix frontend type contracts and async races, and eliminate PostgreSQL test fixture resource leaks. Document audit coverage and successful fmt and CI-scoped Clippy checks.
2026-09-07 21:14:27 +08:00
elky 062e111c03 fix(observability): preserve admin upstream error diagnostics 2026-09-07 10:39:48 +08:00
elky 2281f2b754 refactor(data): remove MySQL and SQLite support
Use PostgreSQL as the only database backend across runtime, schema tooling, installation, Compose, and CI. Update regression tests and reject removed drivers explicitly.
2026-09-07 00:09:42 +08:00
elky 856accdced test(data): expect revoked API-key ciphertext on import 2026-09-05 05:07:55 +08:00
elky f08c2e6729 test(data): expect anonymized deleted API key names 2026-09-05 04:56:14 +08:00
elky e420bc6324 fix(ci): make postgres bootstrap tests apply pending migrations 2026-09-05 04:45:07 +08:00
elky 7ed48e7b58 fix(data): make postgres snapshot migrations idempotent 2026-09-05 03:44:10 +08:00
elky 33d5cd5993 fix(ci): align lint-safe security paths 2026-09-05 03:19:53 +08:00
elky 9ff4d73d5c fix(merge): align security contracts with latest main 2026-09-05 01:31:21 +08:00
elky 313a637982 refactor(data): drop sqlite no-op migration 2026-09-04 11:47:05 +08:00
elky bac6d6866a refactor(data): remove unpublished legacy cleanup migrations 2026-09-04 11:26:59 +08:00
elky 579f2c7cc1 feat(security): harden gateway boundaries and usage policies
Consolidate subscription usage policy enforcement, privacy-safe persistence, and gateway security hardening into one reviewable change.

Includes bounded HTTP and execution envelopes, header and protocol guards, DNS and relay validation, authentication and secret projection hardening, secure backup/install paths, and regression coverage.
2026-09-04 03:45:52 +08:00
ZheFox 89fe9e9f0a fix(runtime): avoid JSON precision stream regression 2026-09-03 16:10:30 +08:00
ZheFox 979dbc4b33 fix(data): preserve jsonb numbers for metadata CAS 2026-09-03 14:37:10 +08:00
ZheFox 587486ab0c Merge remote-tracking branch 'upstream/main' into codex/fix-antigravity-quota 2026-09-03 13:24:39 +08:00
elky 058660ec2e fix(data): include routing order in postgres snapshot 2026-09-03 12:51:37 +08:00
elky 668bf5e40f test(data): include latest migration snapshot 2026-09-03 12:27:12 +08:00
fawney 2cb4d554aa feat(routing): consolidate scheduling strategy configuration 2026-09-03 11:05:59 +08:00