Merge origin/main into codex/gemini-embedding-batch

# Conflicts: # apps/aether-gateway/src/ai_serving/api.rs # apps/aether-gateway/src/ai_serving/planner/passthrough/provider/family/request.rs # apps/aether-gateway/src/ai_serving/planner/standard/family/request.rs # apps/aether-gateway/src/ai_serving/transport.rs # apps/aether-gateway/src/handlers/admin/provider/query/models/model_test/summary.rs # apps/aether-gateway/src/handlers/admin/provider/query/models/model_test/tests.rs # crates/aether-data/src/repository/candidate_selection/postgres.rs # crates/aether-model-fetch/src/strategy.rs
2026-09-02 09:20:22 +08:00 · 2026-05-18 19:02:19 +00:00
parent be2e8e594c afedd90c80
commit ca4d0dc819
446 changed files with 53043 additions and 4780 deletions
--- a/docs/architecture/data-schema-inventory.md
+++ b/docs/architecture/data-schema-inventory.md
@@ -80,7 +80,7 @@ headers; changes there should come only from `compose_schema.sh generate`.
 | Auth config | `auth_modules`, `oauth_providers`, `ldap_configs` | `repository/auth_modules`, `repository/oauth_providers`, `repository/users` | Good candidate for generated CRUD. |
 | Proxy nodes | `proxy_nodes`, `proxy_node_events` | `repository/proxy_nodes` | Good candidate for generated CRUD plus handwritten heartbeat update. |
 | Wallet/billing | `wallets`, `wallet_transactions`, `wallet_daily_usage_ledgers`, `payment_orders`, `payment_callbacks`, `refund_requests`, `redeem_code_batches`, `redeem_codes`, `billing_rules`, `dimension_collectors` | `repository/wallet`, `repository/billing`, `repository/settlement` | Keep settlement/ledger math explicit; generate table definitions and simple reads. |
-| Usage/audit | `usage`, `usage_body_blobs`, `usage_http_audits`, `usage_routing_snapshots`, `usage_settlement_snapshots`, `request_candidates`, `audit_logs` | `repository/usage`, `repository/candidates`, `repository/audit` | Keep core write/audit queries handwritten. |
+| Usage/audit | `usage`, `usage_counter_deltas`, `usage_body_blobs`, `usage_http_audits`, `usage_routing_snapshots`, `usage_settlement_snapshots`, `request_candidates`, `audit_logs` | `repository/usage`, `repository/candidates`, `repository/audit` | Keep core write/audit queries handwritten. |
 | Runtime tasks | `video_tasks`, `gemini_file_mappings`, `announcements`, `announcement_reads` | `repository/video_tasks`, `repository/gemini_file_mappings`, `repository/announcements` | Good candidate for generated CRUD except polling claim logic. |
 | Stats | `stats_*`, `schema_backfills` | backend aggregation modules | Keep aggregation SQL per-driver; generate table/index definitions only. |
 | System | `system_configs` | `repository/system` through backend dispatch | Good candidate for generated CRUD. |
@@ -105,3 +105,12 @@ as explicit generated/override fragments.
   plan if it must move across databases.
 6. If a baseline fragment changes, run `compose_schema.sh compose` before tests
   so the executable SQL artifact is regenerated from the source manifest.
+
+## Performance Notes
+
+- Shared hot counters must use `bigint` and be updated by durable outbox flush
+  workers, not request transactions.
+- `usage_counter_deltas` is append-only until processed; processed rows are
+  retained briefly for audit/replay and then batch-deleted by maintenance.
+- Candidate selection joins stay handwritten but are protected in the gateway by
+  a short TTL, single-flight cache invalidated by provider/routing writes.
--- a/docs/architecture/usage-counter-rootfix.md
+++ b/docs/architecture/usage-counter-rootfix.md
@@ -0,0 +1,279 @@
+# Usage Counter Root Fix
+
+This worktree implements the first production cut of the root fix for
+usage-related lock contention and hot-row pressure.
+
+## Problem
+
+The current Postgres write path mixes:
+
+- request facts (`usage`, audit snapshots, settlement state)
+- shared counters (`api_keys`, `provider_api_keys`, `global_models`)
+- provider quota window JSON updates
+- wallet settlement writes
+
+That means a single request can hold a transaction while touching multiple shared rows, and high-frequency traffic can serialize on the same `api_key_id` / `provider_api_key_id` / wallet rows.
+
+Relevant code paths:
+
+- `crates/aether-data/src/repository/usage/postgres/mod.rs`
+- `crates/aether-data/src/repository/settlement/postgres.rs`
+- `crates/aether-usage-runtime/src/runtime.rs`
+
+## Goal
+
+Remove shared-hot-row updates from the request path without losing correctness.
+
+The request path must become:
+
+1. write immutable request facts
+2. write durable delta records
+3. commit
+
+All shared counters must be derived later by a worker.
+
+## Target Architecture
+
+### 1. Facts first
+
+Keep `usage` as the source of truth for per-request facts:
+
+- request identity
+- status transitions
+- billing state
+- token / cost / latency payloads
+- audit snapshots
+
+### 2. Durable counter outbox
+
+Add a new append-only delta table, for example:
+
+`usage_counter_deltas`
+
+Each row should represent one logical contribution:
+
+- `kind` (`api_key`, `provider_api_key`, `model`, `provider_monthly`,
+  `proxy_node`, `management_token`, `api_key_last_used`, `window`)
+- `target_id`
+- delta fields
+- request id / revision
+- `processed_at`
+
+This table is the bridge between request facts and derived counters.
+
+### 3. Flush worker
+
+Add a background worker that:
+
+1. reads unprocessed deltas with `FOR UPDATE SKIP LOCKED`
+2. aggregates them in memory by `(kind, target_id)`
+3. applies one UPDATE per target
+4. marks the source delta rows processed in the same transaction
+
+This keeps memory useful as a buffer, but never as the only source of truth.
+
+### 4. Read models
+
+Move high-read counters to separate tables or compact materialized read models:
+
+- `api_key_usage_counters`
+- `provider_api_key_usage_counters`
+- `model_usage_counters`
+- `provider_monthly_usage_counters`
+- `provider_api_key_window_usage_counters`
+
+Keep the original business tables (`api_keys`, `provider_api_keys`, `global_models`) for configuration and compatibility only.
+
+## Implementation Boundary
+
+This branch should land the fix in phases, but the direction must not change:
+
+1. Postgres gets the durable outbox and counter flush path first, because the
+   reported production lock wait is on Postgres row locks.
+2. MySQL and SQLite keep their current usage write behavior until their smaller
+   deployment paths are moved to the same contract.
+3. Request-path Postgres writes may still write `usage`, audit blobs, routing
+   snapshots, and settlement pricing snapshots. They must not directly update
+   shared aggregate rows.
+4. Compatibility mirror columns may be updated by the flush worker only, never
+   by the request transaction.
+5. Dashboard/statistics read paths should be migrated after the write pressure
+   is removed, otherwise we risk mixing a large read refactor with the lock fix.
+
+The first executable migration creates the outbox. The first code patch makes
+`upsert_usage_record` enqueue deltas in the same transaction as the usage fact
+write, then a background worker batches those deltas with
+`FOR UPDATE SKIP LOCKED`. Dedicated counter read-model tables remain a follow-up
+after the request-path lock pressure is removed.
+
+## Locking Model
+
+### Keep
+
+- advisory lock per `request_id` for idempotent request transitions
+- wallet row lock only inside settlement, where correctness depends on it
+
+### Remove from request path
+
+- direct `UPDATE api_keys`
+- direct `UPDATE provider_api_keys`
+- direct `UPDATE global_models`
+- direct `UPDATE providers.monthly_used_usd`
+- direct `FOR UPDATE` on provider quota JSON for request-level usage windows
+
+## Memory Cache Rules
+
+Allowed:
+
+- short TTL snapshot cache for read-only admin UI data
+- short TTL + single-flight cache for provider/model candidate selection rows
+- worker-side delta aggregation buffer
+- per-key last-used-at max tracking before flush
+
+Not allowed:
+
+- using memory as the only accounting source
+- using memory as the only settlement source
+- depending on a clipped Redis stream as the only record of usage
+
+## Rollout Plan
+
+1. Stop counting `pending` / `streaming` as shared counter contributions.
+2. Introduce the delta outbox and worker.
+3. Redirect request path to facts + outbox only.
+4. Migrate reads to the new counter tables.
+5. Decommission synchronous hot-row updates.
+6. Move provider quota windows out of request transactions.
+
+## Implemented In This Branch
+
+- `usage_counter_deltas` durable outbox for api key, provider api key, model,
+  provider monthly, proxy node, management token, and api key last-used counters.
+- Postgres usage upsert writes request facts plus outbox rows, not shared counter
+  rows.
+- Postgres settlement enqueues provider monthly usage deltas instead of updating
+  `providers.monthly_used_usd` in the request transaction.
+- Gateway request-adjacent proxy node, management token, and api key last-used
+  writes enqueue durable deltas and fall back to direct writes only when the
+  usage writer is unavailable.
+- Gateway provider/model candidate selection reads use a 5 second in-memory TTL
+  cache with per-key single-flight. Provider/routing catalog writes invalidate
+  this cache alongside provider transport and scheduler affinity caches.
+- Gateway maintenance worker flushes deltas in batches and aggregates in memory
+  inside the worker before applying compatibility counter updates.
+- Daily quota lookup index on `(user_entitlement_id, usage_date)` removes the
+  avoidable aggregate scan in quota settlement checks.
+- Hot counter columns are widened to `bigint` where old bootstrap schemas still
+  used `integer`, preventing long-running counter overflow.
+
+## Integration Pressure Tests
+
+The hotspot benchmarks start a managed local Postgres instance, run the
+migrations, seed a single hot target, then monitor `pg_stat_activity` while the
+load is running. Use a separate target directory when the main worktree target
+lock is not writable.
+
+Usage write path, one hot `api_key` / `provider_api_key` / `global_model`:
+
+```sh
+CARGO_TARGET_DIR=/tmp/aether-rootfix-target \
+cargo run -p aether-testkit --bin usage_counter_hotspot_baseline -- \
+  --requests 5000 \
+  --concurrency 200 \
+  --flush-interval-ms 50 \
+  --monitor-interval-ms 20 \
+  --output /tmp/usage_counter_hotspot_after_5000.json
+```
+
+Settlement path, one hot provider monthly counter:
+
+```sh
+CARGO_TARGET_DIR=/tmp/aether-rootfix-target \
+cargo run -p aether-testkit --bin usage_settlement_hotspot_baseline -- \
+  --requests 5000 \
+  --concurrency 200 \
+  --flush-interval-ms 50 \
+  --monitor-interval-ms 20 \
+  --output /tmp/usage_settlement_hotspot_after_5000.json
+```
+
+Auxiliary hot counters, one hot proxy node / management token / api key
+last-used target:
+
+```sh
+CARGO_TARGET_DIR=/tmp/aether-rootfix-target \
+cargo run -p aether-testkit --bin usage_aux_counter_hotspot_baseline -- \
+  --requests 5000 \
+  --concurrency 200 \
+  --flush-interval-ms 50 \
+  --monitor-interval-ms 20 \
+  --output /tmp/usage_aux_counter_hotspot_after_5000.json
+```
+
+Latest local run on this worktree:
+
+- usage hotspot: 5000 requests, 200 concurrency, p95 173 ms, 0 failures,
+  15000 outbox rows processed, 0 pending rows, 0 `api_keys` /
+  `provider_api_keys` / `global_models` update waiters.
+- settlement hotspot: 5000 requests, 200 concurrency, p95 39 ms, 0 failures,
+  5000 provider monthly deltas processed, `providers.monthly_used_usd = 5.0`,
+  0 provider update waiters.
+
+Run the auxiliary counter hotspot after changing outbox schemas or gateway
+fallback routing; it should drain all pending outbox rows and report zero
+request-path waiters for `proxy_nodes`, `management_tokens`, and `api_keys`.
+
+## Runtime Observability
+
+The same outbox health signals used by the pressure tools are exposed through
+admin runtime endpoints:
+
+- `GET /api/admin/system/stats`
+- `GET /api/admin/monitoring/system-status`
+- `GET /api/admin/stats/performance/providers`
+
+These responses include `usage_counter`:
+
+- `status`: `idle`, `catching_up`, or `backlogged`
+- `outbox_pending_rows`
+- `outbox_processed_rows`
+- `oldest_pending_created_at_unix_secs`
+- `oldest_pending_age_secs`
+- `latest_processed_at_unix_secs`
+- `pending_by_kind`
+
+Operational alerting should page when `status = backlogged`, when pending rows
+continue growing across several flush intervals, or when the oldest pending age
+stays above one minute. A transient non-zero backlog is acceptable during catch
+up bursts.
+
+## Remaining Correctness Locks
+
+Wallet debit settlement and daily quota consumption still use database locks
+because they protect money/quota correctness, not derived counters. Removing
+those locks safely requires a separate wallet debit ledger/reservation worker:
+
+1. request settlement writes an immutable debit intent keyed by `request_id`
+2. a per-wallet worker claims intents with `FOR UPDATE SKIP LOCKED`
+3. the worker applies balance changes and writes final settlement snapshots
+4. request-facing APIs read `pending/settled/insufficient_quota` from the
+   settlement snapshot
+
+Do not replace this with memory-only balance caches. A cache may accelerate
+read-side availability estimates, but the durable ledger must remain the source
+of truth.
+
+## Acceptance Criteria
+
+- request transactions no longer update shared counter rows
+- counter updates become batchable and replayable
+- lock wait time on `api_keys` / `provider_api_keys` drops sharply under concurrency
+- wallet settlement remains correct and isolated
+- dashboard/statistics reads stay fast from dedicated read models
+
+## Open Decisions
+
+- table names for counter read models
+- whether provider window counters live in Postgres only or are dual-written into Redis for display latency
+- flush cadence and batch size defaults
+- whether to keep compatibility snapshot columns as low-frequency mirrors
--- a/docs/development/simple-query-inventory.md
+++ b/docs/development/simple-query-inventory.md
@@ -0,0 +1,90 @@
+# Aether Data Simple Query Inventory
+
+This inventory tracks repository read paths that are intended to use the
+internal `aether-data-query` helpers. The first layer centralizes SQL fragments;
+the newer `SelectQuery` layer lets repositories describe simple `SELECT`
+queries once and render dialect-specific projections for Postgres and SQLite.
+
+## Included In This Pass
+
+- `background_tasks`
+  - `find_run`
+  - `list_runs`
+  - `list_events`
+  - simple `summarize_runs` count/group reads remain behavior-locked in SQL
+- `announcements`
+  - `find_by_id`
+  - `list_announcements`
+  - `count_unread_active_announcements`
+- `auth_modules`
+  - `list_enabled_oauth_providers`
+  - `get_ldap_config`
+- `oauth_providers`
+  - `list_oauth_provider_configs`
+  - `get_oauth_provider_config`
+- `quota`
+  - `find_by_provider_id`
+  - `find_by_provider_ids`
+  - now uses one `SelectQuery` specification for the quota snapshot projection
+    across Postgres and SQLite
+- `provider_catalog`
+  - provider by-id/provider list reads in PG/SQLite
+  - endpoint/key by-id and by-provider-id `IN` reads in PG/SQLite
+  - provider key page filters, search, order, limit/offset in PG/SQLite
+  - key stats by provider ids in PG/SQLite
+- `proxy_nodes`
+  - node list/find reads
+  - event list/filter reads
+- `management_tokens`
+  - `list_management_tokens`
+  - `get_management_token_with_user`
+  - `get_management_token_with_user_by_hash`
+- `pool_scores`
+  - `find_scores_by_identity`
+  - `list_ranked_pool_members`
+  - `list_pool_member_scores`
+  - `list_pool_member_probe_candidates`
+  - `get_pool_member_scores_by_ids`
+- `candidates`
+  - `list_by_request_id`
+  - `list_recent`
+  - `list_by_provider_id`
+  - `list_finalized_by_endpoint_ids_since`
+  - simple finalized status count
+- `gemini_file_mappings`
+  - list/count filters and search
+
+## Deferred
+
+- `usage` aggregation, dashboard, leaderboard, cache-hit, provider/key/user
+  statistics, rebuild paths, and body/blob reads.
+- `candidate_selection` JSON/alias matching and scoring.
+- `wallet` ledger, order, refund, callback, and redeem-code list logic.
+- write/upsert/delete paths, transactions, `RETURNING`, CTEs, window functions,
+  advisory locks, and schema compatibility probes.
+- `users/auth` and `global_models` still contain additional simple read paths.
+  `global_models/sqlite.rs` had pre-existing local edits and must be handled
+  carefully in a dedicated slice.
+
+## Helper Coverage
+
+- dialect-aware identifier quoting
+- dialect-specific SQL expressions through `DialectSql`
+- simple `SELECT` rendering through `SelectQuery`
+- `WHERE`/`AND` sequencing
+- equality and optional equality filters
+- `IN` filters
+- case-insensitive contains/search
+- whitelisted order-by rendering
+- `LIMIT` and `LIMIT/OFFSET`
+
+## Query Abstraction Shape
+
+The intended direction is:
+
+- repository defines table-specific projections, joins, and row mapping
+- `SelectQuery` renders `SELECT ... FROM ... JOIN ...` for the active dialect
+- `SelectStatement` owns dynamic filters, search, ordering, and pagination with
+  stable bind order
+- complex SQL remains hand-written until it has enough repeated structure to
+  justify a dedicated abstraction
--- a/docs/operations/pg-to-single-node-migration.md
+++ b/docs/operations/pg-to-single-node-migration.md
@@ -0,0 +1,238 @@
+# Postgres to Aether Single Node Migration
+
+Chinese version: [pg-to-single-node-migration.zh-CN.md](pg-to-single-node-migration.zh-CN.md)
+
+This runbook migrates an existing Docker Compose Postgres deployment to Aether
+single-node. In this repository, **single-node** means the default SQLite installer mode:
+`install.sh --mode single-node`, a system service backed by SQLite. The Docker Compose
+single-node template is `docker-compose.single-node.yml`, exposed through `--mode compose-single-node`.
+
+The migration script is:
+
+```bash
+scripts/migrate-pg-to-single-node.sh
+```
+
+If the target should stay on Docker Compose instead of becoming a system
+service, use the image-based Compose migration script:
+
+```bash
+scripts/migrate-pg-compose-to-single-node.sh
+```
+
+Both migration scripts pull/install the target single-node version before
+downtime, stop only the source `app`, copy Postgres records directly into a
+temporary SQLite DB without writing a JSONL file, replace the target
+`aether.db`, and start single-node.
+
+You can also use the installer as the unified entrypoint and let `--mode`
+select the migration target:
+
+```bash
+# In interactive mode, first choose the target deployment mode:
+#   1) Docker Compose standard deployment (Postgres + Redis)
+#   2) Docker Compose single-node deployment (SQLite)
+#   3) System service single-node deployment (SQLite)
+# After choosing 2 or 3, choose the data initialization mode:
+#   1) Fresh initialization (do not migrate existing data)
+#   2) Migrate from an existing Docker Compose PG database
+install.sh
+
+# Migrate into a new single-node Docker Compose directory.
+install.sh \
+  --mode compose-single-node \
+  --migrate-from-compose /root/Aether/docker-compose.yml \
+  --compose-dir /opt/aether-single \
+  --replace-existing
+
+# Migrate into the system service + SQLite layout.
+sudo install.sh \
+  --mode single-node \
+  --migrate-from-compose /root/Aether/docker-compose.yml \
+  --replace-existing
+```
+
+Interactive mode first asks for the target deployment shape. If the target is
+`compose-single-node` or `single-node`, the installer then asks for the data
+initialization mode: fresh initialization, or migration from an existing Docker
+Compose PG database. If you choose migration, it tries to detect the source PG
+Compose file from `docker compose ls`, then verifies that the Compose config
+contains the default `app` and `postgres` services. If exactly one match is
+found, it is used as the default prompt value. If detection is ambiguous or
+fails, the installer stops; rerun it with `--migrate-from-compose` to specify
+the source compose path.
+
+The installer only normalizes the entrypoint: `compose-single-node` delegates to
+`scripts/migrate-pg-compose-to-single-node.sh`, while `single-node` delegates to
+`scripts/migrate-pg-to-single-node.sh`.
+
+## What It Does
+
+The script keeps the production cutover window short:
+
+1. Reads the source Compose `.env`.
+2. Builds a single-node env file that preserves `JWT_SECRET_KEY`, `ENCRYPTION_KEY` or
+   `AETHER_GATEWAY_DATA_ENCRYPTION_KEY`, admin settings, port, and app config.
+3. Installs the single-node release with `install.sh --mode single-node --skip-start`.
+4. Preflights SQLite migrations with the installed single-node binary.
+5. Pulls the target single-node image, confirms its `copy` command is available,
+   and verifies that its Docker image ID matches the currently running source
+   `app` image ID.
+6. Uses the target SQLite schema as the migration plan: same-name source
+   Postgres tables and columns are copied into the temporary SQLite DB.
+7. Applies the compressed body and HTTP body detail policy. The default is full,
+   and you can opt into an omit mode for large artifacts.
+8. Checks that the work directory and target SQLite directory have enough free
+   disk space for the temporary and final SQLite files.
+9. Stops only the source `app` service, leaving Postgres and Redis running.
+10. Copies source Postgres records directly into a temporary SQLite database
+   without generating JSONL files.
+11. Replaces the target SQLite DB, including SQLite `-wal`/`-shm` sidecar files
+    when present, and starts the single-node service.
+
+The image check compares Docker image IDs, not just tag strings. If both source
+and target say `latest` but resolve to different image IDs, migration stops.
+Upgrade the source PG Compose `app` to the target single-node version first,
+verify it is healthy, then run the migration. The scripts also check that the
+target image supports direct copy and the request-body omit flag; using a new
+script with an old image stops before cutover to avoid missing data.
+
+## Production Cutover
+
+Before production cutover, take a normal server backup or snapshot. Then run:
+
+```bash
+sudo scripts/migrate-pg-to-single-node.sh \
+  --source-compose /root/Aether/docker-compose.yml \
+  --replace-existing
+```
+
+For Docker Compose single-node cutover instead of a system service:
+
+```bash
+scripts/migrate-pg-compose-to-single-node.sh \
+  --source-compose /root/Aether/docker-compose.yml \
+  --replace-existing
+```
+
+The source Postgres compose directory and target single-node compose directory
+can be different. For example:
+
+```bash
+install.sh \
+  --mode compose-single-node \
+  --migrate-from-compose /root/Aether/docker-compose.yml \
+  --compose-dir /opt/aether-single \
+  --replace-existing
+```
+
+Equivalently, call the lower-level script and pass each target path explicitly:
+
+```bash
+scripts/migrate-pg-compose-to-single-node.sh \
+  --source-compose /root/Aether/docker-compose.yml \
+  --target-compose /opt/aether-single/docker-compose.single-node.yml \
+  --target-env /opt/aether-single/.env.single-node \
+  --target-db /opt/aether-single/data/aether.db \
+  --replace-existing
+```
+
+During cutover, the script stops and removes only the source `app` container to
+free the fixed `aether-app` container name. Postgres, Redis, and their volumes
+remain in place for rollback.
+
+Defaults:
+
+| Setting | Default |
+| --- | --- |
+| Source Compose | `docker-compose.yml` |
+| Single Node install root | `/opt/aether` |
+| Single Node config dir | `/etc/aether` |
+| Target SQLite DB | `/opt/aether/data/aether.db` |
+| Source app service | `app` |
+| Source Postgres service | `postgres` |
+| Single Node service | `aether-gateway` |
+
+The script writes migration artifacts under `./data/pg-to-single-node-<timestamp>` next
+to the source Compose file unless `--work-dir` is provided.
+
+## Rollback
+
+The script leaves the original Postgres and Redis volumes in place. If cutover
+finishes but you need to roll back:
+
+```bash
+sudo systemctl stop aether-gateway
+cd /root/Aether
+docker compose -f docker-compose.yml up -d app
+```
+
+For the Compose single-node script, rollback is the same idea: start the app
+again from the original Postgres compose file.
+
+If the migration fails before cutover completes, the script attempts to restart
+the source `app` service automatically. Pass `--keep-source-stopped-on-error` if
+you want to inspect the stopped source deployment manually instead.
+
+## Data Coverage Guard
+
+The migration does not maintain a separate business-domain table list. The
+target single-node image first builds a temporary SQLite database with its
+normal migrations, then `aether-gateway copy` reads that SQLite schema and copies
+matching public Postgres tables and columns.
+
+If the source Postgres database has a non-empty public table that does not exist
+in the target SQLite schema, the copy stops instead of silently dropping it. It
+ignores lifecycle metadata tables such as `_sqlx_migrations` and
+`schema_backfills`. Extra source columns that are absent from the target schema
+are not copied.
+
+## Request Body Detail Policy
+
+The production migration migrates all migratable data by default. The only
+optional exclusion is request body detail data.
+
+When you choose to skip request bodies, the migration does not copy
+`usage_body_blobs`, `usage_http_audits`, or legacy `usage` request body columns
+such as `request_body`, `provider_request_body`, `response_body`,
+`client_response_body`, and `*_body_compressed`.
+
+Interactive installation lets you choose:
+
+```text
+1) Full migration: migrate all migratable data, including request body details
+2) Skip request bodies: migrate all other data; skip only request body large fields and HTTP body detail tables; source PG is unchanged
+```
+
+For non-interactive full runs:
+
+```bash
+scripts/migrate-pg-to-single-node.sh \
+  --request-body-mode full
+```
+
+For non-interactive omit runs:
+
+```bash
+scripts/migrate-pg-to-single-node.sh \
+  --request-body-mode omit
+```
+
+`omit` only skips writing those large artifacts and detail tables into the
+target SQLite database. It does not delete or clear the source Postgres data.
+
+## Notes
+
+- Single Node requires root or sudo because it writes `/opt/aether`, `/etc/aether`, and
+  the system service definition.
+- The script does not decrypt or re-encrypt provider keys. It preserves the
+  original encryption key and moves encrypted data as-is.
+- Existing target SQLite databases, including `-wal`/`-shm` sidecars, are not
+  replaced unless `--replace-existing` is provided.
+- Disk space checks use `pg_database_size(current_database()) * 2 + 1 GiB` as the
+  conservative estimate for one SQLite copy. If the work directory and target DB
+  directory are on the same filesystem, the script requires enough space for both
+  the temporary and final SQLite files. With `--request-body-mode omit`, the
+  estimate subtracts `usage_body_blobs` and `usage_http_audits` relation sizes.
+- For non-standard source Compose files, set `--app-service` and
+  `--postgres-service` to match the service names.
--- a/docs/operations/pg-to-single-node-migration.zh-CN.md
+++ b/docs/operations/pg-to-single-node-migration.zh-CN.md
@@ -0,0 +1,221 @@
+# Postgres 到 Aether Single Node 迁移
+
+英文版：[pg-to-single-node-migration.md](pg-to-single-node-migration.md)
+
+本文档用于把现有 Docker Compose Postgres 部署迁移到 Aether
+single-node。当前版本里，**single-node** 指默认 SQLite 安装模式：
+`install.sh --mode single-node`，也就是系统服务加 SQLite。Docker Compose
+单机模板是 `docker-compose.single-node.yml`，安装脚本入口是
+`--mode compose-single-node`。
+
+迁移脚本：
+
+```bash
+scripts/migrate-pg-to-single-node.sh
+```
+
+如果目标形态仍然要保持 Docker Compose，而不是系统服务，使用镜像版迁移脚本：
+
+```bash
+scripts/migrate-pg-compose-to-single-node.sh
+```
+
+两种迁移脚本都会先拉取/安装目标 single-node 版本，再停止源 `app`，把 Postgres
+记录直接写入临时 SQLite DB，不落 JSONL 中间文件；复制成功后替换目标
+`aether.db`，最后启动 single-node。
+
+也可以直接用安装脚本作为统一入口，由 `--mode` 选择迁移目标：
+
+```bash
+# 交互式执行时，先选择目标部署模式：
+#   1) Docker Compose 标准部署（Postgres + Redis）
+#   2) Docker Compose 单节点部署（SQLite）
+#   3) 系统服务单节点部署（SQLite）
+# 选择 2 或 3 后，再选择数据初始化方式：
+#   1) 全新初始化（不迁移现有数据）
+#   2) 从现有 Docker Compose PG 数据库迁移
+install.sh
+
+# 迁移到新的 single-node Docker Compose 目录
+install.sh \
+  --mode compose-single-node \
+  --migrate-from-compose /root/Aether/docker-compose.yml \
+  --compose-dir /opt/aether-single \
+  --replace-existing
+
+# 迁移到系统服务 + SQLite
+sudo install.sh \
+  --mode single-node \
+  --migrate-from-compose /root/Aether/docker-compose.yml \
+  --replace-existing
+```
+
+交互模式会先选择目标部署形态。如果目标是 `compose-single-node` 或
+`single-node`，安装脚本会再询问数据初始化方式：全新初始化，或从现有 Docker
+Compose PG 数据库迁移。选择迁移后，脚本会通过 `docker compose ls` 自动探测源
+PG Compose 文件，并确认该 Compose 配置里存在默认的 `app` 和 `postgres` 服务；
+如果能唯一识别，会作为默认值带入提示。探测不到或存在多个候选时会直接中止；
+此时请用 `--migrate-from-compose` 显式指定源 compose 路径。
+
+安装脚本只是统一参数入口：`compose-single-node` 会委托给
+`scripts/migrate-pg-compose-to-single-node.sh`，`single-node` 会委托给
+`scripts/migrate-pg-to-single-node.sh`。
+
+## 迁移内容
+
+脚本会尽量缩短生产停机窗口：
+
+1. 读取源 Compose 目录下的 `.env`。
+2. 生成 single-node 环境文件，保留 `JWT_SECRET_KEY`、`ENCRYPTION_KEY` 或
+   `AETHER_GATEWAY_DATA_ENCRYPTION_KEY`、管理员配置、端口和应用配置。
+3. 执行 `install.sh --mode single-node --skip-start`，提前安装 single-node
+   release，但不启动服务。
+4. 使用已安装的 single-node 二进制预检 SQLite schema migration。
+5. 拉取目标 single-node 镜像，确认其 `copy` 命令可用，并检查源 `app`
+   当前运行镜像 ID 与目标镜像 ID 一致。
+6. 以目标 SQLite schema 作为迁移计划：把源 Postgres 中同名表、同名字段
+   复制到临时 SQLite DB。
+7. 检查请求体明细迁移策略；默认全部迁移，也可以选择只跳过请求体明细。
+8. 检查 work-dir 和目标 SQLite 目录是否有足够空间容纳临时库和正式库。
+9. 只停止源 Compose 的 `app` 服务，保留 Postgres 和 Redis 运行，方便回滚。
+10. 从源 Postgres 直接复制记录到临时 SQLite 数据库，不生成 JSONL 中间文件。
+11. 复制完成后替换目标 SQLite DB，包括 SQLite `-wal`、`-shm` 边车文件，
+    然后启动 single-node 系统服务。
+
+镜像一致性检查比较的是 Docker 镜像 ID，不只是 tag 字符串。即使源和目标都写着
+`latest`，只要实际镜像 ID 不同，迁移也会中止。请先把源 PG Compose 的 `app`
+升级到目标 single-node 相同版本，确认运行正常后再迁移。迁移脚本也会检查目标镜像
+是否支持直接 copy 和请求体跳过开关；如果只是换了脚本但镜像还是旧版本，脚本会
+直接中止，避免漏迁。
+
+## 生产切换
+
+切换前先做一次常规服务器备份或快照。确认后执行：
+
+```bash
+sudo scripts/migrate-pg-to-single-node.sh \
+  --source-compose /root/Aether/docker-compose.yml \
+  --replace-existing
+```
+
+如果要迁移到 Docker Compose single-node，而不是系统服务：
+
+```bash
+scripts/migrate-pg-compose-to-single-node.sh \
+  --source-compose /root/Aether/docker-compose.yml \
+  --replace-existing
+```
+
+源 Postgres Compose 目录和目标 single-node Compose 目录可以不一样。例如：
+
+```bash
+install.sh \
+  --mode compose-single-node \
+  --migrate-from-compose /root/Aether/docker-compose.yml \
+  --compose-dir /opt/aether-single \
+  --replace-existing
+```
+
+等价地，也可以直接调底层脚本并显式传入每个目标路径：
+
+```bash
+scripts/migrate-pg-compose-to-single-node.sh \
+  --source-compose /root/Aether/docker-compose.yml \
+  --target-compose /opt/aether-single/docker-compose.single-node.yml \
+  --target-env /opt/aether-single/.env.single-node \
+  --target-db /opt/aether-single/data/aether.db \
+  --replace-existing
+```
+
+切换时脚本只会停止并移除源 `app` 容器，用来释放固定的 `aether-app`
+容器名；Postgres、Redis 和它们的 volume 都会保留，方便回滚。
+
+默认路径和服务名：
+
+| 配置项 | 默认值 |
+| --- | --- |
+| 源 Compose 文件 | `docker-compose.yml` |
+| single-node 安装目录 | `/opt/aether` |
+| single-node 配置目录 | `/etc/aether` |
+| 目标 SQLite DB | `/opt/aether/data/aether.db` |
+| 源 app 服务 | `app` |
+| 源 Postgres 服务 | `postgres` |
+| single-node 服务 | `aether-gateway` |
+
+除非显式传入 `--work-dir`，脚本会把迁移产物写到源 Compose 文件旁边的
+`./data/pg-to-single-node-<timestamp>`。
+
+## 回滚
+
+脚本会保留原 Postgres 和 Redis volume。迁移已经完成但需要回滚时：
+
+```bash
+sudo systemctl stop aether-gateway
+cd /root/Aether
+docker compose -f docker-compose.yml up -d app
+```
+
+对于 Compose single-node 脚本，回滚思路相同：重新用原 Postgres compose 文件
+拉起 `app`。
+
+如果迁移在切换完成前失败，脚本默认会尝试自动拉起源 `app` 服务。需要失败后
+保持源应用停止以便人工排查时，增加：
+
+```bash
+--keep-source-stopped-on-error
+```
+
+## 数据覆盖保护
+
+迁移不再维护一份额外的业务表清单。目标 single-node 镜像会先用正常
+migrations 建出临时 SQLite 数据库，然后 `aether-gateway copy` 读取这个
+SQLite schema，把源 Postgres 里同名表、同名字段复制过去。
+
+如果源 Postgres 里存在非空 public 表，但目标 SQLite schema 中没有同名表，
+copy 会直接中止，不会静默丢弃。生命周期元数据表 `_sqlx_migrations` 和
+`schema_backfills` 会被忽略。源表中存在但目标 SQLite 不存在的额外字段不会复制。
+
+## 请求体明细策略
+
+single-node SQLite 生产迁移默认迁移所有可迁移数据，唯一可选的跳过项是请求体明细。
+
+选择“不迁移请求体”时，不会迁移 `usage_body_blobs`、`usage_http_audits`，也不会迁移 `usage`
+表里的 `request_body` / `provider_request_body` / `response_body` /
+`client_response_body` / `*_body_compressed` 等请求体大字段。
+
+交互安装时可以选择：
+
+```text
+1) 全部迁移：迁移所有可迁移数据，包括请求体明细
+2) 不迁移请求体：迁移其他所有数据；仅跳过请求体大字段和 HTTP 请求体明细，源 PG 不清除
+```
+
+非交互执行时，全部迁移可以显式指定：
+
+```bash
+scripts/migrate-pg-to-single-node.sh \
+  --request-body-mode full
+```
+
+不迁移请求体可以显式指定：
+
+```bash
+scripts/migrate-pg-to-single-node.sh \
+  --request-body-mode omit
+```
+
+`omit` 只是不把这些大字段和明细表写进目标 SQLite，不会删除或清空源 Postgres。
+
+## 注意事项
+
+- single-node 安装需要 root 或 sudo 权限，因为会写入 `/opt/aether`、
+  `/etc/aether` 和系统服务定义。
+- 脚本不会解密或重新加密供应商密钥；它会沿用源环境的加密密钥，并原样迁移已加密数据。
+- 已存在的目标 SQLite DB，包括 `-wal`、`-shm` 边车文件，只有在传入
+  `--replace-existing` 时才会被替换。
+- 空间检查会用 `pg_database_size(current_database()) * 2 + 1 GiB` 作为单份
+  SQLite 的保守估算。如果 work-dir 和目标 DB 目录在同一个文件系统，会要求同时
+  容纳临时 SQLite 和正式 SQLite。选择 `--request-body-mode omit` 时，
+  估算会扣除 `usage_body_blobs` 和 `usage_http_audits` 的表空间。
+- 非标准 Compose 服务名需要通过 `--app-service` 和 `--postgres-service`
+  明确指定。