mirror of
https://github.com/fawney19/Aether.git
synced 2026-09-02 01:10:23 +08:00
fce7e959e5
Introduce aether-data-schema and driver-specific schema generation for Postgres, MySQL, and SQLite. Split data backends, lifecycle, repositories, and gateway runtime integration across database drivers. Verified with cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings, and cargo test --workspace.
108 lines
7.1 KiB
Markdown
108 lines
7.1 KiB
Markdown
# Aether Data Schema Inventory
|
|
|
|
This inventory is the maintenance map for the three SQL drivers. The executable
|
|
`sqlx` migrations remain under `crates/aether-data/migrations/{postgres,mysql,sqlite}`.
|
|
Do not split already-shipped migration files without also deciding how to handle
|
|
existing `_sqlx_migrations` rows.
|
|
|
|
The maintainable schema source is under `crates/aether-data/schema`. Manifests
|
|
there compose back into the executable SQL files and are checked by tests, so
|
|
the split source is not just documentation.
|
|
|
|
The schema directory separates human-maintained sources from generated/runtime
|
|
outputs:
|
|
|
|
| Layer | Purpose |
|
|
|---|---|
|
|
| `schema/logical/*.toml` | Human-maintained long-term table-structure source. |
|
|
| `schema/drivers/{postgres,mysql,sqlite}/**` | Human-maintained executable-SQL fragments while generation is being promoted incrementally. |
|
|
| `schema/bootstrap/postgres/**` | Human-maintained source fragments for the Postgres empty-database bootstrap snapshot. |
|
|
| `schema/generated/**` | Machine-written SQL from logical schema; checked in for audit and drift detection only. |
|
|
| `migrations/**` | Runtime SQL artifacts composed from manifests. |
|
|
|
|
Generated SQL is not hand-maintained and runtime code does not load it. It
|
|
exists to prove that the logical schema can emit driver SQL and to provide the
|
|
candidate replacement for handwritten fragments.
|
|
|
|
## Logical Type Map
|
|
|
|
| Logical type | Postgres | MySQL | SQLite | Notes |
|
|
|---|---|---|---|---|
|
|
| `id` | `varchar/text` | `varchar` | `text` | Repository DTOs treat ids as strings. |
|
|
| `bool` | `boolean` | `tinyint(1)/boolean` | `integer` | Repositories normalize to Rust `bool`. |
|
|
| `time_unix` | `bigint` or legacy `timestamptz` | `bigint` | `integer` | New cross-driver paths prefer unix seconds/ms. |
|
|
| `json` | `json/jsonb` | `text/json-compatible` | `text` | Application parses through `serde_json::Value`. |
|
|
| `decimal_money` | `numeric` or `double precision` legacy | `double` | `real` | Wallet precision should be reviewed before new money tables. |
|
|
| `blob` | `bytea` | `longblob/blob` | `blob` | Used for compressed body payloads. |
|
|
| `enum` | `enum` or `varchar` legacy | `varchar` | `text` | Repository contracts own allowed values. |
|
|
|
|
## Baseline Source Plan
|
|
|
|
The current executable SQL files are intentionally kept stable for runtime
|
|
compatibility. Their maintainable sources are:
|
|
|
|
| Driver | Executable SQL | Source manifest |
|
|
|---|---|---|
|
|
| Postgres baseline | `migrations/postgres/20260403000000_baseline.sql` | `schema/drivers/postgres/baseline/manifest.txt` |
|
|
| Postgres empty-database snapshot | `aether-data` build output (`OUT_DIR/empty_database_snapshot.sql`) | `schema/bootstrap/postgres/manifest.txt` |
|
|
| MySQL baseline | `migrations/mysql/20260403000000_baseline.sql` | `schema/drivers/mysql/baseline/manifest.txt` |
|
|
| SQLite baseline | `migrations/sqlite/20260403000000_baseline.sql` | `schema/drivers/sqlite/baseline/manifest.txt` |
|
|
|
|
All driver manifests are kept as a small set of numbered SQL fragments. Postgres
|
|
uses execution-phase fragments (`001_types_and_tables.sql`,
|
|
`002_defaults.sql`, `003_constraints.sql`, `004_indexes.sql`,
|
|
`005_foreign_keys.sql`, `006_footer.sql`) so pg_dump ordering remains stable
|
|
when composed. MySQL and SQLite use similarly numbered domain fragments. After
|
|
editing fragments, run:
|
|
|
|
```bash
|
|
bash crates/aether-data/schema/compose_schema.sh compose
|
|
bash crates/aether-data/schema/compose_schema.sh check
|
|
```
|
|
|
|
Use `schema/logical/*.toml` for new table structure first; handwritten driver
|
|
fragments remain for executable migration compatibility and generator gaps.
|
|
|
|
`crates/aether-data/schema/logical/*.toml` is the single-maintenance source for
|
|
table structure. `aether-data-schema` renders it to
|
|
`schema/generated/{postgres,mysql,sqlite}/baseline`, and
|
|
`compose_schema.sh check` verifies that the generated SQL is current and that
|
|
required executable SQL tables are represented in logical schema. The generated
|
|
directory carries its own machine-generated README and per-file `Do not edit`
|
|
headers; changes there should come only from `compose_schema.sh generate`.
|
|
|
|
## Table Inventory
|
|
|
|
| Area | Tables | Owner | Generation target |
|
|
|---|---|---|---|
|
|
| Identity/auth | `users`, `api_keys`, `management_tokens`, `user_preferences`, `user_sessions`, `user_oauth_links` | `repository/users`, `repository/auth`, `repository/management_tokens`, auth modules | Good first candidate for schema manifest/query helper generation. |
|
|
| Provider catalog | `providers`, `provider_api_keys`, `provider_endpoints`, `models`, `global_models`, `api_key_provider_mappings`, `provider_usage_tracking` | `repository/provider_catalog`, `repository/global_models`, scheduler read paths | Keep complex selection SQL handwritten; generate basic CRUD only. |
|
|
| Auth config | `auth_modules`, `oauth_providers`, `ldap_configs` | `repository/auth_modules`, `repository/oauth_providers`, `repository/users` | Good candidate for generated CRUD. |
|
|
| Proxy nodes | `proxy_nodes`, `proxy_node_events` | `repository/proxy_nodes` | Good candidate for generated CRUD plus handwritten heartbeat update. |
|
|
| Wallet/billing | `wallets`, `wallet_transactions`, `wallet_daily_usage_ledgers`, `payment_orders`, `payment_callbacks`, `refund_requests`, `redeem_code_batches`, `redeem_codes`, `billing_rules`, `dimension_collectors` | `repository/wallet`, `repository/billing`, `repository/settlement` | Keep settlement/ledger math explicit; generate table definitions and simple reads. |
|
|
| Usage/audit | `usage`, `usage_body_blobs`, `usage_http_audits`, `usage_routing_snapshots`, `usage_settlement_snapshots`, `request_candidates`, `audit_logs` | `repository/usage`, `repository/candidates`, `repository/audit` | Keep core write/audit queries handwritten. |
|
|
| Runtime tasks | `video_tasks`, `gemini_file_mappings`, `announcements`, `announcement_reads` | `repository/video_tasks`, `repository/gemini_file_mappings`, `repository/announcements` | Good candidate for generated CRUD except polling claim logic. |
|
|
| Stats | `stats_*`, `schema_backfills` | backend aggregation modules | Keep aggregation SQL per-driver; generate table/index definitions only. |
|
|
| System | `system_configs` | `repository/system` through backend dispatch | Good candidate for generated CRUD. |
|
|
|
|
## Logical Schema Coverage
|
|
|
|
Logical schema currently covers the clean baseline table set plus portable
|
|
MySQL/SQLite table-creation migrations. Postgres-only historical follow-up
|
|
migrations remain driver-specific until their schema is normalized or promoted
|
|
as explicit generated/override fragments.
|
|
|
|
## Maintenance Rules
|
|
|
|
1. Keep driver-specific SQL inside driver-specific migration/repository files.
|
|
2. Use logical type names in docs and future schema manifests, not raw database
|
|
type names.
|
|
3. Keep `jsonb` only in Postgres migrations/repositories/tests.
|
|
4. Prefer generated helpers for simple CRUD first; do not rewrite complex usage,
|
|
billing, stats, or candidate-selection queries until contract tests cover the
|
|
behavior.
|
|
5. When adding a new table, update this inventory and add it to the export domain
|
|
plan if it must move across databases.
|
|
6. If a baseline fragment changes, run `compose_schema.sh compose` before tests
|
|
so the executable SQL artifact is regenerated from the source manifest.
|