i/Aether
1
0
Fork 0
mirror of https://github.com/fawney19/Aether.git synced 2026-10-07 18:07:47 +08:00
Code Issues Packages Projects Releases Wiki Activity
3,058 Commits 6 Branches 182 Tags
625456ff248d1ac151c32c11be3d6d9115899f2e
Commit Graph
52 Commits
Author SHA1 Message Date
stabeyandClaude Opus 5.5 45678d9419 test(gateway): decouple first-request deadline test from hyper header timeout 
The partial-preface test raced a 5ms first-request deadline against a
10ms hyper header_read_timeout. On a slow runner both timers expire
before the next poll and tokio::select! may pick the connection branch,
surfacing hyper's header-timeout error instead of the clean deadline
close. Push hyper's timeout out to 30s so only the deadline can fire.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
 2026-09-28 11:46:43 +08:00
AAEE86 7e033d0571 feat(codex): add dynamic CLI client profile 2026-09-23 15:57:37 +08:00
elky 6aeadcd1d7 fix: resolve concurrency hardening lint failures 
Use typed connection admission errors, group HTTP limits, and make test lock lifetimes explicit. Handle fixture reads and remove unnecessary cloning and manual divisibility checks.
 2026-09-10 08:31:47 +08:00
elky ecc16673eb fix: harden concurrency limits and high-RPM runtime paths 
Bound request, stream, queue, and shutdown resource lifetimes. Reduce scheduler and Redis hot-path work and isolate database maintenance. Include regression coverage, load probes, and concurrency audit results.
 2026-09-10 08:14:58 +08:00
elky a90d564931 fix: restore security hardening compatibility and validation 
Restore authorized rule reveal, explicit full HTTP capture and retention, video task business fields, and valid payment URLs. Add opt-in credential preservation for trusted recovery, fix frontend type contracts and async races, and eliminate PostgreSQL test fixture resource leaks. Document audit coverage and successful fmt and CI-scoped Clippy checks.
 2026-09-07 21:14:27 +08:00
elky 522b979052 refactor(transport): remove provider DNS filtering and allowlist settings 2026-09-07 12:07:50 +08:00
elky 2281f2b754 refactor(data): remove MySQL and SQLite support 
Use PostgreSQL as the only database backend across runtime, schema tooling, installation, Compose, and CI. Update regression tests and reject removed drivers explicitly.
 2026-09-07 00:09:42 +08:00
elky e15ea0d5d3 fix(security): configure trusted Fake-IP DNS hosts 2026-09-05 14:05:07 +08:00
elky 33d5cd5993 fix(ci): align lint-safe security paths 2026-09-05 03:19:53 +08:00
elky 579f2c7cc1 feat(security): harden gateway boundaries and usage policies 
Consolidate subscription usage policy enforcement, privacy-safe persistence, and gateway security hardening into one reviewable change.

Includes bounded HTTP and execution envelopes, header and protocol guards, DNS and relay validation, authentication and secret projection hardening, secure backup/install paths, and regression coverage.
 2026-09-04 03:45:52 +08:00
fawney 2cb4d554aa feat(routing): consolidate scheduling strategy configuration 2026-09-03 11:05:59 +08:00
elky 415b2da81b feat(routing): make routing profiles the sole scheduler policy source 
Bootstrap an enabled system-default routing group from the legacy
scheduler config keys on startup, resolve the default ordering config
from that group before falling back to the legacy keys, and stop merging
keep_priority_on_conversion with the legacy flag when a policy is
resolved. Thread the policy-derived ordering config into candidate
preselection so it no longer reads system config independently.

Add per-API-format key priority overrides so a key serving several
formats keeps independent ordering, matching the legacy
global_priority_by_format semantics. Expose keep_priority_on_conversion
in the routing profile editor and read the effective policy in the
model routing preview, monitoring metrics and provider page badge.
 2026-09-02 17:04:04 +08:00
AAEE86 71b54070e8 feat(gateway): Codex/OpenAI Responses WebSocket 代理模式 
在 /v1/responses 上支持 WebSocket 升级，把客户端帧中继到上游 Codex /
OpenAI Responses WebSocket 端点，同时保持既有的路由、鉴权、配额与用量
语义：

- 路由与准入：control/route/ai.rs 识别 WebSocket 升级请求；
  websocket/ingress.rs 复用 API Key 鉴权、IP 规则与并发许可，并引入
  独立的 WebSocket 连接许可
- 中继：websocket/responses/* 按 connection / session / turn 分层，
  帧解析归一化、socket 写入有界、continuation 保持调度亲和性
- 配额：orchestration/codex_quota_breaker.rs 在账号配额耗尽时熔断并
  自动恢复，不再直接断开客户端连接
- 用量：每个 turn 的终态用量落库，request_metadata 记录
  websocket_mode / websocket_transport，管理端与 usage 视图暴露
  is_websocket
- 管理端：provider 可配置 Responses WebSocket 开关
 2026-08-17 14:50:33 +08:00
elky 29fa4aed19 perf(gateway): raise default server pool floor 2026-08-12 16:56:18 +08:00
elky 778cfb1a5c feat(data): complete portable SQL backend parity 
Align MySQL and SQLite schemas, migrations, usage, stats, export, and backfill behavior with the shared data contracts. Extend gateway startup and maintenance support across all SQL drivers.
 2026-07-25 21:28:21 +08:00
elky fc92c4f431 perf(gateway): scale request hot paths for 20k streams 
Shard and singleflight hot-path caches, batch and prioritize candidate and usage lifecycle persistence, and extend database and pressure-test instrumentation for 20k concurrent streams.
 2026-07-22 02:11:08 +08:00
elky 9a47267545 fix(usage): bound terminal event persistence 
Add end-to-end terminal admission, bounded database fallback, and observable overload handling. Preserve first-byte lifecycle state across asynchronous runtime and frontend updates.
 2026-07-16 13:12:30 +08:00
elky 8616fe6ee2 refactor(workspace): enforce layered crate boundaries 2026-07-15 23:47:19 +08:00
elky a728c090a9 fix(gateway): scope concurrency helper to tests 2026-07-12 22:36:48 +08:00
elky 7f61bb43c7 feat(security): harden gateway request and runtime controls 2026-07-12 14:10:54 +08:00
elky 1ac16005f9 Stabilize usage worker autoscale tests 2026-07-02 17:28:25 +08:00
elky f179ee72f9 chore: update gateway pressure observability 2026-06-30 17:01:39 +08:00
elky 7e9424008f Add usage queue worker autoscaling 2026-06-26 14:02:57 +08:00
elky 6f00e9fc67 Improve gateway transport and usage runtime 2026-06-25 22:36:27 +08:00
elky d336d1a7fa Improve gateway scheduling and runtime admission 2026-06-24 01:53:45 +08:00
elky f75894acbb perf: reduce gateway db pressure under load 2026-06-22 00:08:48 +08:00
elky 279735ae7f Auto-size SQL pool defaults 2026-06-21 11:15:40 +08:00
fawney19 ab0a90de97 fix(runtime-state): govern redis connections 2026-05-21 22:53:37 +08:00
fawney19 b84e4a96e2 chore: tune default postgres settings for 2c4g 2026-05-21 01:36:05 +08:00
HsungKayphoon f9f1fa928a refactor: drive pg sqlite copy from target schema 2026-05-16 17:29:14 +08:00
HsungKayphoon 4b12ec8913 feat: add postgres to single-node migration 2026-05-16 15:26:43 +08:00
fawney19 209322b499 feat(gateway): unify background task runtime and storage 2026-05-09 21:19:29 +08:00
fawney19 6247ac3edc refactor: extract runtime state backends 2026-05-08 00:18:12 +08:00
fawney19 fce7e959e5 Add multi-database data layer 
Introduce aether-data-schema and driver-specific schema generation for Postgres, MySQL, and SQLite.

Split data backends, lifecycle, repositories, and gateway runtime integration across database drivers.

Verified with cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings, and cargo test --workspace.
 2026-05-05 18:27:36 +08:00
fawney19 e5bbc797e0 chore(gateway): 调小 Postgres 连接池默认值 
- max_connections 100 → 20
- idle_timeout 60s → 30s
- .env.example 补充连接池相关变量注释
 2026-04-25 09:53:21 +08:00
fawney19andEntropy.Xu fa328e18a1 feat: provider api_formats 可空继承、OpenAI 图片 edit/variation 与用量配额多项补强 
- 鉴权: provider_api_keys.api_formats 改为可空，OAuth 托管 key 自动继承 provider endpoints 激活格式，相关 handler/测试同步更新
- 图片 planner: OpenAI 图片路由新增 edit/variation 操作并完善参数校验、响应合并与流式处理
- 用量: user me usage 返回区分 client_requested_stream/upstream_is_stream，前端 usage 列表筛选与展示增强
- 统计: stats_daily_model 新增 cache_creation_ephemeral_5m/1h tokens 字段与回填链路
- 配额/observability: quota repository 新增内存与 SQL 扩展，admin observability usage 字段扩充
- 其它: OAuth 导入/轮询收敛、provider 汇总与 pool admin 读写链路小修、新增 system_config 缓存与 provider template handler

Closes #318

Co-authored-by: Entropy.Xu <[email protected]>
 2026-04-23 14:42:51 +08:00
fawney19 c8d7dbd8d6 refactor(gateway): 将 CF 头剥离中间件下移至各 router 构建函数并重构为前缀匹配 2026-04-22 21:10:16 +08:00
fawney19 62153d7d36 feat(gateway): 新增 --auto-prepare-database 启动选项并在 docker-compose 默认开启 
- aether-gateway 启动时可自动执行挂起的 migration 与 backfill
- docker-compose.yml 默认开启 AETHER_GATEWAY_AUTO_PREPARE_DATABASE
- 同步更新 .env.example、README 与 deploy.sh 的部署说明
 2026-04-22 18:20:24 +08:00
fawney19 a5e6bd3b62 feat(stats): 新增聚合读路径与回填机制并重构 dashboard/usage 读取链路 
- 新增 stats_user_summary 及 user_daily_provider/api_format/cost_savings 等聚合表
- 扩展 stats_daily/hourly 有效 token 与响应时间等字段，maintenance runtime 同步写入
- 新增 backfill 模块与 --apply-backfills 命令补齐历史聚合数据
- 重写 dashboard_filters、usage_heatmap、user_rollups 查询改走聚合表
- 同步更新 baseline_v2.sql 与 migration 集，README/dev.sh 补充回填用法
 2026-04-22 17:29:39 +08:00
fawney19 5bb08e6aa4 feat(gateway): 重构 usage 数据层、迁移系统与系统导入 
数据库迁移:
- 引入 baseline v2 bootstrap，空库首次启动自动初始化
- 服务启动不再自动执行迁移，需显式 `--migrate` 运行
- 新增 pending migration 检测，schema 落后时拒绝启动

Usage 数据层:
- usage body 存储外部化为独立 blob 表
- 新增 HTTP audit 表拆分存储请求/响应头与 body ref
- 后台清理任务支持 legacy body ref 元数据迁移
- usage runtime 写入迁移到专用 tokio runtime（独立线程池, 8MB 栈）

系统导入/导出:
- 支持用户、API Keys、钱包数据的完整导入
- 兼容 legacy 与 v1.3+ 两种导出格式

其他改进:
- executor outcome 增加 runtime miss 诊断上下文
- 主 tokio runtime 栈大小调整为 8MB
- 前端 provider 管理支持 base URL 配置
- dev.sh 支持 --migrate 参数
 2026-04-13 14:01:22 +08:00
fawney19 9703840a36 feat(proxy): 实现代理节点批量升级回滚、隧道重定向跟随及远程配置管理 
核心功能：
- 新增代理节点批量升级回滚工作流，支持分批升级、健康探针、跳过/重试/取消等操作
- proxy 隧道流处理器支持 HTTP 重定向跟随（最多 10 跳），区分 307/308 可重播与不可重播请求体
- proxy 协议新增 follow_redirects / http1_only 字段，网关侧同步支持
- 新增代理节点远端配置变更接口（名称、允许端口、调度状态、升级目标等）
- 新增代理节点注册/反注册/心跳的 Admin API，及节点过期清理维护任务
- gateway 隧道 owner-relay 支持流式代理大请求体，新增 5 MiB 默认限制
- 新增 ProxyNodeRegistrationMutation / ProxyNodeRemoteConfigMutation 数据类型
- proxy 配置新增重定向重播预算、心跳间隔等参数，TUI 安装向导同步更新
- 前端 ProxyNodes 页面新增批量升级操作面板及滚动进度展示
 2026-04-12 16:02:38 +08:00
fawney19 801e16c988 refactor(gateway): 统一 AETHER_GATEWAY_BIND 为 APP_PORT，新增 API Key 前缀配置和启动自举管理员 
- 绑定地址固定 0.0.0.0，仅通过 APP_PORT 控制端口，简化 CLI/Docker/systemd/dev.sh/前端代理全链路
- 新增 API_KEY_PREFIX 环境变量，抽取 handlers/shared/api_keys.rs 消除 admin/public 重复逻辑
- 新增 bootstrap_admin.rs，启动时通过 ADMIN_* 环境变量在无管理员时自动创建首个本地管理员
- 前端密码输入改用 type=password，API Key 占位符改为动态前缀
- 删除过时的 pyproject.toml/uv.lock 和旧部署文档
- 更新 .env.example/README 反映新配置项
 2026-04-11 17:39:02 +08:00
fawney19 8cd2c4d5bd fix(gateway): 默认监听端口改为 8084 以支持非 root 运行 
- Dockerfile 及 main.rs 默认 bind 从 0.0.0.0:80 改为 0.0.0.0:8084
- docker-compose 以 UID:GID 非 root 用户启动 app 容器
- compose 端口映射与容器内监听端口保持一致，通过 AETHER_GATEWAY_BIND 注入
 2026-04-11 13:55:11 +08:00
fawney19 f68c67021c refactor(build): 切换到 musl 交叉编译 + distroless 镜像方案 
- docker-publish.yml: 改为 cross 交叉编译 amd64/arm64 musl 静态二进制,
  前端在 CI 独立构建, buildx 组装多架构镜像
- Dockerfile.app: 从 139 行容器内编译简化为 24 行纯 COPY 打包
- Dockerfile.app.local: 移除 jemalloc 动态链接 (LD_PRELOAD/libjemalloc2)
- aether-gateway: 引入 tikv-jemallocator 静态链接 jemalloc
 2026-04-11 12:21:33 +08:00
fawney19 cf02a10050 chore: 调整 Postgres 连接池默认参数：增大最大连接数、缩短获取超时 2026-04-11 02:26:33 +08:00
fawney19 6144473ebe feat: 新增 frontdoor 执行回环守卫与多项可观测性增强 
- 新增 frontdoor_loop_guard 模块，检测并拒绝 execution runtime 回环到本地网关的请求（HTTP 508）
- candidate loop 引入 span tracking、执行尝试日志与流式看门狗超时
- 本地故障转移策略支持从 report_context 加载，新增 append_local_failover_policy_to_value
- runtime tracing 美化：移除 identity 前缀，按 span 深度树形缩进，target 固定宽度展示
- Codex OpenAI CLI 补齐 chatgpt-account-id/x-client-request-id/session_id/conversation_id 请求头
- OpenAI CLI same/cross-format 聚合规则放宽以支持 openai:compact 客户端格式，并过滤 error-like 响应体
- auth/proxy/finalize 日志补充 user_id/api_key_id/api_key_name/balance_remaining 等字段
- 启动日志拆分为 starting/ready/config 三段，新增 resolve_bind_http_base_url
- access_log middleware 将生成的 trace_id 回注到下游请求头
- Cargo.toml 启用 serde_json preserve_order 特性
 2026-04-11 01:50:24 +08:00
fawney19 b0b40c16ff feat: 全栈功能增强 - 扩展 provider/pool 管理、完善调度与数据层、重构前端 Pool 页面 
后端:
- 扩展 pool_admin payloads 和 provider query models，增强 endpoint key 管理
- 完善 scheduler-core 候选排序与请求候选逻辑
- 增强 usage-runtime 写入、provider-transport 网络层与 OAuth 刷新
- 改进 AI pipeline 响应转换与流式处理
- 扩展 global_models/provider_catalog 数据层查询能力
- 增强 video-tasks-core 多 provider 支持
- 新增大量集成测试覆盖 pool/keys/provider_query/frontdoor

前端:
- 重构 PoolManagement 页面，拆分状态管理/对话框逻辑到独立模块
- 新增 poolAdvancedDialog/poolSchedulingDialog/poolManagementState/poolMobilePresentation 工具函数及测试
- 改进 Dialog 组件与 provider tabs 显示

部署:
- 更新 Rust CI workflow 和 Dockerfile 构建配置

Closes #275
Co-authored-by: AAEE86 <[email protected]>
 2026-04-09 13:51:50 +08:00
fawney19 5d96d6673b refactor: 大规模模块拆分与代码精简，新增 ai-pipeline/data-contracts 独立 crate 
- 新增 aether-ai-pipeline 和 aether-data-contracts crate，将 pipeline 逻辑与数据契约从 gateway 中解耦
- 重构 admin handlers：拆分单体模块为 auth/billing/endpoint/features/model/observability/provider/system 等独立子模块
- 合并 chat/cli 重复代码路径：精简 conversion、finalize、planner 中的 sync/chat/cli 分支
- 重构 scheduler/executor/data 层，引入 facade 模式降低模块间耦合
- 移除冗余的 intent 模块，将 plan_fallback/policy/stream_path/sync_path 迁移至 executor
- 前端适配：调整 admin API 调用和 provider 模型测试对话框
 2026-04-07 02:50:19 +08:00
fawney19 763ff03a7b refactor: 拆分 gateway 单体为独立 crate，新增 systemd 部署方案 
将 gateway 内部的 model-fetch、provider-transport、scheduler-core、
usage-runtime、video-tasks-core 模块提取为独立 crate；重构 gateway
内部模块结构（state/router/cache/data/query 等）；移除大量遗留模块
文件；新增 systemd 二进制部署骨架及相关文档；更新前端 usage 相关
API 和组件。
 2026-04-05 20:23:16 +08:00
fawney19 cbc811f6ce refactor: 移除 Python upstream 依赖，清理全部 legacy/Python 兼容层 
- 移除 upstream_base_url 参数及 AETHER_GATEWAY_UPSTREAM 环境变量，gateway 不再需要指向 Python 宿主
- 删除所有 LEGACY_*/PYTHON_* 常量、路由组、header 定义及 sunset/phaseout 机制
- 将 legacy_gateway_bridge 重命名为 internal_gateway，executor 相关命名统一为 execution_runtime
- dev.sh 新增 Postgres/Redis 预检查，移除 upstream 相关启动参数和提示
- 新增 ai_public 路由处理器
- 全量适配 handler、test、state、control 等模块的命名和接口变更
 2026-04-04 01:40:24 +08:00