i/Aether
1
0
Fork 0
mirror of https://github.com/fawney19/Aether.git synced 2026-09-03 01:40:21 +08:00
Code Issues Packages Projects Releases Wiki Activity
2,021 Commits 6 Branches 174 Tags
9af15072380c1e2757e3d34fc6f1a63e3a060c34
Commit Graph

43 Commits

Author SHA1 Message Date
zhefox
507f2f8250 fix(usage): treat stream terminal failures as failures on HTTP 200 2026-05-22 09:34:22 +08:00
fawney19
0e6d7539ad Merge remote-tracking branch 'origin/pr/538' 2026-05-21 23:12:42 +08:00
fawney19
d6eb41aa78 Merge remote-tracking branch 'origin/pr/536' 
# Conflicts:
#	apps/aether-gateway/src/execution_runtime/stream/execution.rs
 2026-05-21 23:03:14 +08:00
zhefox
77c2d91eb0 fix(gateway): drain downstream-disconnected streams and stop inferring cancelled usage 2026-05-21 20:30:44 +08:00
zhefox
12abde2aeb fix(usage): handle terminal stream failures and preserve usage updates 2026-05-21 15:52:51 +08:00
stabey
3330b2ac4c refactor(report-context): extract UPSTREAM_IS_STREAM_KEY constant 
The "upstream_is_stream" JSON key flows from the AI execution report
context producer (aether-ai-serving::report_context) through several
consumers — usage runtime metadata copy/move, gateway watchdog, sync
execution decision, observability handlers, and the per-driver usage
repositories. Each site spelled the key as a bare string literal, so a
producer-side rename would silently degrade every consumer to its
fallback (typically assuming streaming) with no compile-time signal.

Introduce a single pub const UPSTREAM_IS_STREAM_KEY in
aether-ai-formats (the lowest crate every consumer already depends on),
re-export from the crate root, and route producer + all map-style
consumers through it. The change is purely a string-literal → constant
swap; behaviour is identical.

Sites left as literals (intentional):
- `json!({"upstream_is_stream": ...})` macro keys, which must be string
  literals at the macro layer; these are also API-response payload
  field names (an external contract that should not silently track
  internal report-context renames).
- SQL column accessors (`try_get::<...>("upstream_is_stream")`), which
  refer to the database schema column, not the JSON key.
- Test fixtures and assertions, which validate the on-the-wire contract
  and should keep verifying the actual string.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-21 15:26:42 +08:00
zhefox
e59e6c3797 fix(gateway): sanitize Claude thinking and handle missing stream finish 2026-05-21 12:30:42 +08:00
Entropy.Xu
6b04a0a3a6 fix(windsurf): 修复 native 工具流式回程 2026-05-21 02:55:05 +08:00
Entropy.Xu
0226e14251 feat(provider): 原生接入 Windsurf provider 2026-05-21 01:02:02 +08:00
fawney19
052de6b96e test: stabilize merged PR checks 2026-05-19 10:18:25 +08:00
ZheFox
995ab302be Merge remote-tracking branch 'upstream/main' 
# Conflicts:
#	apps/aether-gateway/src/handlers/admin/provider/endpoints_admin/payloads.rs
#	apps/aether-gateway/src/handlers/admin/provider/endpoints_admin/reads.rs
#	apps/aether-gateway/src/handlers/admin/provider/endpoints_admin/update.rs
#	apps/aether-gateway/src/tests/control/admin/endpoints/routes.rs
#	frontend/src/features/models/components/GlobalModelFormDialog.vue
#	frontend/src/features/providers/components/ProviderModelFormDialog.vue
#	frontend/src/features/providers/components/provider-tabs/__tests__/model-test-request.spec.ts
#	frontend/src/features/providers/components/provider-tabs/model-test-request.ts
 2026-05-18 22:02:31 +08:00
ZheFox
0b2a8fafce feat(billing): add image output pricing and usage tracking 2026-05-18 02:49:56 +08:00
ZheFox
691ccaaa04 fix(usage): track OpenAI image SSE completion and usage estimates 2026-05-18 01:32:45 +08:00
Kayphoon
ae472d6744 fix(data): require provider id for provider usage aggregation 2026-05-18 00:36:29 +08:00
ZheFox
680b617b00 feat(gateway): route OpenAI image streams through chat bridge 2026-05-17 23:09:50 +08:00
ZheFox
ab0d766f47 fix(usage): stop inferring cache reads from prompt_cache_key 2026-05-17 03:04:33 +08:00
ZheFox
b903f8ff7d fix(usage): estimate cache read tokens for cancelled requests 2026-05-17 02:13:13 +08:00
ZheFox
0483d001b4 fix(usage): bill cancelled terminal usage and preserve total token estimates 2026-05-17 01:11:42 +08:00
fawney19
79d07ac79a Improve request trace upstream diagnostics 2026-05-09 10:46:04 +08:00
fawney19
6083461822 Add editable response header rules 2026-05-06 01:29:42 +08:00
fawney19
f959f02d40 Implement transport profile routing 2026-05-05 22:21:23 +08:00
jiuwovo-ai
3e2eca4fd0 fix: 修复手动代理节点请求不计数及延迟/心跳不显示的问题 (#368) 
* fix: 修复手动代理节点请求不计数及延迟/心跳不显示的问题

问题描述:
- 手动添加的代理节点请求数始终为0，不会递增
- 手动代理节点的延迟和最后心跳时间不显示

根因:
Rust 重写版中缺失了 Python 版的手动代理节点请求计数逻辑。
隧道节点通过心跳上报计数，但手动节点没有心跳机制，
需要在 usage recording 路径中递增计数并更新延迟信息。

修复方案:
1. 在 ExecutionPlan 中提取 proxy 信息，注入到 request_metadata
2. 在 request_metadata 白名单中添加 proxy 字段
3. 新增 INCREMENT_MANUAL_PROXY_NODE_REQUESTS_SQL，
   递增 total_requests/failed_requests，
   同时更新 avg_latency_ms 和 last_heartbeat_at
4. 在 ProxyNodeWriteRepository trait 新增 increment_manual_node_requests 方法
5. 在 write_event_record 中解析 request_metadata 中的 proxy 信息，
   对非隧道模式的手动节点调用递增方法
6. 为 GatewayDataState 实现 ManualProxyNodeCounter trait

影响范围:
- 仅影响手动代理节点的统计数据
- 隧道节点不受影响（继续通过心跳计数）
- 不影响请求转发逻辑

* style: cargo fmt

---------

Co-authored-by: root <root@ser406777952330.local>
 2026-05-03 14:50:36 +08:00
fawney19
7fcb9e6292 feat(usage): record request origin metadata (#362) 
Record client IP and User-Agent in usage request_metadata for local execution requests without adding schema, API, or table fields.

References #357 and supersedes #358.

Co-authored-by: RWDai <27391645+RWDai@users.noreply.github.com>
 2026-04-30 11:26:56 +08:00
fawney19
07a319259b Normalize canonical API formats 2026-04-29 10:20:41 +08:00
fawney19
1d23bf4ecb Isolate standalone key wallet handling 2026-04-27 17:24:31 +08:00
fawney19
ea3dc3257e clean up legacy openai cli adapter names 2026-04-26 23:59:53 +08:00
fawney19
5b914aa78c migrate ai format conversion to responses adapters 2026-04-26 23:59:53 +08:00
fawney19
f30b1f3f6b fix(usage): 流式终端 usage 以更完整值为准，Codex CLI 显式选择 response 解析器 
- 新增 StandardizedUsage::signal_score/is_more_complete_than/choose_more_complete，流式合并与终端落库均按信号完整度择优
- OpenAI Chat/CLI 解析器支持仅 usage 的终结 chunk 与 response.completed usage
- Codex provider 注入 provider_stream_event_api_format=openai:cli，解析器选择改由 report_context 显式决定
- usage_mapper 扩展嵌套 response/message/item 兼容 Claude message_start/message_delta 及 Gemini stream chunks
- usage SQL upsert 在终态（completed/failed/cancelled）时按 GREATEST 写入 token/费用镜像列
 2026-04-25 00:57:01 +08:00
fawney19
db1059e73b fix(usage): 修复 Gemini total 重复累加 cache_read 并补充测试 
- write.rs: Gemini usage 提取 total 时不再叠加 cachedContentTokenCount
- 补充 Claude 大 cache_read 场景下 input_tokens 不被扣减的测试
- 补充前端 getEffectiveInputTokens 对 Claude 格式不减 cache_read 的测试
 2026-04-24 21:20:45 +08:00
fawney19
780f09c1a2 feat(billing): 引入 model_id 精确计费查找路径，传播模型 ID 至用量事件与 report context 
- UsageEventData 新增 model_id / global_model_id 字段，write.rs seed 结构体同步补充
- report_context 新增 model_id / global_model_id / global_model_name，各 payload 构建时从 candidate 传入
- event_enrichment 优先按 model_id 精确查找计费上下文，回退为按名称多轮查找（保留 NoRule 结果降级逻辑）
- BillingReadRepository 新增 find_model_context_by_model_id，memory/sql 分别实现；SQL 查询重构支持按 provider_model_name 和 mappings 匹配并按优先级排序
- pricing.rs 修复：model_tiered_pricing 为空 tiers 时回退到 default_tiered_pricing
- request_metadata 允许字段列表补充 model_id / global_model_id / global_model_name
- admin usage 路由信息输出及脱敏字段列表同步新增三个 model 相关字段
 2026-04-24 14:40:28 +08:00
fawney19
342d4a268c feat(stream-bridge): 支持上游 sync 响应转 SSE 流式输出，记录 client/upstream 流模式差异 
- 新增 sync_to_stream 桥接模块，将非 SSE 上游响应转换为 SSE 格式回传给流式客户端
- stream_pump 检测非 SSE 响应头后缓冲整包并通过桥接逻辑重写为 SSE 帧
- proxy handler 同步支持 sync→stream 聚合与转换（覆盖 openai/claude/gemini 四种格式）
- sync_products 补全 openai:cli 的完整流式事件聚合（text delta、reasoning、tool call 等）
- usage runtime 写入 client_requested_stream / upstream_is_stream 到 request_metadata
- SQL 查询层将两个布尔字段从 request_metadata jsonb 中提取并回传给前端
- 前端 status.ts 新增 resolveUsageStreamLabelSegments，优先读取 client_requested_stream
- RequestDetailDrawer 在流式转换场景下显示"客户端→上游"两段 Badge
 2026-04-23 21:53:50 +08:00
fawney19
25a2b417be refactor: 优化调度候选排序与用量写入链路并改进 Fernet 缓存与前端批量列表 2026-04-21 16:19:07 +08:00
fawney19
2f487fda66 fix(usage): 修复缓存创建 token 展示并在总量中计入缓存组件 
- gateway/admin 用量载荷在 cache_creation_input_tokens 为 0 时回填 ephemeral_5m/1h 合计
- 标准化用量写入时将 cache_creation/cache_read token 计入 total_tokens
- 前端列表与轮询同步新增分类字段，修复缓存 token 列显示
 2026-04-19 02:46:28 +08:00
fawney19
3363592751 Refactor usage body capture and stream terminal reporting 2026-04-18 17:48:21 +08:00
fawney19
53acfbabf6 refactor(gateway): 重构 ai pipeline 规划链路 2026-04-14 01:27:04 +08:00
fawney19
5bb08e6aa4 feat(gateway): 重构 usage 数据层、迁移系统与系统导入 
数据库迁移:
- 引入 baseline v2 bootstrap，空库首次启动自动初始化
- 服务启动不再自动执行迁移，需显式 `--migrate` 运行
- 新增 pending migration 检测，schema 落后时拒绝启动

Usage 数据层:
- usage body 存储外部化为独立 blob 表
- 新增 HTTP audit 表拆分存储请求/响应头与 body ref
- 后台清理任务支持 legacy body ref 元数据迁移
- usage runtime 写入迁移到专用 tokio runtime（独立线程池, 8MB 栈）

系统导入/导出:
- 支持用户、API Keys、钱包数据的完整导入
- 兼容 legacy 与 v1.3+ 两种导出格式

其他改进:
- executor outcome 增加 runtime miss 诊断上下文
- 主 tokio runtime 栈大小调整为 8MB
- 前端 provider 管理支持 base URL 配置
- dev.sh 支持 --migrate 参数
 2026-04-13 14:01:22 +08:00
fawney19
010ab127e2 feat: 扩展 cache creation token 细分统计与 effective_input_tokens 计费逻辑 
- 新增 cache_creation_ephemeral_5m/1h_input_tokens 字段，区分不同 TTL 的缓存写入 token
- 引入 effective_input_tokens（扣除 cache read 后的有效输入 token），暴露给 usage 接口
- billing 规则生成器支持 5m/1h ephemeral cache 独立定价与分级计费
- usage_mapper 增加 Claude/Anthropic 格式映射，修复 OpenAI responses 格式字段兼容性
- 迁移逻辑增强：支持 checksum 容错、applied/pending 数量日志、逐步执行信息输出
- executor 抽离 LocalExecutionRequestOutcome 类型，统一 sync/stream 路径返回语义
- provider-transport auth 层新增 complete passthrough headers 构建逻辑
- 前端 usage 类型全面补充 effective_input_tokens、cache_creation_tokens、total_input_context 字段
 2026-04-10 17:44:55 +08:00
fawney19
5014e2f5fd refactor: 抽离 AI pipeline 与调度共享能力逻辑 2026-04-10 01:46:14 +08:00
fawney19
fe81eafe2c feat: 增强 provider query 端点选择逻辑，支持 openai:responses 格式与 key 级 api_formats 过滤 
- provider_query_selected_fetch_endpoints 支持按 key.api_formats 过滤可用端点
- 端点选择增加 endpoint_supports_rust_models_fetch 检查
- 支持优先级列表之外的自定义 api_format 端点
- 新增 openai:responses 端点的 provider query 集成测试
- 各 crate 补充缺失的 tracing instrument 和小修正
- 前端 Pool 管理页面补充状态逻辑
 2026-04-09 13:55:18 +08:00
fawney19
b0b40c16ff feat: 全栈功能增强 - 扩展 provider/pool 管理、完善调度与数据层、重构前端 Pool 页面 
后端:
- 扩展 pool_admin payloads 和 provider query models，增强 endpoint key 管理
- 完善 scheduler-core 候选排序与请求候选逻辑
- 增强 usage-runtime 写入、provider-transport 网络层与 OAuth 刷新
- 改进 AI pipeline 响应转换与流式处理
- 扩展 global_models/provider_catalog 数据层查询能力
- 增强 video-tasks-core 多 provider 支持
- 新增大量集成测试覆盖 pool/keys/provider_query/frontdoor

前端:
- 重构 PoolManagement 页面，拆分状态管理/对话框逻辑到独立模块
- 新增 poolAdvancedDialog/poolSchedulingDialog/poolManagementState/poolMobilePresentation 工具函数及测试
- 改进 Dialog 组件与 provider tabs 显示

部署:
- 更新 Rust CI workflow 和 Dockerfile 构建配置

Closes #275
Co-authored-by: AAEE86 <ppk0227@hotmail.com>
 2026-04-09 13:51:50 +08:00
fawney19
4fc95adfb9 refactor: 大规模模块拆分与重组，新增 aether-admin crate 
- 新建独立 aether-admin crate 承载 admin 相关共享契约与纯辅助函数
- 拆分 ai_pipeline 下 kiro/private_envelope/conversion/planner 等大文件为子模块目录
- 重组 admin handlers 各业务域（billing/oauth/provider/system/users 等）为目录结构，移除 shared.rs/builders.rs 等反模式
- 移除 ai_pipeline runtime adapters 旧实现（claude/openai/gemini/kiro/vertex/antigravity 等），改由 provider transport 统一承载
- 移除 control_facade/execution_facade/auth_snapshot_facade 等冗余 facade 层
- 拆分 query/billing 与 query/monitoring 模块、state/runtime/payments 与 security 模块
- 扩展架构测试覆盖 admin_billing/admin_model/admin_users 等新模块
- 删除 docs/architecture/refactor-execution-plan.md 已完成的执行计划文档
 2026-04-09 00:10:38 +08:00
fawney19
5d96d6673b refactor: 大规模模块拆分与代码精简，新增 ai-pipeline/data-contracts 独立 crate 
- 新增 aether-ai-pipeline 和 aether-data-contracts crate，将 pipeline 逻辑与数据契约从 gateway 中解耦
- 重构 admin handlers：拆分单体模块为 auth/billing/endpoint/features/model/observability/provider/system 等独立子模块
- 合并 chat/cli 重复代码路径：精简 conversion、finalize、planner 中的 sync/chat/cli 分支
- 重构 scheduler/executor/data 层，引入 facade 模式降低模块间耦合
- 移除冗余的 intent 模块，将 plan_fallback/policy/stream_path/sync_path 迁移至 executor
- 前端适配：调整 admin API 调用和 provider 模型测试对话框
 2026-04-07 02:50:19 +08:00
fawney19
763ff03a7b refactor: 拆分 gateway 单体为独立 crate，新增 systemd 部署方案 
将 gateway 内部的 model-fetch、provider-transport、scheduler-core、
usage-runtime、video-tasks-core 模块提取为独立 crate；重构 gateway
内部模块结构（state/router/cache/data/query 等）；移除大量遗留模块
文件；新增 systemd 二进制部署骨架及相关文档；更新前端 usage 相关
API 和组件。
 2026-04-05 20:23:16 +08:00