mirror of
https://github.com/fawney19/Aether.git
synced 2026-01-08 02:32:27 +08:00
feat: add TTFB timeout detection and improve stream handling
- Add stream first byte timeout (TTFB) detection to trigger failover when provider responds too slowly (configurable via STREAM_FIRST_BYTE_TIMEOUT) - Add rate limit fail-open/fail-close strategy configuration - Improve exception handling in stream prefetch with proper error classification - Refactor UsageService with shared _prepare_usage_record method - Add batch deletion for old usage records to avoid long transaction locks - Update CLI adapters to use proper User-Agent headers for each CLI client - Add composite indexes migration for usage table query optimization - Fix streaming status display in frontend to show TTFB during streaming - Remove sensitive JWT secret logging in auth service
This commit is contained in:
@@ -376,6 +376,9 @@ class BaseMessageHandler:
|
||||
|
||||
使用 asyncio 后台任务执行数据库更新,避免阻塞流式传输
|
||||
|
||||
注意:TTFB(首字节时间)由 StreamContext.record_first_byte_time() 记录,
|
||||
并在最终 record_success 时传递到数据库,避免重复记录导致数据不一致。
|
||||
|
||||
Args:
|
||||
request_id: 请求 ID,如果不传则使用 self.request_id
|
||||
"""
|
||||
@@ -407,6 +410,9 @@ class BaseMessageHandler:
|
||||
|
||||
使用 asyncio 后台任务执行数据库更新,避免阻塞流式传输
|
||||
|
||||
注意:TTFB(首字节时间)由 StreamContext.record_first_byte_time() 记录,
|
||||
并在最终 record_success 时传递到数据库,避免重复记录导致数据不一致。
|
||||
|
||||
Args:
|
||||
ctx: 流式上下文,包含 provider_name 和 mapped_model
|
||||
"""
|
||||
|
||||
Reference in New Issue
Block a user