The cleanup loop break condition used rows_affected() from the UPDATE
statement, but for rows that only had blob/audit refs (no inline
compressed body data), the UPDATE reported 0 affected rows. This caused
the loop to exit after the first batch, skipping the majority of
candidates.
Change the break condition in all 4 cleanup functions from:
if cleaned == 0 || cleaned < batch_size
to:
if rows.len() < batch_size
This ensures the loop continues as long as SELECT returns a full batch,
regardless of how many rows the UPDATE actually modified.
Affected functions:
- cleanup_usage_raw_body_fields
- cleanup_usage_compressed_body_fields
- cleanup_usage_header_fields
- cleanup_usage_stale_body_fields
- extract reasoning effort from provider request body output_config.effort
- include output_config.effort in usage list fallback SQL
- cover the new request body shape in usage metadata tests
- Rename the original health monitor to endpoint health monitor
- Add tab navigation for endpoint, model, and provider health views
- Add model health monitor cards with availability, latency, first-byte latency, and 60-point history
- Add admin-only provider health monitor with collapsible active-provider sections
- Show per-provider model health cards after expanding a provider
- Add backend model health and provider health monitor payload builders
- Add admin endpoint for provider health monitoring
- Add provider-scoped usage breakdown filtering for per-provider model statistics
- Add frontend API types and request helpers for model/provider health data
- Add demo mock data for model and provider health monitoring
- Fix model health timeline time-unit handling so request history segments render correctly
Verification:
- cargo fmt
- npm run type-check
- npm run build
- cargo test -p aether-gateway health_models
- cargo test -p aether-gateway health_providers
- cargo test -p aether-gateway gateway_exposes_frontdoor_manifest_without_proxying_upstream
- extract provider reasoning effort from request body metadata
- extract priority service tier and expose it as service_tier
- show reasoning level and fast badges after model names
- include badges in active request updates and usage list payloads
- add targeted backend and frontend coverage
The stale-pending cleanup task previously hardcoded status_code=504 and a
generic timeout message for every usage row it finalized. When a request
had already been observed as failing — e.g. upstream Connection reset by
peer, watchdog 504, or an authenticated 4xx — the cleanup overwrote that
context with a misleading "服务器超时" outcome and 504 status, hiding the
real cause from the dashboards and customer.
Pull the most recent failed/cancelled candidate per stale request_id and,
if present, finalize the usage row with the candidate's status_code
(defaulting to 502 when none was recorded) and error_message. Requests
that have no terminal candidate (truly stuck pending/streaming) keep the
existing 504 + timeout-message behavior, since they really are timeouts
from the cleanup's perspective. Applied to all three SQL backends with
parameterized UPDATE statements.
The Postgres failed-candidate lookup orders by
COALESCE(finished_at, started_at, created_at) DESC, matching the MySQL
and SQLite ORDER BY clauses so the three backends pick the same
"most recent terminal candidate" under every NULL combination of timing
columns.
Co-Authored-By: Claude Opus 4.7 <[email protected]>
fawney19