Retry pre-response transport failures across candidates with an explicit stop policy, and propagate end-to-end timing into usage records and UI diagnostics.
Remove legacy body, import, cookie, PII, and tunnel replay caps while preserving optional operator-configured gateway limits.
Preserve exact request payloads and model client surface and API operation explicitly.
Add Anthropic compatibility profiles, bounded stream commitment, and scoped OAuth retry behavior across provider transports.
Shard and singleflight hot-path caches, batch and prioritize candidate and usage lifecycle persistence, and extend database and pressure-test instrumentation for 20k concurrent streams.
The "upstream_is_stream" JSON key flows from the AI execution report
context producer (aether-ai-serving::report_context) through several
consumers — usage runtime metadata copy/move, gateway watchdog, sync
execution decision, observability handlers, and the per-driver usage
repositories. Each site spelled the key as a bare string literal, so a
producer-side rename would silently degrade every consumer to its
fallback (typically assuming streaming) with no compile-time signal.
Introduce a single pub const UPSTREAM_IS_STREAM_KEY in
aether-ai-formats (the lowest crate every consumer already depends on),
re-export from the crate root, and route producer + all map-style
consumers through it. The change is purely a string-literal → constant
swap; behaviour is identical.
Sites left as literals (intentional):
- `json!({"upstream_is_stream": ...})` macro keys, which must be string
literals at the macro layer; these are also API-response payload
field names (an external contract that should not silently track
internal report-context renames).
- SQL column accessors (`try_get::<...>("upstream_is_stream")`), which
refer to the database schema column, not the JSON key.
- Test fixtures and assertions, which validate the on-the-wire contract
and should keep verifying the actual string.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
ZheFox