Commit Graph
13 Commits
Author SHA1 Message Date
elky 7323d41fbe feat(routing): move sticky-key retries into routing policy with lazy attempts
Replace the provider/endpoint max_retries fields as the source of same-key
retries with a routing policy setting, sticky_key_attempts (default 2). Only
the first-ranked candidate is retried on the same key; every failover
candidate gets a single attempt so failover keeps advancing instead of
retrying each fallback key.

Materialize exactly one attempt per candidate and derive same-key retries in
the attempt loop after a candidate-scoped failure, so the retry budget no
longer inflates up-front materialization and needs no upper bound. The budget
travels in the report context; retries reuse the plan with a fresh candidate
id and incremented retry index. Pool groups only retry their first key within
the retry-index stride.

Expose the setting in the routing profile editor and the set_scheduling rule
action, and drop the max_retries input from the provider form.
2026-09-02 20:48:40 +08:00
elky 4148ab1931 fix(routing): harden routed pool scheduling 2026-07-27 22:06:28 +08:00
elky 10d369f59c feat(providers): add provider transfer limits 2026-07-26 15:06:56 +08:00
MMEXA dfa121dd5b feat(openai): align GPT-5.6 and Codex request contracts 2026-07-11 07:40:12 +08:00
elky e9b4efc2d4 fix(ai-serving): preserve explicit request encoding 2026-06-10 18:16:55 +08:00
fawney19 92813e6122 feat: add routing profile scheduling policies 2026-05-18 11:03:49 +08:00
fawney19 e3574e1918 Track scheduler affinity epochs and key sorting 2026-05-11 01:45:49 +08:00
fawney19 bacb14e5f0 refactor: lazy pool key scheduling 2026-05-11 00:12:05 +08:00
fawney19 79d07ac79a Improve request trace upstream diagnostics 2026-05-09 10:46:04 +08:00
fawney19 6dd7464793 Merge remote-tracking branch 'origin/aether-rust-pioneer' into pr-375-session-scope 2026-05-05 22:34:47 +08:00
fawney19 f959f02d40 Implement transport profile routing 2026-05-05 22:21:23 +08:00
fawney19 98421126f2 refactor: carry normalized client session affinity 2026-05-05 21:34:17 +08:00
fawney19 c130d0e2c9 refactor ai serving modules and crates 2026-05-02 13:23:54 +08:00