feat(routing): move sticky-key retries into routing policy with lazy attempts

Replace the provider/endpoint max_retries fields as the source of same-key
retries with a routing policy setting, sticky_key_attempts (default 2). Only
the first-ranked candidate is retried on the same key; every failover
candidate gets a single attempt so failover keeps advancing instead of
retrying each fallback key.

Materialize exactly one attempt per candidate and derive same-key retries in
the attempt loop after a candidate-scoped failure, so the retry budget no
longer inflates up-front materialization and needs no upper bound. The budget
travels in the report context; retries reuse the plan with a fresh candidate
id and incremented retry index. Pool groups only retry their first key within
the retry-index stride.

Expose the setting in the routing profile editor and the set_scheduling rule
action, and drop the max_retries input from the provider form.
This commit is contained in:
elky
2026-09-02 20:48:40 +08:00
parent 415b2da81b
commit 7323d41fbe
40 changed files with 851 additions and 570 deletions
+25 -1
View File
@@ -23,7 +23,11 @@ pub struct RoutingPoolPolicyOverride {
pub scheduling_presets: Vec<RoutingSchedulingPreset>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Default)]
/// Default number of attempts on the first-ranked (sticky) candidate before
/// failing over: one retry on the same key.
pub const DEFAULT_STICKY_KEY_ATTEMPTS: u32 = 2;
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct RoutingDefaultPolicy {
#[serde(default)]
pub priority_mode: RoutingSetPriorityMode,
@@ -31,6 +35,26 @@ pub struct RoutingDefaultPolicy {
pub scheduling_mode: RoutingSchedulingMode,
#[serde(default)]
pub keep_priority_on_conversion: bool,
/// Total attempts on the first-ranked candidate before moving on. Later
/// candidates always get a single attempt so failover keeps advancing.
/// `0` and `1` both mean no same-key retry.
#[serde(default = "default_sticky_key_attempts")]
pub sticky_key_attempts: u32,
}
impl Default for RoutingDefaultPolicy {
fn default() -> Self {
Self {
priority_mode: RoutingSetPriorityMode::default(),
scheduling_mode: RoutingSchedulingMode::default(),
keep_priority_on_conversion: false,
sticky_key_attempts: DEFAULT_STICKY_KEY_ATTEMPTS,
}
}
}
fn default_sticky_key_attempts() -> u32 {
DEFAULT_STICKY_KEY_ATTEMPTS
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, Default)]