Model Failover

When an LLM call fails with a transient error (currently: completion timeout), the runtime can fail over to the next candidate model in the chain — same conversation, same turn, no repeated tool calls.

How It Works

1. Candidate chain — model resolution produces an ordered list of fallback models alongside the primary pick.
2. Transient failure — the error classifier decides whether a failure is transient (advance to next model) or permanent (abort).
3. In-place switch — the runtime swaps the model, rebuilds options, and retries the same turn. The conversation history is unchanged.
4. Observability — every attempt and every switch is observable through observers, streaming events, and bus events.

Opt-In

Failover is disabled by default. Enable it globally or per-request:

// Global: in DI setup
services.AddAiClevernessRuntime(options =>
{
    options.EnableModelFailover = true;
});

// Per-request: override in parameters
var request = new AgentRequest(
    Goal: "Research pricing for provider X",
    Parameters: new Dictionary<string, object>
    {
        ["enable_model_failover"] = true,
        ["model_fallback_chain"] = new[] { "gpt-4o", "claude-3.5-sonnet" }
    });

Candidate Chain Sources

The chain is resolved in this priority order:

Source	When used
model_fallback_chain request parameter	Explicit list of model names — takes precedence
ModelResolutionResult.Fallbacks	Built automatically by capability resolution
Empty (no failover)	No chain available or failover disabled

When using an explicit chain and a model catalog is registered, model names are validated against the catalog. Unknown names are skipped with a warning log. Without a catalog, names are passed through as-is. In both cases the chain is normalized: the active model is excluded and duplicates are removed, so failover never retries the current model.

Pinned Models

If you set a specific model via AgentPropertyKeys.Model without providing a fallback chain, the model is considered pinned. Failover is disabled regardless of the enable flag. Pinned means pinned.

Turn Budget

A failed attempt does not count against maxTurns. The turn counter is rewound on failover so consumers see the same budget semantics as before — the retry reuses the same logical turn in the loop counter, the execution state, and the event stream (no second TurnStartedEvent for that turn).

Chain Exhaustion

When all candidates in the chain have been tried and failed, the run fails with an error message:

LLM failover chain exhausted after 3 attempts; last model tried: 'model-c' on turn 0

This is identifiable programmatically via FailureEvent.Phase == "ModelFailover".

Event Sequence

For a single failover on turn N, consumers observe this exact sequence:

1. TurnStartedEvent          { Turn = N }
2. OnLlmCalledAsync          (messages sent to model A)
3. [timeout / error]
4. OnLlmCallCompletedAsync   { Model = A, Success = false, Classification = TransientAdvance }
5. FailureEvent              { Phase = "LlmCompletion", IsTransient = true }
6. OnModelSwitchedAsync      { from = A, to = B, reason = "..." }
7. ModelSwitchedAgentEvent   { From = A, To = B }
8. ModelSwitchedBusEvent     { From = A, To = B }
9. OnLlmCalledAsync          (same messages sent to model B)
10. [success]
11. OnLlmCallCompletedAsync  { Model = B, Success = true }

The retried attempt reuses the logical turn — there is no second TurnStartedEvent and no extra turn counted in the execution state.

Observability Hooks

Channel	Event	When
Observer	OnLlmCallCompletedAsync(LlmCallInfo)	Every attempt (success, error, timeout)
Observer	OnModelSwitchedAsync(from, to, reason)	Every model switch
Streaming	ModelSwitchedAgentEvent	Every model switch
Bus	ModelSwitchedBusEvent	Every model switch

Example: Capability-Routed Failover

With two capability profiles registered (a fast text model and a slower fallback), resolution automatically builds the chain:

// Profile 1: fast model (priority 1)
// Profile 2: slower model (priority 2)
// Resolution picks the fast model as primary, slower as fallback.

services.AddAiClevernessRuntime(options =>
{
    options.EnableModelFailover = true;
    options.DefaultCompletionTimeoutSeconds = 10;
});

// When the fast model times out, the runtime switches to the slower model
// transparently — the consumer sees a single successful result.

Error Classification

The DefaultLlmErrorClassifier classifies failures:

Failure	Classification	Action
Per-turn timeout (OperationCanceledException, caller token alive)	TransientAdvance	Advance to next candidate
User cancellation	Permanent	Abort
Any other exception	Permanent	Abort

Rate-limit and unavailable-model signals can be added to the classifier without touching the tool loop.

Interaction with Quality Gates

Quality gates operate above the tool loop. If a quality gate rejects the result and triggers a retry, the entire loop restarts with a fresh turn counter. The candidate chain is re-resolved from the current context state. ModelExecutionInfo.Attempt is preserved across quality retries.