Skip to content

Conversation

@christian-bromann
Copy link
Member

This PR introduces modelRetryMiddleware, a new agent middleware that provides granular retry control for model calls within agents. This middleware leverages the extended RunnableRetry functionality to offer production-ready retry capabilities with configurable backoff, exception filtering, and failure handling strategies.

Motivation

Previously, applying retry logic to models used in createAgent was challenging:

  • Using .withRetry() directly on models changed their type, causing incompatibility with createAgent
  • No way to configure retry behavior at the agent middleware level
  • Limited control over which exceptions trigger retries and how failures are handled

This middleware solves these issues by providing retry logic at the middleware level, allowing fine-grained control without type incompatibilities.

Features

Retry Configuration Options

  • maxRetries: Maximum number of retry attempts after the initial call (default: 2)
  • retryOn: Filter which exceptions trigger retries (function or array of error constructors)
  • onFailure: Control failure behavior:
    • "raise" (default): Re-raise the exception
    • "return_message": Return an AIMessage with error details
    • Custom function: Format error message as needed
  • backoffFactor: Exponential backoff multiplier (default: 2.0)
  • initialDelayMs: Initial delay before first retry (default: 1000ms)
  • maxDelayMs: Maximum delay cap (default: 60000ms)
  • jitter: Enable random jitter to avoid thundering herd (default: true)

Implementation Details

  • Reuses RunnableRetry: Leverages the extended retry logic from RunnableRetry.withRetry() by applying retry options directly to the model instance (see feat(core): extended retry logic for RunnableRetry #9446)
  • Middleware Integration: Seamlessly integrates with the agent middleware system via wrapModelCall hook
  • Type Safe: Works with create_agent without type incompatibilities
  • Error Handling: Provides flexible failure handling strategies beyond simple exception raising

@changeset-bot
Copy link

changeset-bot bot commented Nov 19, 2025

⚠️ No Changeset found

Latest commit: 42ecc07

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants