Skip to content

Conversation

@christian-bromann
Copy link
Member

This PR extends the RunnableRetry class with granular retry control options, enabling fine-grained retry policies for base chat models and other runnables. This addresses the limitation where using .withRetry() on LLMs was incompatible with create_agent due to type changes, and provides production-ready retry capabilities similar to ToolRetryMiddleware.

Note: this change is fully backwards compatible and only makes additive changes.

Changes

Extended RunnableRetry Class

Added support for advanced retry configuration options:

  • retryOn: Filter which exceptions trigger retries (function or array of error constructors)
  • backoffFactor: Control exponential backoff multiplier (default: 2.0)
  • initialDelayMs: Set initial delay before first retry (default: 1000ms)
  • maxDelayMs: Cap maximum delay between retries (default: 60000ms)
  • jitter: Enable/disable random jitter to avoid thundering herd (default: true)

Usage Examples

Basic Usage (Backward Compatible)

const model = new ChatOpenAI({ model: "gpt-4o" });
const modelWithRetry = model.withRetry({ stopAfterAttempt: 3 });

Advanced Retry Options

const model = new ChatOpenAI({ model: "gpt-4o" });
const modelWithRetry = model.withRetry({
  stopAfterAttempt: 5,
  retryOn: (error) => {
    // Only retry on rate limit errors
    return error.message.includes("rate limit") || 
           error.message.includes("429");
  },
  backoffFactor: 2.0,
  initialDelayMs: 1000,
  maxDelayMs: 60000,
  jitter: true,
});

Retry Specific Error Types

class RateLimitError extends Error {}
class TimeoutError extends Error {}

const modelWithRetry = model.withRetry({
  stopAfterAttempt: 4,
  retryOn: [RateLimitError, TimeoutError],
  backoffFactor: 1.5,
});

@changeset-bot
Copy link

changeset-bot bot commented Nov 19, 2025

⚠️ No Changeset found

Latest commit: 1becced

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@christian-bromann
Copy link
Member Author

We decided within the team to not move forward adding this to RunnableRetry.

@christian-bromann christian-bromann deleted the cb/retry-extended branch November 19, 2025 21:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants