[BUG] RubyLLM not retrying POST requests

[AlexVPopov]

Basic checks

• I searched existing issues - this hasn't been reported
• I can reproduce this consistently
• This is a RubyLLM bug, not my application code

What's broken?

Hey, folks,

I think I've discovered a bug, unless, of course, I'm not missing something. I think RubyLLM won't retry POST requets, because it doesn't set the methods option on faraday-retry and also doesn't provide a way to set it.

• The faraday-retry middleware accepts a methods option and falls back to %i[delete get head options put], if no methods are given.
• RubyLLM doesn't set the methods option, hence only idempotent methods are retried, post requests are never retried.

I'm hitting this on Anthropic's POST v1/messages endpoint. Tagging @tpaulshippy here as well, as I know he uses Anthropic heavily as well.

How to reproduce

1. RubyLLM has a default config for automatic retries, so need for additional configuration regarding retries. Set Anthropic as provider or make sure that the ask method will issue a POST request.

2. Stub an error response and see, that requests are not retried. Something like:

    context "when Anthropic API returns an error" do
      before do
        # Stub the Anthropic Messages API endpoint with proper error format
        stub_request(:post, "https://api.anthropic.com/v1/messages")
          .to_return(
            status: 500,
            body: {
              type: "error",
              error: {
                type: "api_error",
                message: "An unexpected error has occurred internal to Anthropic's systems."
              },
              request_id: "req_018EeWyXxfu5pfWkrYcMdjWG"
            }.to_json,
            headers: {
              "Content-Type" => "application/json",
              "request-id" => "req_018EeWyXxfu5pfWkrYcMdjWG"
            }
          )
      end

      it "retries the request" do
        RubyLLM.ask "Something"

        expect(WebMock).to have_requested(:post, "https://api.anthropic.com/v1/messages").times(4) # 1 call + 3 (default) retries, this should fail
      end
    end

Failure/Error: expect(WebMock).to have_requested(:post, "https://api.anthropic.com/v1/messages").times(4)
     
       The request POST https://api.anthropic.com/v1/messages was expected to execute 4 times but it executed 1 time

Expected behavior

POST requests should be retried.

What actually happened

POST requests are not retried.

Environment

• RubyLLM 1.6

[AlexVPopov]

Easiest fix would be to hardcode the methods in setup_retries:

    def setup_retry(faraday)
      faraday.request :retry, {
        max: @config.max_retries,
        interval: @config.retry_interval,
        interval_randomness: @config.retry_interval_randomness,
        backoff_factor: @config.retry_backoff_factor,
        exceptions: retry_exceptions,
        retry_statuses: [429, 500, 502, 503, 504, 529],
        methods: %i[delete get head options post put]
      }
    end

Alternatively, this can be exposed as a configuration option.

[crmne]

You are right! Didn't catch that one. I quickly tried to add post (and patch) and I was having trouble with certain requests taking forever. Probably connected to Faraday 1, but not sure. Will be able to take a look next week.

Feel free to take a look in the meantime and I'm thinking whether we should disable automatic retries. What do you guys think?

[AlexVPopov]

Hey @crmne

Frankly, I think automatic retries are nice, I wouldn't want them disabled, just working for all methods.

Would you accept a PR with hardcoded values for methods (I'll add patch as well), so something like:

    def setup_retry(faraday)
      faraday.request :retry, {
        max: @config.max_retries,
        interval: @config.retry_interval,
        interval_randomness: @config.retry_interval_randomness,
        backoff_factor: @config.retry_backoff_factor,
        exceptions: retry_exceptions,
-       retry_statuses: [429, 500, 502, 503, 504, 529]
+       retry_statuses: [429, 500, 502, 503, 504, 529],
+       methods: %i[delete get head options patch post put]
      }
    end

[crmne]

I would but as I said I tried it already and the tests wouldn't finish. They get stuck at some requests.