fix: fail close provider-scoped routes when no provider endpoints match#158
Draft
petersimmons1972 wants to merge 26 commits into
Draft
fix: fail close provider-scoped routes when no provider endpoints match#158petersimmons1972 wants to merge 26 commits into
petersimmons1972 wants to merge 26 commits into
Conversation
… (#1) Adds discovery.type: "fc" — Olla polls Flight Controller /registry every 15s and atomically reconciles its backend set, meeting the <30s convergence SLO. - internal/adapter/discovery/fc_repository.go: FCDiscoveryPoller with Poll() and RunLoop(). Converts FC RegistryEntry{Host,Models[]{Name,Port}} into EndpointConfig slices and calls LoadFromConfig (thread-safe atomic map swap already in StaticEndpointRepository). Fail-open: FC unreachable → warn and preserve existing endpoint set. - internal/adapter/discovery/fc_repository_test.go: 3 tests — convert, remove-stale (core acceptance criterion), fail-open on FC unavailability - internal/config/types.go: FCDiscoveryConfig{RegistryURL, PollInterval}; DiscoveryConfig.FC added - internal/app/services/discovery.go: "fc" switch case validates registry_url, defaults poll_interval to 15s, runs initial blocking poll at startup, launches background RunLoop goroutine; fcPollerCancel called on Stop() ADV.1 advisory recommendation: Path A chosen over Path C (Reconciler) because Olla has no hot-reload path — ConfigMap volume mount convergence is 30-60s+ making <30s SLO unreliable without an Olla fork change anyway. Co-authored-by: Peter Simmons <petersimmons1972@gmail.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
…aker-config fix(olla): honor endpoint circuit breaker config
…fig-followup fix(olla): finish breaker config rollout for oblivion-gb10-vllm
…tus-main feat(olla): expose breaker state in endpoint status
…dels-main feat(olla): enforce allowed model aliases
Co-authored-by: Peter Simmons <petersimmons1972@gmail.com>
Co-authored-by: Peter Simmons <petersimmons1972@gmail.com>
Co-authored-by: Peter Simmons <petersimmons1972@gmail.com>
… (#1) Adds discovery.type: "fc" — Olla polls Flight Controller /registry every 15s and atomically reconciles its backend set, meeting the <30s convergence SLO. - internal/adapter/discovery/fc_repository.go: FCDiscoveryPoller with Poll() and RunLoop(). Converts FC RegistryEntry{Host,Models[]{Name,Port}} into EndpointConfig slices and calls LoadFromConfig (thread-safe atomic map swap already in StaticEndpointRepository). Fail-open: FC unreachable → warn and preserve existing endpoint set. - internal/adapter/discovery/fc_repository_test.go: 3 tests — convert, remove-stale (core acceptance criterion), fail-open on FC unavailability - internal/config/types.go: FCDiscoveryConfig{RegistryURL, PollInterval}; DiscoveryConfig.FC added - internal/app/services/discovery.go: "fc" switch case validates registry_url, defaults poll_interval to 15s, runs initial blocking poll at startup, launches background RunLoop goroutine; fcPollerCancel called on Stop() ADV.1 advisory recommendation: Path A chosen over Path C (Reconciler) because Olla has no hot-reload path — ConfigMap volume mount convergence is 30-60s+ making <30s SLO unreliable without an Olla fork change anyway. Co-authored-by: Peter Simmons <petersimmons1972@gmail.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Peter Simmons <petersimmons1972@gmail.com>
Co-authored-by: Peter Simmons <petersimmons1972@gmail.com>
Co-authored-by: Peter Simmons <petersimmons1972@gmail.com>
…n-aifleet Rebase AI Fleet Olla changes onto upstream main
Fix model config and discovery defaults
…var regression Closes #79
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary\n- Provider-scoped route selection now uses strict compatibility filtering so requests do not fall back to unrelated endpoint types.\n- Added regression test for healthy non-matching endpoints returning provider-specific not-found.\n\n## Changes\n- internal/app/handlers/handler_provider_common.go: added and switched provider endpoint resolution to it.\n- internal/app/handlers/handler_provider_test.go: added .\n\n## Validation\n- go test ./internal/app/handlers\n\nCloses #84