fix(agentcore): complete fix for circular import deadlock

The initial fix (f6bc8d2) was incomplete. It restored direct imports
in litellm/__init__.py but missed two critical functions that
proxy_server.py imports at module level:

1. load_credentials_from_list (proxy_server.py:56)
2. _add_custom_logger_callback_to_specific_event (proxy_server.py:464)

When these functions weren't pre-imported, Python had to dynamically
import them from utils.py, which triggered "import litellm" at
utils.py:56, recreating the circular dependency chain that prevented
Uvicorn from starting.

This complete fix adds both functions to the import list, breaking
the circular dependency cycle.

Related to PR BerriAI#17171 which introduced lazy loading on Dec 3, 2025.

Fixes: ECS health check failures where migrations complete but Uvicorn
never starts, resulting in UNHEALTHY containers.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

Author: busla

issue.md

@@ -0,0 +1,125 @@
+## Bug: PR #17171 introduces circular import deadlock that prevents proxy server from starting
+
+### Description
+
+PR #17171 ("Lazy-load utils to reduce memory + import time") introduced a circular import deadlock that prevents the FastAPI proxy server from starting. This causes health check failures in containerized deployments (ECS, Kubernetes, etc.) because Uvicorn never initializes the HTTP server.
+
+### Symptoms
+
+- ✅ Container starts successfully
+- ✅ Database migrations complete
+- ✅ APScheduler background jobs start
+- ❌ **Uvicorn HTTP server never starts** (no "Started server process" or "Uvicorn running" logs)
+- ❌ Health check endpoints (`/health`, `/health/liveness`) unreachable
+- ❌ Container marked as UNHEALTHY and terminated after repeated failures
+
+### Root Cause
+
+The lazy loading system in `litellm/__init__.py` creates a circular import deadlock:
+
+```python
+# Import chain that causes the deadlock:
+
+1. proxy_server.py:56 → from litellm.utils import load_credentials_from_list
+   ↓
+2. utils.py:56 → import litellm
+   ↓
+3. litellm/__init__.py → Sets up __getattr__ for lazy loading
+   ↓
+4. [later] Code accesses litellm.ModelResponse
+   ↓
+5. __getattr__("ModelResponse") → _lazy_import_utils("ModelResponse")
+   ↓
+6. Tries: from .utils import ModelResponse
+   ↓
+7. DEADLOCK: utils.py is still being imported from step 2!
+```
+
+**Why this hangs:** When `proxy_server.py` imports from `litellm.utils` before `litellm` finishes initializing, Python starts loading `utils.py`. When `utils.py` imports `litellm`, the module sets up lazy loading via `__getattr__`. Later, when code accesses `litellm.ModelResponse`, the `__getattr__` handler tries to import from `utils` again—but `utils.py` is still being loaded from step 2, creating an infinite wait.
+
+### Affected Files
+
+The circular dependency involves:
+- `litellm/__init__.py` (added `__getattr__` lazy loading)
+- `litellm/_lazy_imports.py` (new file, handles deferred imports)
+- `litellm/utils.py` (imports `litellm` at line 56)
+- `litellm/proxy/proxy_server.py` (imports from `litellm.utils` at line 56)
+
+### Reproduction
+
+1. Deploy litellm proxy with commit `56328e6535` or later
+2. Start the container
+3. Observe that migrations complete but Uvicorn never starts
+4. Health checks fail → container terminated
+
+OR locally:
+
+```bash
+# This will hang indefinitely with the lazy loading:
+python -m litellm.proxy.proxy_cli --port 4000
+```
+
+### Workaround/Fix
+
+The fix is to revert the lazy loading system and restore direct imports in `litellm/__init__.py`, **ensuring ALL functions imported by `proxy_server.py` are included**:
+
+```python
+# Replace lazy loading __getattr__ with direct imports:
+from .utils import (
+    client,
+    exception_type,
+    get_optional_params,
+    # ... all other utils functions
+    ModelResponse,
+    ModelResponseStream,
+    load_credentials_from_list,  # CRITICAL: Used by proxy_server.py:56
+    _add_custom_logger_callback_to_specific_event,  # CRITICAL: Used by proxy_server.py:464
+    # etc.
+)
+
+from .cost_calculator import completion_cost, cost_per_token, response_cost_calculator
+from litellm.litellm_core_utils.litellm_logging import Logging, modify_integration
+
+# Remove the __getattr__ function entirely
+```
+
+**Note**: The initial fix (commit `f6bc8d2f62`) was incomplete because it didn't include `load_credentials_from_list` and `_add_custom_logger_callback_to_specific_event` in the import list, causing the circular import to persist when `proxy_server.py` imported these functions.
+
+### Related Commits
+
+- **Breaking change:** `56328e6535` - PR #17171 (Dec 3, 2025)
+- **Incomplete fix:** `f6bc8d2f62` - "fix(agentcore): remove lazy loading to resolve circular import deadlock" (missing 2 imports)
+- **Complete fix:** TBD - Added `load_credentials_from_list` and `_add_custom_logger_callback_to_specific_event` to import list
+
+### Why the Initial Fix Failed
+
+The first fix (commit `f6bc8d2f62`) restored direct imports but was incomplete. When `proxy_server.py` imported at line 56:
+```python
+from litellm.utils import load_credentials_from_list
+```
+
+Since `load_credentials_from_list` wasn't in the pre-imported list in `litellm/__init__.py`, Python had to dynamically import it from `utils.py`, which triggered `import litellm` at `utils.py:56`, recreating the circular dependency.
+
+### Impact
+
+- **Severity:** Critical - Proxy server cannot start
+- **Affected deployments:** All containerized deployments with health checks (ECS, Kubernetes, Docker)
+- **Version:** All versions after Dec 3, 2025 (commit `56328e6535`)
+
+### Suggested Solution
+
+1. **Short-term:** Revert PR #17171 or apply the fix from commit `f6bc8d2f62`
+2. **Long-term:** If lazy loading is desired for performance, restructure the imports to break the circular dependency:
+   - Move the `import litellm` statement in `utils.py` to local imports where needed
+   - OR ensure `proxy_server.py` doesn't import from `litellm.utils` before `litellm` is fully initialized
+   - OR use a different lazy loading mechanism that doesn't rely on `__getattr__` during module initialization
+
+### Environment
+
+- Python version: 3.12.8
+- Deployment: AWS ECS (also affects any containerized deployment)
+- Base image: Chainguard Wolfi
+
+---
+
+**Note:** This issue was discovered when health checks failed in production ECS deployment. The fix has been tested and confirmed to resolve the issue.

litellm/__init__.py

@@ -1102,6 +1102,8 @@ def add_known_models():
     get_provider_fields,
     ModelResponseListIterator,
     get_valid_models,
+    load_credentials_from_list,
+    _add_custom_logger_callback_to_specific_event,
 )
 
 from .llms.bytez.chat.transformation import BytezChatConfig