[Refactor#2] litellm/init – Lazy-load utils to reduce memory + import time #17171
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Lazy-load most functions and response types from utils.py to avoid loading tiktoken and other heavy dependencies at import time. This significantly reduces memory usage when importing completion from litellm. Changes: - Made utils functions (exception_type, get_litellm_params, ModelResponse, etc.) lazy-loaded via __getattr__ - Made ALL_LITELLM_RESPONSE_TYPES lazy-loaded - Fixed circular imports by updating files to import directly from litellm.utils or litellm.types.utils instead of from litellm - Kept client decorator as immediate import since it's used at function definition time Only client is now imported immediately from utils.py; all other utils functions and response types are loaded on-demand when accessed.
- Move _lazy_import_cost_calculator and _lazy_import_litellm_logging to _lazy_imports.py - Update functions to use _get_litellm_globals() for consistency - Remove top-level imports and registry, import lazily in __getattr__ instead - Ensures _lazy_imports.py is only loaded when needed, maintaining true lazy loading
…tions - Move get_model_info import from module level to method level in bedrock image transformations - Add shared get_cached_model_info() function in common_utils.py to avoid code duplication - Cache the import after first use to maintain performance - Fixes circular import error during litellm module initialization
- Add Type to typing imports to fix F821 undefined name error - Fix function call from get_rag_ingestion_class to get_ingestion_class
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…nation - Remove unused 'litellm' import from amazon_nova_transformation.py - Remove unused imports (Dict, AsyncHTTPHandler, get_async_httpx_client, httpxSpecialProvider) from ingestion.py - Add noqa comment with explanation for PLR0915 in _lazy_imports.py to preserve true lazy import behavior
busla
added a commit
to aproorg/litellm
that referenced
this pull request
Dec 4, 2025
The initial fix (f6bc8d2) was incomplete. It restored direct imports in litellm/__init__.py but missed two critical functions that proxy_server.py imports at module level: 1. load_credentials_from_list (proxy_server.py:56) 2. _add_custom_logger_callback_to_specific_event (proxy_server.py:464) When these functions weren't pre-imported, Python had to dynamically import them from utils.py, which triggered "import litellm" at utils.py:56, recreating the circular dependency chain that prevented Uvicorn from starting. This complete fix adds both functions to the import list, breaking the circular dependency cycle. Related to PR BerriAI#17171 which introduced lazy loading on Dec 3, 2025. Fixes: ECS health check failures where migrations complete but Uvicorn never starts, resulting in UNHEALTHY containers. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Title
[Refactor#2] litellm/init – Lazy-load utils to reduce memory + import time
Relevant issues
LiteLLM import time & memory consumption on import.
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unitType
🐛 Bug Fix
Changes