feat: Add Render Free Tier support with Lite mode - #354
Conversation
Co-authored-by: MasumRab <8943353+MasumRab@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
|
Merging to
|
Reviewer's GuideAdds a Google GenAI-based Lite mode for Render Free Tier by wiring Gemini text and embedding APIs into the RAG and Gemma client layers, updating configuration to select providers via env vars, and adjusting setup/deployment scripts and docs for lightweight, API-only deployment. Sequence diagram for GoogleGenAI-based RAG embeddingssequenceDiagram
participant AgentRAG as AgentRAG
participant AppConfig as AppConfig
participant GoogleGenAIEmbedderAdapter as GoogleGenAIEmbedderAdapter
participant GoogleGenAIAPI as GoogleGenAIAPI
AgentRAG->>AppConfig: read rag_embedding_provider
AppConfig-->>AgentRAG: rag_embedding_provider=google_genai
AgentRAG->>AgentRAG: set embedding_model (default override to text-embedding-004)
AgentRAG->>GoogleGenAIEmbedderAdapter: initialize(model_name)
GoogleGenAIEmbedderAdapter->>GoogleGenAIAPI: create genai.Client()
GoogleGenAIAPI-->>GoogleGenAIEmbedderAdapter: client instance
AgentRAG->>GoogleGenAIEmbedderAdapter: encode(texts)
GoogleGenAIEmbedderAdapter->>GoogleGenAIEmbedderAdapter: normalize texts to list
GoogleGenAIEmbedderAdapter->>GoogleGenAIAPI: models.embed_content(model, contents)
GoogleGenAIAPI-->>GoogleGenAIEmbedderAdapter: embeddings response
GoogleGenAIEmbedderAdapter->>GoogleGenAIEmbedderAdapter: convert to numpy array or list
GoogleGenAIEmbedderAdapter-->>AgentRAG: embedding vectors
Sequence diagram for Gemma client using GoogleGenAI providersequenceDiagram
actor User
participant AgentApp as AgentApp
participant AppConfig as AppConfig
participant GemmaFactory as GemmaClientFactory
participant GoogleGenAIGemmaClient as GoogleGenAIGemmaClient
participant GoogleGenAIAPI as GoogleGenAIAPI
User->>AgentApp: send chat_prompt
AgentApp->>GemmaFactory: get_gemma_client()
GemmaFactory->>AppConfig: read gemma_provider
AppConfig-->>GemmaFactory: gemma_provider=google_genai
GemmaFactory-->>AgentApp: GoogleGenAIGemmaClient instance
AgentApp->>GoogleGenAIGemmaClient: invoke(prompt)
GoogleGenAIGemmaClient->>GoogleGenAIAPI: models.generate_content(model, contents)
GoogleGenAIAPI-->>GoogleGenAIGemmaClient: response
GoogleGenAIGemmaClient-->>AgentApp: response.text
AgentApp-->>User: model reply
Class diagram for Gemma clients and GoogleGenAI embedderclassDiagram
class AppConfig {
+str rag_store
+str rag_embedding_provider
+bool dual_write
+str gemma_provider
+str gemma_model_name
+str vertex_project_id
+str vertex_location
}
class GemmaClient {
<<interface>>
+invoke(prompt, kwargs) str
}
class VertexAIGemmaClient {
+invoke(prompt, kwargs) str
}
class OllamaGemmaClient {
+base_url
+model_name
+generate_url
+timeout
+invoke(prompt, kwargs) str
}
class GoogleGenAIGemmaClient {
+client
+model_name
+GoogleGenAIGemmaClient()
+invoke(prompt, kwargs) str
}
class GemmaClientFactory {
+get_gemma_client() GemmaClient
}
class DeepSearchRAG {
-config
-embedding_provider
-embedder
-embedding_dim
+DeepSearchRAG(config, embedding_model)
}
class GoogleGenAIEmbedderAdapter {
+client
+model_name
+GoogleGenAIEmbedderAdapter(model_name)
+encode(texts)
+get_sentence_embedding_dimension() int
}
GemmaClient <|.. VertexAIGemmaClient
GemmaClient <|.. OllamaGemmaClient
GemmaClient <|.. GoogleGenAIGemmaClient
GemmaClientFactory ..> GemmaClient : create
GemmaClientFactory ..> VertexAIGemmaClient
GemmaClientFactory ..> OllamaGemmaClient
GemmaClientFactory ..> GoogleGenAIGemmaClient
DeepSearchRAG o--> GoogleGenAIEmbedderAdapter : uses_when_google_genai
DeepSearchRAG ..> AppConfig
AppConfig ..> GemmaClientFactory : configures
AppConfig ..> DeepSearchRAG : configures
Flow diagram for setup_env.sh with Render Lite modeflowchart TD
A_start["Start_setup_env.sh"] --> B_check_Node_pnpm["Check_Node_and_pnpm"]
B_check_Node_pnpm --> C_backend_setup["Enter_backend_directory"]
C_backend_setup --> D_check_uv["Check_if_uv_available"]
D_check_uv -->|uv_found| E_check_RENDER_uv["Is_RENDER_true_for_uv"]
D_check_uv -->|uv_not_found| F_check_RENDER_pip["Is_RENDER_true_for_pip"]
E_check_RENDER_uv -->|true| G_uv_render_install["uv_pip_install_editable_and_minimal_dependencies"]
E_check_RENDER_uv -->|false| H_uv_full_install["uv_sync_dev_and_install_dev_dependencies"]
G_uv_render_install --> I_uv_install_runtime["uv_pip_install_uvicorn_and_google_genai"]
H_uv_full_install --> I_uv_install_runtime
F_check_RENDER_pip -->|true| J_pip_render_install["pip_install_editable_minimal"]
F_check_RENDER_pip -->|false| K_pip_full_install["pip_install_editable_with_dev"]
J_pip_render_install --> L_pip_runtime["pip_install_uvicorn_and_google_genai"]
K_pip_full_install --> L_pip_runtime
I_uv_install_runtime --> M_playwright_decision["Is_RENDER_true_for_Playwright"]
L_pip_runtime --> M_playwright_decision
M_playwright_decision -->|true| N_skip_playwright["Skip_Playwright_browsers"]
M_playwright_decision -->|false| O_install_playwright["Install_Playwright_browsers"]
N_skip_playwright --> P_frontend_setup["Enter_frontend_directory"]
O_install_playwright --> P_frontend_setup
P_frontend_setup --> Q_pnpm_available["Check_pnpm_for_frontend"]
Q_pnpm_available -->|yes| R_pnpm_install_build["pnpm_install_and_build"]
Q_pnpm_available -->|no| S_log_error["Log_missing_pnpm_but_continue"]
R_pnpm_install_build --> T_finish_messages["Print_completion_and_run_instructions"]
S_log_error --> T_finish_messages
T_finish_messages --> U_end["End_setup_env.sh"]
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
Warning Rate limit exceeded
⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (52)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a "Lite mode" to enable cost-effective deployment of the application on Render's Free Tier. It achieves this by shifting the heavy lifting of LLM inference and RAG embeddings from local models to Google GenAI's API-based services, significantly reducing memory and CPU footprint. The changes include new client implementations, configuration updates, an optimized setup script, and a detailed deployment guide to facilitate easy setup within Render's resource constraints. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 5 issues, and left some high level feedback:
- Consider moving
GoogleGenAIEmbedderAdapterout of theDeepSearchRAG.__init__method and avoiding the hard-coded768dimension so that the adapter can be reused more easily and can correctly reflect the embedding size for non-text-embedding-004models. - In
GoogleGenAIGemmaClient.invoke, all**kwargsare currently ignored; if you expect callers to pass options like temperature or max tokens, it would be safer to either map supported parameters through togenerate_contentor validate and reject unknown options explicitly. - In
setup_env.sh, the change to not exit whenpnpmis missing means the service can deploy without a built frontend; if that’s undesirable in most environments, consider failing fast or printing a stronger warning so misconfigured deployments are more obvious.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Consider moving `GoogleGenAIEmbedderAdapter` out of the `DeepSearchRAG.__init__` method and avoiding the hard-coded `768` dimension so that the adapter can be reused more easily and can correctly reflect the embedding size for non-`text-embedding-004` models.
- In `GoogleGenAIGemmaClient.invoke`, all `**kwargs` are currently ignored; if you expect callers to pass options like temperature or max tokens, it would be safer to either map supported parameters through to `generate_content` or validate and reject unknown options explicitly.
- In `setup_env.sh`, the change to not exit when `pnpm` is missing means the service can deploy without a built frontend; if that’s undesirable in most environments, consider failing fast or printing a stronger warning so misconfigured deployments are more obvious.
## Individual Comments
### Comment 1
<location path="backend/src/agent/gemma_client.py" line_range="80" />
<code_context>
self.requests = requests
self.base_url = app_config.ollama_base_url
- self.model_name = app_config.gemma_model_name
+ self.model_name = "gemini-1.5-flash" if app_config.gemma_model_name == "gemma:7b" else app_config.gemma_model_name
self.generate_url = f"{self.base_url}/api/generate"
self.timeout = timeout
</code_context>
<issue_to_address>
**issue (bug_risk):** Using a Gemini model name when the provider is Ollama is likely to break calls to the Ollama API.
Here `OllamaGemmaClient` still targets the Ollama REST API (`/api/generate`), but `self.model_name` is remapped to `"gemini-1.5-flash"` when `gemma_model_name == "gemma:7b"`. Unless your Ollama instance defines a model with that exact name, calls will likely 404 or behave unexpectedly. If the goal is to use Gemini via Google GenAI, it’s safer to keep this client for Ollama-only models and rely on `GoogleGenAIGemmaClient` when `GEMMA_PROVIDER=google_genai`, rather than renaming the model inside the Ollama client.
</issue_to_address>
### Comment 2
<location path="backend/src/agent/rag.py" line_range="102" />
<code_context>
+ except ImportError:
+ return embeddings
+
+ def get_sentence_embedding_dimension(self):
+ # Default for text-embedding-004
+ return 768
</code_context>
<issue_to_address>
**issue (bug_risk):** Hard-coding the embedding dimension to 768 may be incorrect for non-default Google GenAI embedding models.
The adapter always returns `768` from `get_sentence_embedding_dimension`, but that’s only valid for `text-embedding-004`. If `embedding_model` is set to a different model, this will cause dimension mismatches with FAISS/Chroma indexes and stored vectors. Please either derive the dimension from model metadata (if available), or enforce `text-embedding-004` for this adapter and raise a clear error when another model is configured.
</issue_to_address>
### Comment 3
<location path="backend/src/agent/rag.py" line_range="70-73" />
<code_context>
- logger.info(f"Loading embedding model: {embedding_model}")
- self.embedder = SentenceTransformer(embedding_model)
+# Load embedding model
+ self.embedding_provider = getattr(self.config, 'rag_embedding_provider', 'sentence-transformers')
+
+ if self.embedding_provider == "google_genai":
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Unrecognized `rag_embedding_provider` values silently fall back to `sentence-transformers`, which can hide configuration errors.
Right now any value other than `"google_genai"` is treated as "use sentence-transformers", and if that library isn’t installed the user just sees a `"sentence-transformers required"` error. A typo in `RAG_EMBEDDING_PROVIDER` would therefore surface as a misleading dependency error instead of a clear config issue. Consider explicitly validating the provider against an allowed set (e.g., `{"sentence-transformers", "google_genai"}`) and raising a clear `ValueError` for unknown values before applying the fallback.
```suggestion
# Load embedding model
self.embedding_provider = getattr(self.config, 'rag_embedding_provider', 'sentence-transformers')
allowed_providers = {"sentence-transformers", "google_genai"}
if self.embedding_provider not in allowed_providers:
raise ValueError(
f"Unknown rag_embedding_provider '{self.embedding_provider}'. "
f"Expected one of: {', '.join(sorted(allowed_providers))}"
)
if self.embedding_provider == "google_genai":
```
</issue_to_address>
### Comment 4
<location path="backend/src/agent/gemma_client.py" line_range="148" />
<code_context>
+ if not os.getenv("GEMINI_API_KEY"):
+ logger.warning("GEMINI_API_KEY not found in environment. Client initialization may fail.")
+
+ def invoke(self, prompt: str, **kwargs) -> str:
+ """
+ Generate text completion using Google GenAI SDK.
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Silently ignoring `**kwargs` in the Google GenAI client can lead to surprising behavior for callers.
`invoke` accepts `**kwargs` but doesn’t pass them to `self.client.models.generate_content`, so options like `temperature` or `max_output_tokens` are silently ignored. Consider either forwarding a vetted subset of kwargs to `generate_content`, or removing `**kwargs` (or asserting it’s empty) so incorrect usage is surfaced early.
</issue_to_address>
### Comment 5
<location path="DEPLOY_RENDER.md" line_range="53" />
<code_context>
+### Optional Configuration
+If you wish to customize models further for Lite mode, you can override the defaults:
+* `GEMMA_MODEL_NAME`: Default is `gemini-1.5-flash` (when `GEMMA_PROVIDER` is `google_genai`).
+* `RAG_EMBEDDING_PROVIDER`: The model used for generating embeddings. Default is `text-embedding-004`.
</code_context>
<issue_to_address>
**issue:** Clarify whether `RAG_EMBEDDING_PROVIDER` refers to a provider or a specific model, and align the description with the earlier table.
In the earlier config table, `RAG_EMBEDDING_PROVIDER` is documented as a provider (e.g., `google_genai`), but here it’s described as a model with default `text-embedding-004`. Please clarify whether this variable is meant to store a provider name or a model name, and update both references so they describe the same thing consistently.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| self.requests = requests | ||
| self.base_url = app_config.ollama_base_url | ||
| self.model_name = app_config.gemma_model_name | ||
| self.model_name = "gemini-1.5-flash" if app_config.gemma_model_name == "gemma:7b" else app_config.gemma_model_name |
There was a problem hiding this comment.
issue (bug_risk): Using a Gemini model name when the provider is Ollama is likely to break calls to the Ollama API.
Here OllamaGemmaClient still targets the Ollama REST API (/api/generate), but self.model_name is remapped to "gemini-1.5-flash" when gemma_model_name == "gemma:7b". Unless your Ollama instance defines a model with that exact name, calls will likely 404 or behave unexpectedly. If the goal is to use Gemini via Google GenAI, it’s safer to keep this client for Ollama-only models and rely on GoogleGenAIGemmaClient when GEMMA_PROVIDER=google_genai, rather than renaming the model inside the Ollama client.
| except ImportError: | ||
| return embeddings | ||
|
|
||
| def get_sentence_embedding_dimension(self): |
There was a problem hiding this comment.
issue (bug_risk): Hard-coding the embedding dimension to 768 may be incorrect for non-default Google GenAI embedding models.
The adapter always returns 768 from get_sentence_embedding_dimension, but that’s only valid for text-embedding-004. If embedding_model is set to a different model, this will cause dimension mismatches with FAISS/Chroma indexes and stored vectors. Please either derive the dimension from model metadata (if available), or enforce text-embedding-004 for this adapter and raise a clear error when another model is configured.
| # Load embedding model | ||
| self.embedding_provider = getattr(self.config, 'rag_embedding_provider', 'sentence-transformers') | ||
|
|
||
| if self.embedding_provider == "google_genai": |
There was a problem hiding this comment.
suggestion (bug_risk): Unrecognized rag_embedding_provider values silently fall back to sentence-transformers, which can hide configuration errors.
Right now any value other than "google_genai" is treated as "use sentence-transformers", and if that library isn’t installed the user just sees a "sentence-transformers required" error. A typo in RAG_EMBEDDING_PROVIDER would therefore surface as a misleading dependency error instead of a clear config issue. Consider explicitly validating the provider against an allowed set (e.g., {"sentence-transformers", "google_genai"}) and raising a clear ValueError for unknown values before applying the fallback.
| # Load embedding model | |
| self.embedding_provider = getattr(self.config, 'rag_embedding_provider', 'sentence-transformers') | |
| if self.embedding_provider == "google_genai": | |
| # Load embedding model | |
| self.embedding_provider = getattr(self.config, 'rag_embedding_provider', 'sentence-transformers') | |
| allowed_providers = {"sentence-transformers", "google_genai"} | |
| if self.embedding_provider not in allowed_providers: | |
| raise ValueError( | |
| f"Unknown rag_embedding_provider '{self.embedding_provider}'. " | |
| f"Expected one of: {', '.join(sorted(allowed_providers))}" | |
| ) | |
| if self.embedding_provider == "google_genai": |
| if not os.getenv("GEMINI_API_KEY"): | ||
| logger.warning("GEMINI_API_KEY not found in environment. Client initialization may fail.") | ||
|
|
||
| def invoke(self, prompt: str, **kwargs) -> str: |
There was a problem hiding this comment.
suggestion (bug_risk): Silently ignoring **kwargs in the Google GenAI client can lead to surprising behavior for callers.
invoke accepts **kwargs but doesn’t pass them to self.client.models.generate_content, so options like temperature or max_output_tokens are silently ignored. Consider either forwarding a vetted subset of kwargs to generate_content, or removing **kwargs (or asserting it’s empty) so incorrect usage is surfaced early.
| ### Optional Configuration | ||
| If you wish to customize models further for Lite mode, you can override the defaults: | ||
| * `GEMMA_MODEL_NAME`: Default is `gemini-1.5-flash` (when `GEMMA_PROVIDER` is `google_genai`). | ||
| * `RAG_EMBEDDING_PROVIDER`: The model used for generating embeddings. Default is `text-embedding-004`. |
There was a problem hiding this comment.
issue: Clarify whether RAG_EMBEDDING_PROVIDER refers to a provider or a specific model, and align the description with the earlier table.
In the earlier config table, RAG_EMBEDDING_PROVIDER is documented as a provider (e.g., google_genai), but here it’s described as a model with default text-embedding-004. Please clarify whether this variable is meant to store a provider name or a model name, and update both references so they describe the same thing consistently.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds support for a 'Lite mode' deployment on Render's Free Tier by integrating with the google-genai APIs. The changes are generally well-structured, including a new deployment guide and updates to the setup script. However, I've identified a few critical issues in the new client implementations that will cause runtime failures and break existing functionality for the Ollama provider. Additionally, there are opportunities to simplify the setup script and improve the accuracy of the new documentation. My review includes code suggestions to address these critical bugs and other improvements.
| self.requests = requests | ||
| self.base_url = app_config.ollama_base_url | ||
| self.model_name = app_config.gemma_model_name | ||
| self.model_name = "gemini-1.5-flash" if app_config.gemma_model_name == "gemma:7b" else app_config.gemma_model_name |
There was a problem hiding this comment.
This change incorrectly forces the OllamaGemmaClient to use a Gemini model name (gemini-1.5-flash) if the configured model is the default gemma:7b. The Ollama client should use the model name as configured for Ollama, as it does not serve Gemini models by default. This change will break existing Ollama setups and should be reverted.
| self.model_name = "gemini-1.5-flash" if app_config.gemma_model_name == "gemma:7b" else app_config.gemma_model_name | |
| self.model_name = app_config.gemma_model_name |
| class GoogleGenAIGemmaClient(GemmaClient): | ||
| """Client for Gemma models using the Google GenAI SDK (requires GEMINI_API_KEY).""" | ||
|
|
||
| def __init__(self): | ||
| """ | ||
| Initialize Google GenAI client. | ||
| """ | ||
| try: | ||
| from google import genai | ||
| import os | ||
| except ImportError: | ||
| logger.error("google-genai not installed.") | ||
| raise ImportError("Please install 'google-genai' to use GoogleGenAIGemmaClient") | ||
|
|
||
| # Will automatically pick up GEMINI_API_KEY from environment | ||
| self.client = genai.Client() | ||
| self.model_name = "gemini-1.5-flash" if app_config.gemma_model_name == "gemma:7b" else app_config.gemma_model_name | ||
|
|
||
| # Basic validation that we have an API key | ||
| if not os.getenv("GEMINI_API_KEY"): | ||
| logger.warning("GEMINI_API_KEY not found in environment. Client initialization may fail.") | ||
|
|
||
| def invoke(self, prompt: str, **kwargs) -> str: | ||
| """ | ||
| Generate text completion using Google GenAI SDK. | ||
| """ | ||
| try: | ||
| # We don't want to pass unrecognized kwargs to genai, so we filter out or ignore them | ||
| # For this simple implementation, we just pass the prompt | ||
| response = self.client.models.generate_content( | ||
| model=self.model_name, | ||
| contents=prompt | ||
| ) | ||
| return response.text | ||
| except Exception as e: | ||
| logger.error(f"Google GenAI request failed: {e}") | ||
| raise e |
There was a problem hiding this comment.
The implementation of GoogleGenAIGemmaClient has a few issues that will lead to runtime errors:
- Incorrect API Usage: The call
self.client.models.generate_contentis not a valid method on theModelServiceClient. This will raise anAttributeError. - Incomplete API Key Check: The code only checks for the
GEMINI_API_KEYenvironment variable, but thegoogle-genailibrary also supportsGOOGLE_API_KEY.
I suggest refactoring the client to use the recommended high-level genai.GenerativeModel API, which is simpler and more robust. The proposed suggestion fixes these issues.
class GoogleGenAIGemmaClient(GemmaClient):
"""Client for Gemma models using the Google GenAI SDK (requires GEMINI_API_KEY or GOOGLE_API_KEY)."""
def __init__(self):
"""
Initialize Google GenAI client.
"""
try:
from google import genai
import os
except ImportError:
logger.error("google-genai not installed.")
raise ImportError("Please install 'google-genai' to use GoogleGenAIGemmaClient")
api_key = os.getenv("GEMINI_API_KEY") or os.getenv("GOOGLE_API_KEY")
if not api_key:
logger.warning("GEMINI_API_KEY or GOOGLE_API_KEY not found in environment. Client initialization may fail.")
genai.configure(api_key=api_key)
self.model_name = "gemini-1.5-flash" if app_config.gemma_model_name == "gemma:7b" else app_config.gemma_model_name
self.model = genai.GenerativeModel(self.model_name)
def invoke(self, prompt: str, **kwargs) -> str:
"""
Generate text completion using Google GenAI SDK.
"""
try:
# The high-level API doesn't accept arbitrary kwargs, so we ignore them.
response = self.model.generate_content(contents=prompt)
return response.text
except Exception as e:
logger.error(f"Google GenAI request failed: {e}")
raise e| else | ||
| echo "ERROR: pnpm not found even after installation attempt." | ||
| exit 1 | ||
| # do not exit, just error log |
There was a problem hiding this comment.
Removing exit 1 when pnpm is not found can lead to silent build failures in automated environments like Render. If pnpm is missing, the frontend build will fail, but the script will continue and exit successfully. This will result in a broken deployment where the backend is running but the frontend is missing. It's safer to exit with an error to ensure the build process fails explicitly.
| # do not exit, just error log | |
| exit 1 |
| * **Build Command:** `./setup_env.sh` | ||
| * **Start Command:** | ||
| ```bash | ||
| cd backend && export PYTHONPATH=$PYTHONPATH:$(pwd)/src && python -m uvicorn agent.app:app --host 0.0.0.0 --port $PORT |
There was a problem hiding this comment.
The export PYTHONPATH... part of the start command is redundant. The backend package is installed in editable mode (-e .) by setup_env.sh, which correctly adds the src directory to Python's path. The command can be simplified for clarity and to rely on standard package installation behavior.
| cd backend && export PYTHONPATH=$PYTHONPATH:$(pwd)/src && python -m uvicorn agent.app:app --host 0.0.0.0 --port $PORT | |
| cd backend && python -m uvicorn agent.app:app --host 0.0.0.0 --port $PORT |
| ### Optional Configuration | ||
| If you wish to customize models further for Lite mode, you can override the defaults: | ||
| * `GEMMA_MODEL_NAME`: Default is `gemini-1.5-flash` (when `GEMMA_PROVIDER` is `google_genai`). | ||
| * `RAG_EMBEDDING_PROVIDER`: The model used for generating embeddings. Default is `text-embedding-004`. |
There was a problem hiding this comment.
This documentation line is misleading. The key is incorrectly listed as RAG_EMBEDDING_PROVIDER, which controls the provider, not the model. More importantly, the codebase does not currently allow overriding the embedding model for the google_genai provider. The documentation should be updated to reflect that text-embedding-004 is the hardcoded default and cannot be changed via environment variables.
| * `RAG_EMBEDDING_PROVIDER`: The model used for generating embeddings. Default is `text-embedding-004`. | |
| * **RAG Embedding Model**: When `RAG_EMBEDDING_PROVIDER` is `google_genai`, the model defaults to `text-embedding-004` and is not currently configurable. |
| embedding_model = "text-embedding-004" if embedding_model == "sentence-transformers/all-MiniLM-L6-v2" else embedding_model | ||
| logger.info(f"Loading Google GenAI embedding model: {embedding_model}") | ||
|
|
||
| class GoogleGenAIEmbedderAdapter: |
There was a problem hiding this comment.
Defining the GoogleGenAIEmbedderAdapter class inside the __init__ method is not ideal for maintainability or performance. It causes the class to be redefined every time a DeepSearchRAG instance is created. Please move this class definition to the module level for better code structure and to avoid this re-creation.
|
|
||
| def get_sentence_embedding_dimension(self): | ||
| # Default for text-embedding-004 | ||
| return 768 |
There was a problem hiding this comment.
The embedding dimension 768 is a magic number. It's specific to the text-embedding-004 model. This should be defined as a named constant to improve readability and make it easier to update if the model changes in the future.
| return 768 | |
| # Default for text-embedding-004 | |
| TEXT_EMBEDDING_004_DIMENSION = 768 | |
| return TEXT_EMBEDDING_004_DIMENSION |
| if command -v uv &> /dev/null; then | ||
| echo "Using uv for dependency management..." | ||
| # Sync dependencies including dev (which now has playwright) | ||
| uv sync --dev || uv pip install -e ".[dev]" | ||
|
|
||
| # Also install the package itself in editable mode if not covered by sync | ||
| uv pip install -e . | ||
| # If Render Free Tier, we skip dev dependencies to save space and time | ||
| if [ "$RENDER" = "true" ]; then | ||
| echo "Render environment detected. Installing production dependencies..." | ||
| uv pip install -e . | ||
| uv pip install uvicorn google-genai | ||
| else | ||
| uv sync --dev || uv pip install -e ".[dev]" | ||
| uv pip install -e . | ||
| uv pip install uvicorn google-genai | ||
| fi | ||
| else | ||
| echo "uv not found. Falling back to pip..." | ||
| echo "Recommendation: Install uv for faster, more reliable setups." | ||
| pip install -e ".[dev]" | ||
| # Ensure uvicorn is installed for production run | ||
| pip install uvicorn | ||
| if [ "$RENDER" = "true" ]; then | ||
| pip install -e . | ||
| else | ||
| pip install -e ".[dev]" | ||
| fi | ||
| # Ensure uvicorn and google-genai are installed | ||
| pip install uvicorn google-genai | ||
| fi |
There was a problem hiding this comment.
There are several redundant dependency installation commands in this block.
- In the
uvelseblock (lines 49-51),uv pip install -e .anduv pip install uvicorn google-genaiare redundant becauseuv sync --devalready installs the project and all its dependencies. - In the
pipblock (line 61),pip install uvicorn google-genaiis redundant becausepip install -e .orpip install -e ".[dev]"will install these packages as they are listed inpyproject.toml.
Removing these redundant commands will simplify the script and rely on the package manager's dependency resolution.
Co-authored-by: MasumRab <8943353+MasumRab@users.noreply.github.com>
|
|
@jules Resolve conflicts: Avoid full repo diff - focus only on your changed paths. |
1 participant
google-genaifor API-based model inference.GoogleGenAIEmbedderAdapterinDeepSearchRAG.gemini-1.5-flash,text-embedding-004) when usinggoogle_genaiprovider.setup_env.shto install minimal Python dependencies for Free Tier whenRENDER=true, bypassing Playwright browsers.DEPLOY_RENDER.mdprioritizing manual web service over Blueprint to avoid payments.PR created automatically by Jules for task 6186272092202799273 started by @MasumRab
Summary by Sourcery
Add Google GenAI-based Lite mode and Render Free Tier deployment support by switching RAG embeddings and Gemma inference to hosted APIs and optimizing setup for low-resource environments.
New Features:
Enhancements:
Documentation: