Skip to content

Commit f48aa0f

Browse files
committed
update tests etc.
1 parent f3c8139 commit f48aa0f

File tree

5 files changed

+651
-633
lines changed

5 files changed

+651
-633
lines changed

README.md

Lines changed: 68 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,13 +69,14 @@ For complete code examples, see the following notebooks:
6969
| Basic grid study | [00_grid_study.ipynb](https://github.com/redis-applied-ai/redis-retrieval-optimizer/blob/main/docs/examples/grid_study/00_grid_study.ipynb) |
7070
| Custom grid study | [01_custom_grid_study.ipynb](https://github.com/redis-applied-ai/redis-retrieval-optimizer/blob/main/docs/examples/grid_study/01_custom_grid_study.ipynb) |
7171
| Bayesian Optimization | [00_bayes_study.ipynb](https://github.com/redis-applied-ai/redis-retrieval-optimizer/blob/main/docs/examples/bayesian_optimization/00_bayes_study.ipynb) |
72+
| Search study | [00_search_study.ipynb](https://github.com/redis-applied-ai/redis-retrieval-optimizer/blob/main/docs/examples/search_study/00_search_study.ipynb) |
7273
| Embedding model comparison | [00_comparison.ipynb](https://github.com/redis-applied-ai/redis-retrieval-optimizer/blob/main/docs/examples/comparison/00_comparison.ipynb) |
7374

7475
---
7576

7677
## 🚀 Quick Start
7778

78-
The Retrieval Optimizer supports two *study* types: **Grid** and **Bayesian Optimization**. Each is suited to a different stage of building a high-quality search system.
79+
The Retrieval Optimizer supports three *study* types: **Grid**, **Bayesian Optimization**, and **Search Study**. Each is suited to a different stage of building a high-quality search system.
7980

8081
### Grid
8182

@@ -85,6 +86,10 @@ Use a grid study to explore the impact of different **embedding models** and **r
8586

8687
Once you've identified a solid starting point, use Bayesian optimization to **fine-tune your index configuration**. This mode intelligently selects the most promising combinations to test, in place of exhaustive testing (which is time-consuming). Bayesian optimization mode is especially useful for balancing **cost, speed, and latency** as you work toward a production-ready solution.
8788

89+
### Search Study
90+
91+
Use a search study when you have an **existing Redis index** and want to quickly test different search methods against it without recreating the index or data. This is ideal for A/B testing search strategies or evaluating custom search methods on production data.
92+
8893
## Running a Grid study
8994

9095
#### Study config
@@ -242,6 +247,68 @@ metrics = run_bayes_study(
242247

243248
---
244249

250+
## Running a Search study
251+
252+
Use a search study when you have an **existing Redis index** and want to quickly test different search methods against it without recreating the index or data. This is ideal for A/B testing search strategies or evaluating custom search methods on production data.
253+
254+
### Search study config
255+
```yaml
256+
embedding_model:
257+
dim: 768
258+
dtype: float32
259+
embedding_cache_name: vec-cache
260+
model: sentence-transformers/all-mpnet-base-v2
261+
type: hf
262+
index_name: cars
263+
queries: "../resources/cars/car_queries.json"
264+
qrels: "../resources/cars/car_qrels.json"
265+
ret_k: 6
266+
search_methods:
267+
- base_vector
268+
- pre_filter_vector
269+
study_id: test-search-study
270+
```
271+
272+
### Code
273+
```python
274+
import os
275+
from redis_retrieval_optimizer.search_study import run_search_study
276+
from dotenv import load_dotenv
277+
278+
# load environment variables containing necessary credentials
279+
load_dotenv()
280+
281+
redis_url = os.environ.get("REDIS_URL", "redis://localhost:6379/0")
282+
283+
# Define custom search methods
284+
def gather_vector_results(search_method_input):
285+
# Your vector search implementation
286+
pass
287+
288+
def gather_pre_filter_results(search_method_input):
289+
# Your pre-filtered search implementation
290+
pass
291+
292+
CUSTOM_SEARCH_METHOD_MAP = {
293+
"base_vector": gather_vector_results,
294+
"pre_filter_vector": gather_pre_filter_results
295+
}
296+
297+
metrics = run_search_study(
298+
config_path="search_study_config.yaml",
299+
redis_url=redis_url,
300+
search_method_map=CUSTOM_SEARCH_METHOD_MAP
301+
)
302+
```
303+
304+
### Example output
305+
| search_method | avg_query_time | recall | precision | ndcg |
306+
|-------------------|----------------|--------|-----------|---------|
307+
| base_vector | 0.002605 | 0.9 | 0.23 | 0.717676|
308+
| pre_filter_vector | 0.001177 | 1.0 | 0.25 | 0.914903|
309+
310+
---
311+
245312
## 🔍 Search Methods
246313

247314
Below is a comprehensive table documenting the built-in search methods available in the Retrieval Optimizer:

0 commit comments

Comments
 (0)