feat: add AI Libraries page to website - #602
Conversation
Implement comprehensive AI Libraries page showing AI/ML libraries that support Valkey for caching, vector storage, and memory management. Changes: - Update build/init-topics-and-clients.sh to accept AI path as 3rd parameter - Add ai_json_path configuration to config.toml - Create content/ai/_index.md with 14 AI libraries listed - Create templates/ai-list.html for AI libraries display with proper whitespace control to prevent excessive blank lines in output - Update ai-list.html to display Description and Valkey Uses fields between Repo and Installation for better context - Add ai_json_path() macro to templates/macros/docs.html - Update templates/default.html to add AI Libraries to navigation menu - Update .github/workflows/zola-deploy.yml to include AI libraries in build - Add LOCAL_TESTING.md documentation for local development - Add build-ai to .gitignore The page displays AI libraries organized by language (Python, JavaScript, Java, Lua) with details including: - Repository link - Description (what the library does and why to use it) - Valkey Uses (how Valkey is used: vector store, caching, memory, etc.) - Installation instructions - Version and release date - License Successfully tested locally with Zola build - 257 pages generated including the new AI libraries page at /ai/ with clean HTML output and all new fields displaying correctly. Signed-off-by: Joseph Brinkman <joe.brinkman@improving.com>
📝 WalkthroughWalkthroughChangesAdds an AI Libraries documentation page that loads library metadata from AI Libraries documentation
Assessment against linked issues
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 Checkov (3.3.8).github/workflows/zola-deploy.ymlTraceback (most recent call last): Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Depends on valkey-io/valkey-doc#459 |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/zola-deploy.yml:
- Around line 60-62: Pin the valkey-doc checkout used by the CI workflow and the
clone command in LOCAL_TESTING.md to the same explicit branch, tag, or commit.
Update .github/workflows/zola-deploy.yml around build/init-topics-and-clients.sh
and LOCAL_TESTING.md at the valkey-doc clone so both environments consume the
intended revision consistently.
In `@build/init-topics-and-clients.sh`:
- Around line 86-90: Update the build-ai symlink creation block to replace stale
links rather than invoking ln against an existing destination. Preserve the
existing conditional, quote the $3 target path, and use the same forced
symbolic-link handling as the build-clients path: -ln -sfn "$3" ./build-ai.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 32359ff3-0bda-4be2-ad04-34383ce63be0
📒 Files selected for processing (9)
.github/workflows/zola-deploy.yml.gitignoreLOCAL_TESTING.mdbuild/init-topics-and-clients.shconfig.tomlcontent/ai/_index.mdtemplates/ai-list.htmltemplates/default.htmltemplates/macros/docs.html
| ./build/init-topics-and-clients.sh ../valkey-doc/topics \ | ||
| ../valkey-doc/clients | ||
| ../valkey-doc/clients \ | ||
| ../valkey-doc/ai |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Files:"
git ls-files .github/workflows/zola-deploy.yml LOCAL_TESTING.md || true
echo
echo ".github/workflows/zola-deploy.yml:"
if [ -f .github/workflows/zola-deploy.yml ]; then
nl -ba .github/workflows/zola-deploy.yml | sed -n '1,120p'
fi
echo
echo "LOCAL_TESTING.md relevant lines:"
if [ -f LOCAL_TESTING.md ]; then
nl -ba LOCAL_TESTING.md | sed -n '1,80p'
fi
echo
echo "Search valkey-doc references in workflow and docs:"
rg -n --hidden --glob '!node_modules' --glob '!dist' --glob '!build' 'valkey-doc|Checkout valkey-doc|init-topics-and-clients|459' .github workflows LOCAL_TESTING.md . 2>/dev/null | head -200Repository: valkey-io/valkey-io.github.io
Length of output: 302
🌐 Web query:
valkey-io valkey-doc issue 459 or PR 459 AI metadata schema
💡 Result:
There is no record of an issue or pull request #459 specifically related to "AI metadata schema" within the official valkey-io/valkey-doc repository [1][2]. In the context of the Valkey ecosystem: - The valkey-io/valkey-doc repository tracks documentation issues, such as fulltext search documentation (#403) and benchmark updates (#414), but issue #459 is not associated with an AI metadata schema initiative [1][3][4]. - PR #459 in the main valkey-io/valkey repository is unrelated to AI metadata; it concerns unit test migration [5]. - Valkey Search supports metadata filtering for AI-driven workloads (e.g., vector search) through Numeric and Tag fields, which are indexed alongside vectors in Hash or JSON data types [6][7]. These are documented as standard search field types rather than a specialized "AI metadata schema" [7][8]. If you are looking for documentation regarding metadata schemas for AI applications, you may be referring to generic vector search patterns, where metadata is stored as attributes (fields) within a Valkey Hash or JSON document and then explicitly declared in a Valkey Search index schema [6][9][7].
Citations:
- 1: https://github.com/valkey-io/valkey-doc/issues
- 2: https://github.com/valkey-io/valkey-doc
- 3: Adding Fulltext Search Documentation valkey-doc#403
- 4: Update Valkey-Benchmark documentation with details of new changes valkey-doc#414
- 5: Migrate endianconv.c unit tests to new test framework valkey#458
- 6: https://valkey.io/topics/search/
- 7: https://valkey.io/blog/introducing-valkey-search/
- 8: https://valkey-py.readthedocs.io/en/latest/examples/search_vector_similarity_examples.html
- 9: https://valkey-py.readthedocs.io/en/latest/examples/search_json_examples.html
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo ".github/workflows/zola-deploy.yml:"
sed -n '1,140p' .github/workflows/zola-deploy.yml
echo
echo "LOCAL_TESTING.md relevant lines:"
sed -n '1,90p' LOCAL_TESTING.md
echo
echo "Valkey-doc references in tracked docs/workflow:"
rg -n 'valkey-doc|Checkout valkey-doc|clone https://github.com/valkey-io/valkey-doc.git|../valkey-doc/ai|init-topics-and-clients|459' .github/workflows/zola-deploy.yml LOCAL_TESTING.md built 2>/dev/null || trueRepository: valkey-io/valkey-io.github.io
Length of output: 7173
🌐 Web query:
valkey-io valkey-doc#459 or github.com/valkey-io/valkey-doc/pull/459
💡 Result:
There is no record of a pull request #459 in the valkey-io/valkey-doc repository [1][2]. While pull requests exist surrounding that number in the valkey-io/valkey-doc repository (such as #423, #426, #427, #429, #432, #436, and #441), the specific number #459 is not associated with that repository [3][4][5][6][7][8][9]. In the context of the core valkey-io/valkey repository, pull request #459 refers to "Migrate zmalloc.c unit tests to new test framework," which was closed in favor of a replacement pull request (#493) due to conflicts [10][11]. Any reference to #459 in documentation or other contexts is likely unrelated to the valkey-doc repository [2].
Citations:
- 1: https://github.com/valkey-io/valkey-doc
- 2: Valkey Command Reference Spaghetti valkey-doc#31
- 3: Update Benchmark topic with latest valkey-benchmark.c changes valkey-doc#441
- 4: Document server_name, valkey_version, and redis_version INFO fields valkey-doc#427
- 5: Update documentation for Valkey 9.1.0-rc1 valkey-doc#423
- 6: Update CLI options with new environment variables valkey-doc#432
- 7: Add Valkey for AI documentation page valkey-doc#436
- 8: Update RDMA client support valkey-doc#429
- 9: Add Redis compatibility reference to the migration page valkey-doc#426
- 10: [New] Migrate zmalloc.c unit tests to new test framework. valkey#493
- 11: Migrate endianconv.c unit tests to new test framework valkey#458
Pin valkey-doc to the same revision in CI and local setup.
.github/workflows/zola-deploy.yml checks out valkey-io/valkey-doc at the default branch, and LOCAL_TESTING.md clones the same default branch, while the build depends on ../valkey-doc/ai. Either pin both paths to the intended dependency branch/tag/commit or make the merged dependency branch explicit to avoid AI content/build metadata being out of sync.
📍 Affects 2 files
.github/workflows/zola-deploy.yml#L60-L62(this comment)LOCAL_TESTING.md#L32-L33
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/zola-deploy.yml around lines 60 - 62, Pin the valkey-doc
checkout used by the CI workflow and the clone command in LOCAL_TESTING.md to
the same explicit branch, tag, or commit. Update
.github/workflows/zola-deploy.yml around build/init-topics-and-clients.sh and
LOCAL_TESTING.md at the valkey-doc clone so both environments consume the
intended revision consistently.
| #create symlink to AI libraries, except if it already exists | ||
| if [ ! -L build-ai -o "$(readlink build-ai)" != "$3" ]; then | ||
| ln -s $3 ./build-ai | ||
| fi | ||
| echo "Symlink to AI libraries has been created at ./build-ai " |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== repo files matching script =="
fd -a 'init-topics-and-clients\.sh$' . || true
echo "== script excerpt =="
if [ -f build/init-topics-and-clients.sh ]; then
nl -ba build/init-topics-and-clients.sh | sed -n '70,100p'
fi
echo "== readlink behavior in repo shell if available =="
command -v readlink || true
python3 - <<'PY'
import os, sys, tempfile, subprocess
td=tempfile.mkdtemp()
os.chdir(td)
os.symlink('/old', 'build-ai')
# mimic: if [ ! -L build-ai -o "$(readlink build-ai)" != "$3" ]; then ln -s $3 ./build-ai
old='/new/path'
# /new/path splits in unquoted shell; if no /new file exists, ln fails
result=subprocess.run(['ln','-s','/new/path','./build-ai'], text=True, capture_output=True)
print('returncode=', result.returncode)
print('stderr=', repr(result.stderr))
print('links=', os.readlink('build-ai'))
PYRepository: valkey-io/valkey-io.github.io
Length of output: 313
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== script excerpt =="
if [ -f build/init-topics-and-clients.sh ]; then
sed -n '70,100p' build/init-topics-and-clients.sh
printf '\n---\n'
fi
echo "== readlink/ln semantics probe =="
tmp="$(mktemp -d)"
cd "$tmp"
python3 - <<'PY'
import os, tempfile, subprocess
td=tempfile.mkdtemp()
os.chdir(td)
os.symlink('/old', 'build-ai')
cmd=['ln','-s','/new/path','./build-ai']
result=subprocess.run(cmd, text=True, capture_output=True, check=False)
print('cmd=', ' '.join(cmd))
print('returncode=', result.returncode)
print('stderr=', repr(result.stderr))
print('symlink_target=', os.readlink('build-ai'))
PYRepository: valkey-io/valkey-io.github.io
Length of output: 1077
Replace stale build-ai links instead of leaving them in place.
When build-ai is a symlink pointing elsewhere, this branch runs ln -s $3 ./build-ai, which fails because the destination already exists and the stale symlink is not updated. Quote $3 to avoid splitting paths with spaces, and use -ln -sfn "$3" ./build-ai; the existing path handling is already present for build-clients.
🧰 Tools
🪛 Shellcheck (0.11.0)
[warning] 87-87: Prefer [ p ] || [ q ] as [ p -o q ] is not well defined.
(SC2166)
[info] 88-88: Double quote to prevent globbing and word splitting.
(SC2086)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@build/init-topics-and-clients.sh` around lines 86 - 90, Update the build-ai
symlink creation block to replace stale links rather than invoking ln against an
existing destination. Preserve the existing conditional, quote the $3 target
path, and use the same forced symbolic-link handling as the build-clients path:
-ln -sfn "$3" ./build-ai.
Source: Linters/SAST tools
|
| Filename | Overview |
|---|---|
| build/init-topics-and-clients.sh | Adds a required AI directory argument and symlink, but breaks the existing two-argument setup command and mishandles paths containing spaces. |
| templates/ai-list.html | Renders the AI catalog, but one missing first entry hides the page while a missing later entry can fail the full build. |
| content/ai/_index.md | Defines the AI page and its curated list of 14 external metadata files. |
| .github/workflows/zola-deploy.yml | Supplies the new AI metadata directory to the deployment setup script. |
| templates/macros/docs.html | Adds an AI JSON path macro consistent with the existing client metadata macro. |
Reviews (1): Last reviewed commit: "feat: add AI Libraries page to website" | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| if [ -z "$3" ]; then | ||
| echo "You must supply a path to the AI libraries directory as the third argument" | ||
| exit 1 | ||
| fi |
There was a problem hiding this comment.
The existing README.md setup command passes only the topics and clients paths. This new check exits before creating any links or topic files, so contributors following the main build instructions cannot start the local site. Either keep the third argument optional or update every existing caller in the same change.
Artifacts
Repro: isolated executable harness for the documented two-argument bootstrap invocation
- Evidence file captured while the check ran.
- The full command output behind this check.
| if [ ! -L build-ai -o "$(readlink build-ai)" != "$3" ]; then | ||
| ln -s $3 ./build-ai | ||
| fi |
There was a problem hiding this comment.
The directory check accepts a quoted path containing spaces, but the same value is expanded without quotes in ln. A valid local path such as /Users/me/My Projects/valkey-doc/ai is split into multiple arguments, so setup fails or creates the wrong links.
| if [ ! -L build-ai -o "$(readlink build-ai)" != "$3" ]; then | |
| ln -s $3 ./build-ai | |
| fi | |
| if [ ! -L build-ai -o "$(readlink build-ai)" != "$3" ]; then | |
| ln -s "$3" ./build-ai | |
| fi |
| {%- set ai_path = section.extra.recommended_ai_paths | first -%} | ||
| {%- set first_ai = load_data(path = docs::ai_json_path(ai_path=ai_path), format="json", required = false) -%} | ||
| {%- if not first_ai %} |
There was a problem hiding this comment.
If only /python/cognee.json is renamed, removed, or temporarily absent, this optional probe evaluates false and hides all 13 remaining valid libraries behind the page-not-found message. Page availability should not depend on one arbitrary catalog entry.
| <div class="language-clients"> | ||
| <h2 id="{{ language | slugify }}">{{ docs::format_language(language=language) }}</h2> | ||
| {%- for path in section.extra.recommended_ai_paths -%} | ||
| {%- set ai_lib = load_data(path = docs::ai_json_path(ai_path= path), format="json") -%} |
There was a problem hiding this comment.
Unlike the initial optional probe, each later catalog entry is loaded as required. If any listed upstream JSON file is renamed or absent from the checked-out valkey-doc revision, Zola stops the full site build instead of omitting that library, blocking deployment of unrelated pages.
2 participants
Add AI Libraries page to website
Description
This PR implements a comprehensive AI Libraries page on valkey.io to showcase AI/ML frameworks and libraries that support Valkey for caching, vector storage, and memory management. The page mirrors the existing Client Libraries page structure and displays 14 AI libraries organized by programming language.
Related Issue
Fixes #595
Dependencies
The valkey-doc PR must be merged first as it adds the new
valkey_usesfield and improved descriptions to the AI library JSON files that this website template consumes.Changes
Build System
build-aisymlinkai_json_path = "../build-ai/"configuration../valkey-doc/aiparameterbuild-aito ignore generated symlinksContent & Templates
recommended_ai_pathsai_json_path()helper macroDocumentation
New Page Structure
The AI Libraries page displays for each library:
Libraries are organized by language:
Example Output
Testing
Local Testing Results
Screenshots
(Add screenshots of the rendered AI Libraries page after pushing)
Checklist
Analytics Support
The page includes the standard Google Tag Manager (GTM) integration (
GTM-MFFCB7SR) that allows the Valkey team to:GTM administrators can configure link tracking to see click-through rates to individual library repositories.
Notes for Reviewers
Template Design: The
ai-list.htmltemplate is intentionally simplified compared toclient-list.html- it doesn't include a feature comparison table since AI libraries have different integration patterns (vector store vs caching vs memory management)Field Ordering: The Description and Valkey Uses fields appear between Repo and Installation to provide immediate context about what the library does and how it uses Valkey before showing installation instructions
Whitespace Control: Template uses
{%-and-%}for whitespace control to prevent excessive blank lines in the generated HTMLBuild Process: The init script creates a symlink (
build-ai -> ../valkey-doc/ai) so the website can access the JSON files from the valkey-doc repositoryFuture Enhancements
Potential improvements for future PRs:
Page URL after merge: https://valkey.io/ai/