Skip to content

Look up config properties in the published reference, not the UI sample - #455

Merged
JakeSCahill merged 4 commits into
mainfrom
fix/property-lookup-published-reference
Nov 12, 2026
Merged

JakeSCahill merged 4 commits into
mainfrom
fix/property-lookup-published-reference

Conversation

@JakeSCahill

Copy link
Copy Markdown
Contributor

Ask AI's lookup_config_property tool read redpanda-properties.json from the UI bundle. That file is a 9-property sample, and production serves it too, so common properties such as default_topic_replications and write_caching_default came back "not found". The tool tells the model to use it for grounded values, so it was steering answers away from exact values.

The tool now reads the reference the property tooltips already use: the file that <meta name="properties-json-url"> names, which the docs build publishes per release. Today that's /streaming/current/reference/_attachments/redpanda-properties-v26.2.4.json, with 691 properties. Solutions pages carry the same meta.

  • On a page with no meta, it says the page names no property reference, so the model falls back to docs search. It never reads the sample outside the docs-ui preview.
  • In the docs-ui preview, which has no docs build behind it, it still reads the bundled sample.
  • It caches by URL, and retries after a failed fetch.
  • It also returns cloudEditable, cloudReadonly and cloudByocOnly when the reference has them.

Tests: tests/property-lookup-tool (7 cases), wired into test:all and validate-build.yml. With main's loader restored, 6 of the 7 fail. The tests pass on Node 18 (node:18-alpine) and lint is clean. A run against the live 26.2.4 file returns default_topic_replications (default 1, cluster) and write_caching_default, and reports an unknown name as not found.

@coderabbitai

coderabbitai Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 756b0813-b121-451a-930a-9ff1cdd0471f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review

No actionable comments were generated in the recent review.

Walkthrough

The property lookup tool now uses the property-reference URL in page metadata. UI-preview pages without that URL use the UI-root JSON file; other pages without a reference return an error. Successful references are cached by URL, and failed loads can be retried. Lookup results include three Cloud-related fields. New tests cover reference selection, caching, retries, and preview fallback. The tests are added to the full test sequence and build validation workflow.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant PageMetadata
  participant lookup_config_property
  participant fetch
  PageMetadata->>lookup_config_property: provide properties-json-url
  lookup_config_property->>fetch: request property reference
  fetch-->>lookup_config_property: return property records
  lookup_config_property-->>PageMetadata: return property data and Cloud fields
Loading

Suggested reviewers: feediver1

Merge Risk: Minimal · previous review fce83

The property lookup change is mergeable after normal checks; no actionable risk is established by the supplied evidence.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Documentation-only edits and deletions within survi… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed

Check name Status Explanation
Description check Passed The description clearly explains the change from the UI sample to the published property reference, fallback behavior, caching, returned fields, and test coverage.
Title check Passed The title clearly and concisely summarizes the main change: looking up configuration properties in the published reference instead of the UI sample.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 3 functions across 2 files. (3 skipped: 3 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches

Generate docstrings

  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 1cd8f0ac-ff9d-4559-964c-fe73f298a1b9
Commits

Reviewing files that changed from the base of the PR and between 78442c3 and fce8331.

Files selected for processing: 5

  • .github/workflows/validate-build.yml
  • package.json
  • src/js/react/agentTools.js
  • tests/property-lookup-tool/fixture-published.json
  • tests/property-lookup-tool/property-lookup-tool.test.js

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

@netlify

netlify Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for docs-ui ready!

Name Link
🔨 Latest commit 86b9347
🔍 Latest deploy log https://app.netlify.com/projects/docs-ui/deploys/6af44421ed0eaa64b7cd56ee
😎 Deploy Preview https://deploy-preview-455--docs-ui.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
Lighthouse
Lighthouse
1 paths audited
Performance: 50 (🟢 up 2 from production)
Accessibility: 89 (no change from production)
Best Practices: 83 (no change from production)
SEO: 89 (no change from production)
PWA: -
View the detailed breakdown and full score reports
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@JakeSCahill
JakeSCahill marked this pull request as ready for review October 27, 2026 17:16
@JakeSCahill
JakeSCahill requested a review from a team as a code owner October 27, 2026 17:16

@Feediver1 Feediver1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at: fce8331
Verdict: Blocked (1 blocker)

Summary

Ask AI's lookup_config_property now reads the file named by <meta name="properties-json-url"> (same as the tooltips) instead of the 9-property UI sample, caches by URL, retries after a failed fetch, and returns the Cloud flags. A clear improvement: the meta is present on streaming, Cloud, Connect, and home pages (all resolve to /streaming/current/reference/_attachments/redpanda-properties-v26.2.4.json, 691 properties).

Security: the fetch URL comes from the page's own build-time meta (same-origin relative path), and the data goes to the model as a JSON tool result, not into the DOM, so this change adds no XSS surface. Failure fallback is safe: no meta returns found: false with a clear error, a failed fetch is retried next time.

Verified: node --test tests/property-lookup-tool 7/7 locally. CI passes. Checked claims against the live 26.2.4 JSON.

Blocker

  1. Properties marked exclude_from_docs come back as normal results. src/js/react/agentTools.js:~420. The published file has 3: enable_developmental_unrecoverable_data_corrupting_features, redpanda.remote.allowgaps, redpanda.virtual.cluster.id. The docs hide them on purpose, and the old sample never had them. Now the model gets found: true with a default and scope, and can present a data-corrupting developer flag as a supported setting. Fix: if (!rec || rec.exclude_from_docs) return { found: false, ... }, plus a test case.

Should fix

  1. The docs-ui preview claim is wrong. The PR says the preview still reads the bundled sample. It doesn't: the preview page emits <meta name="properties-json-url" content="/streaming/reference/_attachments/redpanda-properties-v25.1.1.json"> (from preview-src/ui-model.yml through head-meta.hbs), which 404s on https://deploy-preview-455--docs-ui.netlify.app. propertiesUrl() (agentTools.js:375) returns the meta before it checks isUiPreview, so every preview lookup errors. The test passes only because it leaves out the meta the real preview emits. Fix: check isUiPreview first, or fall back to the sample when the meta fetch 404s in preview.
  2. Raw AsciiDoc conditionals in description. redpanda.remote.readreplica and replication.factor descriptions contain ifdef::env-cloud[] ... endif::[] text, so the model can tell a Self-Managed user that Remote Read Replicas are BYOC-only. Return description_html stripped to text (what the tooltips use) or strip the conditional blocks.
  3. Aliases are not resolved. The published file has aliases (for example data_transforms_per_function_memory_limit ← wasm_per_function_memory_limit, cloud_storage_max_segment_readers_per_shard ← cloud_storage_max_readers_per_shard). A lookup by the old name returns "not in the property reference". Build an alias map on load.
  4. Tool description contradicts the new fields. agentTools.js:~396 still says "do NOT use it to answer Redpanda Cloud configuration questions" while the result now includes cloudEditable, cloudReadonly, cloudByocOnly. Update the description to say when those flags apply.

Suggestions

  1. uint64 precision: r.json() turns virtual_cluster_min_producer_ids default 18446744073709551615 into 18446744073709552000. Only 1 property in 26.2.4 is affected. 19-property-tooltips.js:483 already has parsePreservingBigInts. Reuse it.
  2. The result doesn't say which release the values come from. The meta is per page version, so on an older-version page the tool returns that version's defaults. Include the latest-redpanda-tag meta value as version in the result.
  3. The loader duplicates 19-property-tooltips.js (URL, preview detection, fetch, cache) and the two already differ (hostname vs isUiPreview, bigint-safe vs plain, localStorage vs in-memory). Expose one shared loader to avoid a second ~700 KB fetch and parse per page view.

Merge and deploy

  • Conflicts with docs-ui#456 in package.json (test:all). Both are clean against main; the second to merge needs a rebase.
  • No playbook bump needed (docs-site/antora-playbook.yml:128 uses releases/latest). Takes effect after a docs-ui release and a docs-site rebuild.

…ublished-reference

# Conflicts:
#	package.json
…ookup

- Return found: false for properties marked exclude_from_docs.
- In the docs-ui preview, read the bundled sample before the page meta,
  which names a file the preview does not have.
- Return description_html as plain text, so AsciiDoc ifdef blocks never
  reach the model.
- Resolve old names through each record's aliases.
- Parse with the tooltips' bigint-safe parser, shared on window.
- Return the reference's release as version, and say in the tool
  description which fields apply to Redpanda Cloud.
@JakeSCahill

Copy link
Copy Markdown
Contributor Author

@Feediver1 thanks for the review. Fixes are in 0d77523, on top of a merge of main (d7aa803) that resolves the #456 test:all conflict.

Blocker

  1. exclude_from_docs: fixed in 0d77523. Those properties now return found: false. Added a test with enable_developmental_unrecoverable_data_corrupting_features.

Should fix
2. Preview: you were right, the preview emits a meta for a file it does not have. Fixed in 0d77523: the preview check now runs first. Added a test where the preview page has that meta.
3. AsciiDoc conditionals: fixed in 0d77523. The tool returns description_html as plain text, with inline code kept in backticks. Added a test with redpanda.remote.readreplica. Against the live 26.2.4 file, no returned description has an ifdef/endif or an HTML tag left in it.
4. Aliases: fixed in 0d77523. An alias map is built on load, and the result has alias set when an old name was used. Tested with wasm_per_function_memory_limit.
5. Tool description: updated in 0d77523. It now says type, default, and description are Self-Managed values, and explains when each Cloud flag applies.

Suggestions
6. uint64: fixed in 0d77523. The tooltips script now puts parsePreservingBigInts on window, and the tool uses it (falling back to JSON.parse). virtual_cluster_min_producer_ids now returns 18446744073709551615. Tested.
7. Version: fixed in 0d77523. The result has version from latest-redpanda-tag, which head-meta always emits next to properties-json-url. It is null in the preview. Tested.
8. Shared loader: I'm leaving this for a follow-up. The tooltips loader is an IIFE in site.js with localStorage caching and the tool is a lazy ES module, so merging them is a bigger refactor. The tool also only fetches when the model calls it, and that is the same URL the tooltips already fetched, so the browser cache usually serves it.

Each new test fails when its fix is reverted (one at a time, from a backup) and passes with the fix: 14/14 in tests/property-lookup-tool. Lint and gulp bundle pass.

@github-actions

This comment has been minimized.

…ublished-reference

# Conflicts:
#	package.json
@github-actions

Copy link
Copy Markdown

📋 Docs review: ✅ Ready to approve

Re-review of the changes since 0d77523 (docs-review report). Earlier merge-blockers stay listed until the review that raised them reports them fixed.

What the last review said: previous report.

Not approved automatically: automatic approval is switched off for redpanda-data/docs-ui.

Since the last review (at 0d77523)

  • No longer reported: 1
    • PR and preview checks: The PR has merge conflicts with its base branch: rebase or merge the base before it can merge.
  • Still reported: 0
  • New: 0

Matched by review and finding text. A finding reworded between runs counts as one no longer reported plus one new.

✅ PR and preview checks: no merge-blockers

Merge order (not blocking):

✅ Best-fit review (code review (GitHub workflow and npm scripts)): no merge-blockers

Review: PR 455 (incremental re-review since 0d77523)

The only change since the last review is merge-conflict resolution in .github/workflows/validate-build.yml and package.json — restoring three CI test steps and four test: scripts that main added, while keeping this PR's own test:property-lookup-tool wiring. Reviewed as code (GitHub workflow + npm script configuration).

Merge-blockers

None.

Suggestions

None.

What works well

  • The resolution matches main exactly where it should: the three restored steps in .github/workflows/validate-build.yml:100-105 are byte-identical in name and command to main's, and package.json's test:all is main's chain with only npm run test:property-lookup-tool inserted. Nothing from either side was dropped.
  • Every script the delta adds resolves to a real directory containing a matching test file: tests/ui-attributes/, tests/page-attribute-prefix/, tests/json-attribute-guard/, tests/marketing-links/. All npm run references inside test:all resolve to defined scripts.
  • The PR's own step (.github/workflows/validate-build.yml:110-111) stays consistent with the job's Chrome-free premise documented at lines 58-64: tests/property-lookup-tool/property-lookup-tool.test.js imports only node:test, node:assert, node:fs, node:path, and node:vm, so PUPPETEER_SKIP_DOWNLOAD cannot break it.
  • The newly wired test:ui-attributes manifest check is safe for this PR: the two source files it changes (src/js/19-property-tooltips.js, src/js/react/agentTools.js) contain no page.attributes/asciidoc.attributes reads that tests/ui-attributes/scan-attribute-reads.js would pick up, so src/ui-attributes.json needs no update under CLAUDE.md's manifest rule.

@micheleRP micheleRP left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Jake! Reading properties-json-url (the same reference the tooltips use) works against the live site: current, Cloud, Connect, and home resolve to redpanda-properties-v26.2.6.json (691 properties), 26.1 to v26.1.14, and 25.3 to v25.3.11, all 200. Pages on 25.2 and older have no meta and fall back to docs search, which beats the old 9-property sample. Against the live JSON, aliases (wasm_per_function_memory_limit), exact uint64 values, and per-version names behave correctly. 14/14 property-lookup tests pass, head-meta and latest-version tests pass, and eslint has no new errors.

Non-blocking:

  • The description still says "7 cases" and "26.2.4"; it's 14 tests now and the live file is v26.2.6.
  • referenceVersion() duplicates readRelease() (the bot noted this too).

@Feediver1, your fce8331 blocker looks fixed to us. Could you confirm?

@JakeSCahill
JakeSCahill merged commit 0b559d9 into main Nov 12, 2026
36 checks passed
@JakeSCahill
JakeSCahill deleted the fix/property-lookup-published-reference branch November 12, 2026 10:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants