Skip to content

docs: add macOS native cursor capture test pipeline#673

Merged
siddharthvaddem merged 2 commits into
siddharthvaddem:mainfrom
kaili-yang:docs/mac-native-cursor
Jun 2, 2026
Merged

docs: add macOS native cursor capture test pipeline#673
siddharthvaddem merged 2 commits into
siddharthvaddem:mainfrom
kaili-yang:docs/mac-native-cursor

Conversation

@kaili-yang
Copy link
Copy Markdown
Contributor

@kaili-yang kaili-yang commented Jun 1, 2026

Summary

  • Added docs/testing/macos-native-cursor.md documenting the macOS native cursor capture feature
  • Covers how the cursor helper works (bitmap capture, SHA-256 dedup, asset payload protocol), build steps, permissions, and a manual test checklist
  • Explains expected sidecar shape and known limitations to make future maintenance and iteration easier

Summary by CodeRabbit

  • Documentation
    • Added a macOS native cursor testing and diagnostics guide covering setup and run steps, required system permissions, a structured manual test checklist (bitmap capture, hotspot alignment, click detection, multi‑display behavior, memory stability), expected recording contents, capture/backend routing notes, and known limitations.

Documents the cursor helper binary, permissions, manual test checklist,
expected sidecar shape, and known limitations for the darwin native
cursor path.
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Jun 1, 2026

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: dbeb84ec-6837-451f-85dd-702329f7a214

📥 Commits

Reviewing files that changed from the base of the PR and between 53be8df and a280179.

📒 Files selected for processing (1)
  • docs/testing/macos-native-cursor.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • docs/testing/macos-native-cursor.md

📝 Walkthrough

Walkthrough

Adds a new macOS native cursor helper testing and diagnostics document describing helper behavior, build/run and override steps, permissions, a prioritized manual test checklist, expected cursor sidecar JSON, integration routing, and platform-specific limitations.

Changes

macOS Native Cursor Helper Testing & Diagnostics Documentation

Layer / File(s) Summary
Overview and helper architecture
docs/testing/macos-native-cursor.md
Explains helper sampling of NSCursor.currentSystem, hashing/deduping bitmaps into stable asset IDs, interaction tagging, and optional Accessibility-based affordance substitution.
Build, smoke test, and quick validation
docs/testing/macos-native-cursor.md
Documents building native helpers, running the standalone smoke test with expected ready/sample JSON output, and instructions to validate asset changes when hovering (including Accessibility-driven assets).
Configuration, permissions, and override
docs/testing/macos-native-cursor.md
Describes OPENSCREEN_MAC_CURSOR_HELPER_EXE override, required macOS permissions (Screen Recording mandatory, Accessibility optional), fallback behavior when Accessibility is not granted, and restart requirements after permission changes.
Manual test checklist (P0–P2)
docs/testing/macos-native-cursor.md
Priority-grouped manual tests covering bitmap capture fidelity, Accessibility affordance substitution, Retina hotspot alignment, click detection/session sidecar expectations, graceful degradation, multi-display behavior, and long-running memory stability.
Expected recording JSON / sidecar structure
docs/testing/macos-native-cursor.md
Specifies what a healthy .openscreen cursor sidecar should contain (provider: "native", non-empty assets/samples) and the failure signature when helper unavailable/unready.
Integration, routing, and known limitations
docs/testing/macos-native-cursor.md
Describes native capture backend routing via ScreenCaptureKit helper, current availability rules, diagnostic workflow recap, and known platform limitations (Intel build mismatch, dev-mode Accessibility trust quirks, CGS-layer cursor visibility).

Estimated Code Review Effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly Related PRs

  • siddharthvaddem/openscreen#671: Introduces the native macOS cursor helper that this documentation describes, including NSCursor bitmap capture, asset deduplication, and Accessibility-driven affordance substitution.

Suggested Reviewers

  • siddharthvaddem

Poem

at 2am I read the mac cursor lore,
bitmaps hashed, deduped, and mapped to an id or more.
perms toggled, hotspots lined up neat,
smoke-test says "ready"—lowkey a treat.
docs stitched up, quirks called out, mission complete. ✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the main change—adding documentation for the macOS native cursor capture test pipeline, which aligns with the new docs file added.
Description check ✅ Passed The description covers the essential changes (new doc, what it covers, purpose) but lacks detail in Testing section and doesn't explicitly mark the Type of Change checkbox.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 53be8df129

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread docs/testing/macos-native-cursor.md Outdated
Comment thread docs/testing/macos-native-cursor.md Outdated
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/testing/macos-native-cursor.md`:
- Line 113: The doc uses two different sidecar key names
("cursor-recording-data" and "cursorRecordingData"); pick one canonical key
(prefer the project’s canonical JSON/key naming convention — e.g.,
cursorRecordingData if code uses camelCase or cursor-recording-data if
kebab-case) and update all occurrences (including the references near the
interactionType: "click" and "mouseup" guidance and the duplicate at the later
occurrence) so the test steps consistently reference that single key; ensure any
example JSON, text references, and the phrase "recording session sidecar" all
show the chosen key.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 77e53d8e-aea3-404c-a678-f4d37571c0e6

📥 Commits

Reviewing files that changed from the base of the PR and between d2dd44a and 53be8df.

📒 Files selected for processing (1)
  • docs/testing/macos-native-cursor.md

Comment thread docs/testing/macos-native-cursor.md Outdated
- Unify sidecar key name to camelCase cursorRecordingData (was mixing
  cursor-recording-data and cursorRecordingData in the same checklist)
- Graceful-degradation test now removes both binary copies so the
  missing-helper fallback is actually exercised (build/ and bin/darwin-*)
- Healthy recording section points to <videoPath>.cursor.json sidecar
  instead of the .openscreen project file, which does not embed cursor data
@kaili-yang kaili-yang closed this Jun 2, 2026
@kaili-yang kaili-yang reopened this Jun 2, 2026
@siddharthvaddem siddharthvaddem merged commit 94a00c0 into siddharthvaddem:main Jun 2, 2026
26 of 27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants