Skip to content

fix(cursor): native cursor overlay fixes for multi-display, crop, zoom and export consistency#2

Merged
EtienneLescot merged 3 commits into
EtienneLescot:feat/macos-native-capture-pipelinefrom
auberginewly:fix/cursor-canvas-clip
May 18, 2026
Merged

fix(cursor): native cursor overlay fixes for multi-display, crop, zoom and export consistency#2
EtienneLescot merged 3 commits into
EtienneLescot:feat/macos-native-capture-pipelinefrom
auberginewly:fix/cursor-canvas-clip

Conversation

@auberginewly
Copy link
Copy Markdown

@auberginewly auberginewly commented May 16, 2026

Summary

A series of fixes for the macOS native cursor overlay across the
multi-display recording, video crop, zoom, rounded-corner and export
scenarios — keeping the preview path (VideoPlayback.tsx) and the
export path (frameRenderer.ts) consistent — plus a user-facing
toggle to allow the cursor to overflow the canvas.

Problems & fixes

Multi-display ghost cursor & motion trail

Recording display A, moving the mouse to non-recorded display B clamped
the cursor to the canvas edge and kept it visible, producing a long
trail flying back from the edge on return.

  • In macNativeCursorRecordingSession, mark visible: false when out of
    bounds — this triggers hideNativeCursorPreview() and resets smoothing /
    motion blur, so the cursor snaps cleanly on return.
  • Add hysteresis (only hide after 3 consecutive out-of-bounds samples,
    ≈100ms); fast swipes out-and-back are handled by clip-path at the edge
    instead of flickering invisible.

Cursor overflowing the video canvas

The enlarged cursor overlay bled into the editor background. The
ancestor's overflow:hidden was broken by transform-style: preserve-3d
on composite3DRef.

  • Switch to dynamic clip-path: inset() (unaffected by preserve-3d),
    clipping precisely via baseMaskRef (incl. crop) + border radius.
  • Move nativeCursorClipRef out of the preserve-3d container so
    clip-path stays correct under 3D-rotation zoom regions.
  • Make the clip-path camera-aware so it follows cameraContainer
    scale/translation.

Preview / export cursor size mismatch

renderAsset.width is absolute px; preview canvas ~600px vs export
1920px gave a ~3x size discrepancy.

  • Both paths now normalize by maskRect.width / croppedVideoWidth, so
    the cursor occupies the same fraction of the video in both.

Lost rounded corners & zoom over-crop on export

  • Export goes through Canvas 2D; PIXI WebGL mask alpha is unreliable at
    corners → explicit roundRect clip enforces rounded corners (shadow
    path erases square corners via destination-out + evenodd, then
    redraws).
  • Export cropped off part of the video while zoomed: the PIXI mask
    is inside cameraContainer and scales with zoom so the video fills the
    magnified bounds, but the static maskRect rounded clip on export still
    used the small non-zoom size. Extracted cameraAwareMaskRect() to
    share one camera transform across shadow / direct / cursor paths;
    degrades to the original static behavior at zoom=1 (no regression).

Consistency cleanup

  • Cache maskBorderRadius in LayoutCache so preview and export share
    the same rounded-corner value.
  • Remove a conflicting borderRadiusRef sync effect: a standalone
    effect overwrote the screenCover→0 (full-bleed) corrected value with
    the raw prop, making vertical-stack/split presets disagree between
    preview and export.

New feature: "Clip to Canvas" toggle

Clipping the cursor to the video bounds is now optional. Added a
cursorClipToBounds setting (default on, fully backward compatible)
in the Show Cursor settings section:

  • Default on → cursor is clipped to the video canvas (current behavior).
  • Off → the cursor is allowed to overflow into the background, in both
    the preview and the exported MP4/GIF.
  • Gating is applied only to the cursor clip (preview clip-path,
    export drawNativeCursor); the video rounded-corner clips in
    compositeWithShadows are fully decoupled and unaffected.
  • Session-only, not persisted (consistent with the other cursor
    settings).

The entire cursor settings section (Show Cursor / Size / Smoothing /
Motion Blur / Click Bounce / Clip to Canvas) is now i18n-backed across
all 11 locales (previously hardcoded English).

Key files

File Change
electron/native-bridge/cursor/recording/macNativeCursorRecordingSession.ts out-of-bounds visible:false + hysteresis
src/components/video-editor/VideoPlayback.tsx dynamic camera-aware clip-path, moved out of preserve-3d, size normalization, removed redundant effect, clip gated by toggle
src/components/video-editor/videoPlayback/layoutUtils.ts expose maskBorderRadius on LayoutResult
src/lib/exporter/frameRenderer.ts cameraAwareMaskRect() helper, rounded corners, camera-aware clip, size normalization, cursor clip gated by toggle
src/components/video-editor/{VideoEditor,SettingsPanel}.tsx, types.ts cursorClipToBounds state + Switch UI + i18n
src/lib/exporter/{videoExporter,gifExporter}.ts thread cursorClipToBounds into export config
src/i18n/locales/*/settings.json (11 locales) new cursor section

Test plan

  • Single display: cursor near the edge is clipped, no bleed into editor background; clipped to rounded corners with border radius; confined to cropped area with crop
  • Multi-display: moving to non-recorded display hides the cursor immediately; returning shows it with no trail; fast swipe out-and-back does not flicker invisible
  • Zoom: cursor stays inside the video bounds in both preview and exported mp4; zoom=1 behaves as before
  • Rounded corners + zoom: export keeps correct rounded corners and does not over-crop content
  • Preview vs export: cursor occupies the same fraction of the video
  • "Clip to Canvas" toggle off: cursor overflows the canvas in both preview and exported MP4/GIF; video rounded corners still correct
  • "Clip to Canvas" toggle on (default): behavior identical to before
  • Cursor settings labels are translated when switching language (11 locales)
  • npm run i18n:check, tsc --noEmit, npm run build-vite all pass

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added cursor clipping option in settings to control whether the cursor is confined to video canvas bounds
  • Improvements

    • Enhanced cursor visibility tracking when cursor moves outside display boundaries
    • Improved cursor rendering consistency between on-screen preview and exported videos

Review Change Stack

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 16, 2026

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 3ddb1cb0-4cc8-494e-9003-fc5a98dd0ff4

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@auberginewly auberginewly force-pushed the fix/cursor-canvas-clip branch from 712bd02 to 1d2a421 Compare May 16, 2026 13:02
@auberginewly auberginewly changed the title fix(cursor): clip overlay to canvas boundary and hide cursor on non-recorded display fix(cursor): native cursor overlay 多显示器/裁剪/zoom/导出一致性修复 May 17, 2026
@auberginewly auberginewly force-pushed the fix/cursor-canvas-clip branch from 64a755a to 055e04a Compare May 17, 2026 08:27
@auberginewly auberginewly changed the title fix(cursor): native cursor overlay 多显示器/裁剪/zoom/导出一致性修复 fix(cursor): native cursor overlay fixes for multi-display, crop, zoom and export consistency May 17, 2026
@auberginewly
Copy link
Copy Markdown
Author

现在我是已经把光标 overflow 给删掉了 修了多余运动轨迹采集的 bug 我来加一个可选 overflow or not 的开关

After more testing I've reconsidered the cursor-overflow approach:

Removed the overflow clipping. The hard clip-path/canvas clipping that confined the cursor to the video bounds is gone — clipping the native cursor at the edge looked worse than letting it sit slightly outside in some cases.
Fixed the real root cause: redundant motion-trail sampling. The phantom trail wasn't really an overflow problem — it came from stale out-of-bounds samples being collected and fed into smoothing. That's now fixed at the sampling layer, so the trail/ghosting is gone without needing to clip.
Next: make overflow opt-in. I'll add an optional "clip cursor to canvas / allow overflow" toggle so the behavior is configurable instead of hard-coded, rather than forcing one choice on everyone.
Will push the toggle commit shortly.

@auberginewly
Copy link
Copy Markdown
Author

@EtienneLescot

This part is done 👇

Multi-display ghost cursor / motion-trail fix
Cursor-overflow clipping (preserve-3d / camera-aware / rounded corners / crop)
Preview vs export cursor size normalization
Lost rounded corners & zoom over-crop on export — fixed
New optional "Clip to Canvas" toggle (default on, backward compatible; turn off to let the cursor overflow the canvas)
Entire cursor settings section now i18n-backed across 11 locales
i18n:check / tsc / build-vite all pass locally.

Flow: please @EtienneLescot review and merge this PR into feat/macos-native-capture-pipeline first; it then flows upstream into main via siddharthvaddem#573. Ready for review — thanks! 🙏

@EtienneLescot
Copy link
Copy Markdown
Owner

@auberginewly Thanks for the contribution, the feature looks relevant and useful to me overall.

I’d like to keep this PR focused and easy to review before merging it into the macOS native capture branch. Could you please squash/reorganize the current commits into 1 or 2 clean commits? There are currently quite a few intermediate commits for the size of the change, and a cleaner history would make the contribution easier to understand.

One technical point to double-check before merge: in cameraAwareMaskRect() inside src/lib/exporter/frameRenderer.ts, the export path clamps the transformed mask bounds to the stage using Math.max(0, ...) / Math.min(stageW/stageH, ...). In the preview path, the CSS clip-path can naturally use bounds that extend outside the canvas. Could you verify whether this can create a preview/export mismatch when zoom/pan pushes the mask outside the stage, especially with rounded corners? If needed, please adjust the export behavior so it matches preview.

Apart from that, the functional scope looks good to me: cursor canvas clipping, editor/export consistency, and the related cursor settings i18n all seem relevant.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🧹 Nitpick comments (1)
src/i18n/locales/zh-TW/settings.json (1)

12-12: Pre-existing structural issue: empty speed object.

Line 12 contains an empty "speed": {} object nested within the zoom section. This is not part of the current changes but may cause confusion or linting issues.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/i18n/locales/zh-TW/settings.json` at line 12, The JSON contains an
extraneous empty "speed" object inside the "zoom" section; remove the empty
"speed": {} entry (or replace it with the proper localized keys/values if
intended) so the "zoom" object only contains meaningful translation entries—look
for the "zoom" object and the "speed" key in settings.json and delete or
populate that empty "speed" entry.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/components/video-editor/SettingsPanel.tsx`:
- Around line 1399-1407: The new Switch (checked={cursorClipToBounds},
onCheckedChange={onCursorClipToBoundsChange}) lacks an accessible name; update
the Switch to include either an aria-label (e.g.,
aria-label={t("cursor.clipToBounds")}) or wire it to the existing text using
aria-labelledby by giving the text element an id and passing that id to
aria-labelledby on the Switch so screen readers announce what the control
toggles within SettingsPanel.

In `@src/components/video-editor/VideoPlayback.tsx`:
- Around line 2036-2055: The native cursor clip DIV (nativeCursorClipRef) is
placed outside composite3DRef so it doesn't receive the same
contain-scale/rotateX/Y/Z 3D transforms and thus the preview cursor stays
unrotated; fix by ensuring the cursor clip and image share the same 3D transform
as the video—either move the nativeCursorClipRef (and its nativeCursorImageRef)
inside the composite3DRef container or apply the identical transform
styles/classes (preserve-3d + the computed
contain-scale/rotateX/rotateY/rotateZ) to nativeCursorClipRef at the same point
where composite3DRef is updated so preview and export remain consistent.

In `@src/i18n/locales/vi/settings.json`:
- Around line 194-201: The translation for cursor.motionBlur ("Mờ chuyển động")
is inconsistent with effects.motionBlur ("Làm mờ chuyển động"); update the value
for "cursor.motionBlur" in the settings.json locale so it matches the existing
phrasing used by "effects.motionBlur" to ensure consistent terminology across
the UI (look for the "cursor.motionBlur" and "effects.motionBlur" keys and
replace the cursor entry with the same string as effects.motionBlur).

In `@src/i18n/locales/zh-TW/settings.json`:
- Around line 195-202: The translation for cursor.motionBlur should match the
existing term used elsewhere: replace the value for "cursor.motionBlur"
(currently "運動模糊") with the same string used by "effects.motionBlur" ("動態模糊") so
both keys use consistent terminology; update the "cursor" object entry for
motionBlur accordingly.

In `@src/lib/exporter/frameRenderer.ts`:
- Around line 529-546: The cameraAwareMaskRect function currently clamps mask
edges to the stage using Math.max/Math.min which forces export clipping inside
the canvas; remove those clamps so the computed rect can extend outside the
stage: compute left/top as camX + camS * maskX and right/bottom as camX + camS *
(maskX + maskW) (using layoutCache.maskRect, layoutCache.stageSize, and
animationState.appliedScale/x/y to locate the code), then return x=left, y=top,
width=right-left, height=bottom-top and br=layoutCache.maskBorderRadius * camS
without applying Math.max/Math.min clamping.

---

Nitpick comments:
In `@src/i18n/locales/zh-TW/settings.json`:
- Line 12: The JSON contains an extraneous empty "speed" object inside the
"zoom" section; remove the empty "speed": {} entry (or replace it with the
proper localized keys/values if intended) so the "zoom" object only contains
meaningful translation entries—look for the "zoom" object and the "speed" key in
settings.json and delete or populate that empty "speed" entry.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 16b1f9bc-6281-4dfe-826e-bc1f3999b154

📥 Commits

Reviewing files that changed from the base of the PR and between 7675895 and 8e65240.

📒 Files selected for processing (20)
  • electron/native-bridge/cursor/recording/macNativeCursorRecordingSession.ts
  • src/components/video-editor/SettingsPanel.tsx
  • src/components/video-editor/VideoEditor.tsx
  • src/components/video-editor/VideoPlayback.tsx
  • src/components/video-editor/types.ts
  • src/components/video-editor/videoPlayback/layoutUtils.ts
  • src/i18n/locales/ar/settings.json
  • src/i18n/locales/en/settings.json
  • src/i18n/locales/es/settings.json
  • src/i18n/locales/fr/settings.json
  • src/i18n/locales/ja-JP/settings.json
  • src/i18n/locales/ko-KR/settings.json
  • src/i18n/locales/ru/settings.json
  • src/i18n/locales/tr/settings.json
  • src/i18n/locales/vi/settings.json
  • src/i18n/locales/zh-CN/settings.json
  • src/i18n/locales/zh-TW/settings.json
  • src/lib/exporter/frameRenderer.ts
  • src/lib/exporter/gifExporter.ts
  • src/lib/exporter/videoExporter.ts

Comment thread src/components/video-editor/SettingsPanel.tsx
Comment thread src/components/video-editor/VideoPlayback.tsx
Comment thread src/i18n/locales/vi/settings.json
Comment thread src/i18n/locales/zh-TW/settings.json
Comment thread src/lib/exporter/frameRenderer.ts
…iew and export

- Add nativeCursorClipRef div (outside preserve-3d) with CSS inset() clip-path that
  tracks the camera-transformed video boundary, including border-radius
- Add cameraAwareMaskRect() in FrameRenderer that computes the same boundary for
  Canvas 2D clip in the export path; remove stage-clamping so rounded corners match
  the preview's inset() behavior when zoom/pan pushes the mask off-stage
- Cache maskBorderRadius in LayoutCache so both shadow and direct composite paths
  can apply camera-aware rounded clipping
- Fix double mask.x offset introduced by nativeCursorMaskRef; replace mask div with
  clip-path on the outer wrapper
- Normalize cursor size relative to maskRect.width so preview and export scale match
- Clip cursor to canvas boundary and hide on non-recorded display
- Wire cursorClipToBounds flag through FrameRenderConfig and VideoExporter
Add a cursor.clipToBounds toggle to the Settings panel (default on) that controls
whether the native cursor is clipped to the video canvas boundary in both preview
and export. Wire up 11 locale files (ar, en, es, fr, ja-JP, ko-KR, ru, tr, vi,
zh-CN, zh-TW) with the new cursor settings section.
@auberginewly auberginewly force-pushed the fix/cursor-canvas-clip branch from 8e65240 to cfd0042 Compare May 18, 2026 09:46
@auberginewly
Copy link
Copy Markdown
Author

Done — two things addressed:

Commits squashed
The 20 intermediate commits are now 2 clean ones on fix/cursor-canvas-clip:

  • feat(cursor): clip native cursor to camera-aware video bounds in preview and export
  • feat(cursor): add cursorClipToBounds setting with i18n translations

cameraAwareMaskRect() inconsistency — confirmed and fixed
The concern was valid. The export path was clamping the transformed mask bounds to stage dimensions via Math.max(0, …) / Math.min(stageW/stageH, …), while the preview's CSS inset() clip-path naturally accepts out-of-bounds values.

The specific mismatch with rounded corners: when zoom/pan pushes the mask rect partially off-stage (e.g. left edge is at x = -30), the clamped export path would start the roundRect clip at x = 0 — producing a visible rounded corner at the stage edge. The preview has no such rounding there (the browser straight-cuts at the stage edge). This removed the stageW/stageH clamping and lets Canvas 2D clip naturally to its bounds, mirroring CSS behaviour:

// before
const left  = Math.max(0,      camX + camS * maskX);
const right = Math.min(stageW, camX + camS * (maskX + maskW));

// after — canvas clips out-of-bounds naturally, same as CSS inset()
x:     camX + camS * maskX,
width: camS * maskW,

The fix applies to all three call-sites (cursor clip, shadow composite, direct composite).

@EtienneLescot
Copy link
Copy Markdown
Owner

@auberginewly thanks for your swift reply ;-)
I let you see code rabbit comments and we should be all good!

@auberginewly
Copy link
Copy Markdown
Author

@auberginewly thanks for your swift reply ;-) I let you see code rabbit comments and we should be all good!

I see!Thanks!this is my first time to write a new feature,and I’ll try my best☺️

… i18n

- Add aria-label to cursorClipToBounds Switch so screen readers announce the control
- Mirror composite3D 3D transform onto nativeCursorClipRef so the cursor clip layer
  rotates with the video during 3D zoom regions (cursor stays outside preserve-3d
  so clip-path continues to work; only the transform string is mirrored)
- Fix vi cursor.motionBlur: "Mờ chuyển động" → "Làm mờ chuyển động" to match
  effects.motionBlur phrasing
- Fix zh-TW cursor.motionBlur: "運動模糊" → "動態模糊" to match effects.motionBlur
@auberginewly
Copy link
Copy Markdown
Author

is that all right?👀

@EtienneLescot
Copy link
Copy Markdown
Owner

@auberginewly yes, this looks good to me now. I checked the latest commit 7cf8791: the accessibility/i18n comments are addressed, and the 3D preview/export consistency fix is minimal and aligned with the existing transform flow. I also replied to the CodeRabbit threads with the relevant commit references. Thanks for the quick iteration!

@EtienneLescot EtienneLescot merged commit 90a0447 into EtienneLescot:feat/macos-native-capture-pipeline May 18, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants