fix: normalize test body `invocationDetails` from stack traces #32699

astone123 · 2025-10-10T19:39:17Z

This PR closes https://github.com/cypress-io/cypress-services/issues/11991

Additional details

There are a few cases where we don't correctly derive the test body invocation details from the stack trace. This PR fixes those scenarios by looking for at eval or at Suite.eval in Chrome stack traces, and lines with anonymous functions in Firefox stack traces. This will allow Cypress Studio and "Open in IDE" features to work in more cases than they did before.

One of these situations is when it has been re-written as itGrep which happens in the @cypress/grep and @bahmutov/cy-grep plugins. This allows us to determine the correct invocation details for the test.

Steps to test

I tested this with the @bahmutov/cy-grep package. Install the package and register it in your support file in accordance with the documentation. Open Cypress Studio by clicking the "edit in studio" button next to a test. Verify that the test content is loaded correctly and you can edit and save the test as expected.

How has the user experience changed?

Users of @bahmutov/cy-grep package can now use Cypress studio as intended

PR Tasks

Have tests been added/updated?
[na] Has a PR for user-facing changes been opened in cypress-documentation?
[na] Have API changes been updated in the type definitions?

Note

Normalizes test-body invocationDetails by trimming stacks (Chrome/Firefox, including wrapped it), updates reporter to send correct IDE targets, adds comprehensive e2e/unit tests, and updates changelog.

Driver:
- stack_utils: add stackTrimmedToTestInvocation and enhance getInvocationDetails(specWindow, root, 'test') to trim to actual test invocation (Chromium: at eval/at Suite.eval; Firefox: anonymous frame). Export InvocationDetails.
- mocha.ts: pass 'test' when setting test.invocationDetails.
App/Reporter:
- E2E verifies "Open in IDE" sends correct file/line/column for before hook, basic test body, and wrapped it.
Tests:
- New e2e stack_utils-invocationDetails.cy.ts validating Chrome/Firefox and wrapped it line/column/stack.
- Unit tests for stack trimming and fallback behavior; add wrapped-it.cy.js fixture.
Changelog:
- Add bugfix noting corrected stack traces used for test invocation details.

^{Written by Cursor Bugbot for commit 50418a8. This will update automatically on new commits. Configure here.}

… invocationDetails

cypress · 2025-10-10T20:22:11Z

cypress Run #67355

Run Properties: Passed #67355 • 50418a845c: only modify stacks for e2e tests

Project	`cypress`
Branch Review	`astone123/fix-itgrep-trace`
Run status	`Passed #67355`
Run duration	`19m 13s`
Commit	`50418a845c: only modify stacks for e2e tests`
Committer	`astone123`
View all properties for this run ↗︎

Test results
Failures	`0`
Flaky	`14`
Pending	`1108`
Skipped	`4`
Passing	`26602`
View all changes introduced in this branch ↗︎

Warning

Partial Report: The results for the Application Quality reports may be incomplete.

UI Coverage `45.48%`
Untested elements	`188`
Tested elements	`161`

Accessibility `97.98%`
Failed rules	`4 critical` `8 serious` `2 moderate` `2 minor`
Failed elements	`101`

astone123 · 2025-10-28T13:07:05Z

@mschile @mabela416
After looking at fixing this on the cloud side for a bit, I think the best way to handle the grep stack issue is with a solution like this on the app side. We use the invocation details in various places across the app to decide which test to focus in studio mode and other things, so normalizing the stack on the cloud side doesn't fix the issue entirely. Let me know what you think

mabela416

I tested it locally and it works

AtofStryker · 2025-10-31T15:51:12Z

packages/driver/src/cypress/stack_utils.ts

      }
    }

+    // if the stack includes the 'itGrep' function, remove any lines that include it


Likely good to reference what itGrep is and where it is coming from (same with above) https://github.com/cypress-io/cypress/blob/develop/npm/grep/src/register.ts#L77. people who aren't familiar with @cypress/grep will struggle to understand what this means.

The other concern I have is that there could be something legitimate in the stack that has itGrep in it that we want to capture but are omitting it here. Or is this only considered when trying to calculate the row/column in the spec file where the error occurred?

Added a comment there. I'm pretty sure this function is specifically used to identify where the test was executed from, so this shouldn't affect anything else. If it does, it'll be limited to use of the grep plugin

astone123 · 2025-11-03T16:17:38Z

@mschile I updated this so that it works for tests with .only as well as tests that are nested in suite definitions

packages/driver/src/cypress/stack_utils.ts

Co-authored-by: Matt Schile <[email protected]>

packages/driver/src/cypress/mocha.ts

packages/driver/src/cypress/stack_utils.ts

packages/app/cypress/e2e/runner/reporter.hooks.cy.ts

cursor · 2025-11-13T14:37:39Z

packages/driver/src/cypress/stack_utils.ts

+          line.trim().startsWith('at eval') ||
+          line.trim().startsWith('at Suite.eval')
+        )
+      })


Bug: Flawed Stack Trimming Breaks Test Debugging

The Chromium stack trimming logic uses startsWith('at eval') and startsWith('at Suite.eval'), which incorrectly matches function names like evalScripts, evaluate, or Suite.evalSomething. This causes the stack to be trimmed at the wrong location, breaking invocation details for tests when such function names appear in the stack trace.

Can probably add a space to at eval and at Suit.eval

good call, added a space to the end of those

mschile · 2025-11-13T20:11:00Z

packages/driver/src/cypress/stack_utils.ts

+    // There are cases where there are other lines in the stack trace before the invocation (eg. `context.it.only`, `createRunnable`, etc)
+    // The actual test invocation line starts with either 'at eval' or 'at Suite.eval',
+    // so remove all lines until we reach the test invocation line


Suggested change

// There are cases where there are other lines in the stack trace before the invocation (eg. `context.it.only`, `createRunnable`, etc)

// The actual test invocation line starts with either 'at eval' or 'at Suite.eval',

// so remove all lines until we reach the test invocation line

// The actual test invocation line starts with either 'at eval' or 'at Suite.eval',

// so remove all lines until we reach the test invocation line

mschile · 2025-11-13T20:15:59Z

packages/driver/src/cypress/stack_utils.ts

+  })
+
+  // if we removed all the lines then something went wrong. return the original stack instead
+  if (modifiedStack.trim() === 'Error') {


This seems brittle, I think in the stackWithLinesRemoved callback above, you can create a copy of the lines and then do your processing on that copy. Before returning lines, you can check if all of the lines were removed and if they were, you can return the original lines, otherwise return the copy.

updated this to create a copy of the lines and check to make sure there are lines in the copy before returning it

mschile · 2025-11-13T20:29:07Z

packages/driver/cypress/e2e/cypress/invocationDetails.cy.ts

I don't think this spec is correct (possibly AI generated nonsense?).

Removed this spec in favor of packages/driver/cypress/e2e/cypress/stack_utils-invocationDetails.cy.ts

mschile · 2025-11-13T20:31:05Z

packages/driver/cypress/e2e/cypress/invocationDetails.cy.ts

We also have an existing stack_utils.cy.js file that we can add to for any getInvocationDetails tests.

Ended up making a separate spec for these instead of adding them to stack_utils.cy.js

cursor · 2025-11-14T20:08:06Z

packages/driver/src/cypress/stack_utils.ts

+    // if the hook is the test, and it's an e2e test,, we will try to remove the lines that are not the actual invocation of the test
+    if (type === 'test' && specWindow.Cypress.testingType !== 'component') {
+      stack = stackTrimmedToTestInvocation(stack, specWindow)
+    }


Bug: Prevent errors from missing Cypress object.

Accessing specWindow.Cypress.testingType without checking if specWindow.Cypress exists will throw an error when users quickly reload tests, since specWindow.Cypress can be null or undefined in those cases. The check should be inside the existing if (specWindow.Cypress) block or include its own null check.

fix: (studio) remove itGrep lines from stack trace when determining…

3bb1471

… invocationDetails

astone123 self-assigned this Oct 10, 2025

astone123 requested review from AtofStryker and mschile October 10, 2025 19:40

astone123 closed this Oct 10, 2025

astone123 reopened this Oct 28, 2025

Merge branch 'develop' into astone123/fix-itgrep-trace

b6bc0f5

astone123 requested a review from mabela416 October 28, 2025 13:07

mabela416 approved these changes Oct 30, 2025

View reviewed changes

AtofStryker reviewed Oct 31, 2025

View reviewed changes

astone123 added 3 commits November 3, 2025 11:12

handle .only and suites

926e1fc

Merge branch 'develop' into astone123/fix-itgrep-trace

49165cf

Merge branch 'develop' into astone123/fix-itgrep-trace

61311ee

cursor bot reviewed Nov 3, 2025

View reviewed changes

packages/driver/src/cypress/stack_utils.ts Outdated Show resolved Hide resolved

Merge branch 'develop' into astone123/fix-itgrep-trace

0a8de00

cursor bot reviewed Nov 6, 2025

View reviewed changes

packages/driver/src/cypress/stack_utils.ts Outdated Show resolved Hide resolved

for test block hooks, trim the stack to find the invocation details

a1480d8

astone123 changed the title ~~fix: (studio) remove itGrep lines from stack trace when determining invocationDetails~~ fix: normalize test body invocationDetails from stack traces Nov 6, 2025

astone123 added 2 commits November 6, 2025 17:27

Merge branch 'develop' into astone123/fix-itgrep-trace

36064a6

fix stop only

65d656c

cursor bot reviewed Nov 6, 2025

View reviewed changes

packages/driver/src/cypress/stack_utils.ts Show resolved Hide resolved

astone123 added 2 commits November 7, 2025 09:42

fix cypress object access

c0fb620

Merge branch 'develop' into astone123/fix-itgrep-trace

96a9a10

cursor bot reviewed Nov 7, 2025

View reviewed changes

packages/driver/src/cypress/stack_utils.ts Outdated Show resolved Hide resolved

astone123 requested a review from AtofStryker November 7, 2025 15:57

AtofStryker approved these changes Nov 7, 2025

View reviewed changes

packages/driver/src/cypress/stack_utils.ts Outdated Show resolved Hide resolved

update comment to use generic host

d0dfb88

astone123 and others added 5 commits November 12, 2025 14:46

Update packages/driver/src/cypress/stack_utils.ts

a7e5bb2

Co-authored-by: Matt Schile <[email protected]>

Update packages/driver/src/cypress/stack_utils.ts

7bb0f76

Co-authored-by: Matt Schile <[email protected]>

Update packages/driver/src/cypress/mocha.ts

42d6495

Co-authored-by: Matt Schile <[email protected]>

Update packages/driver/src/cypress/stack_utils.ts

1929cba

Co-authored-by: Matt Schile <[email protected]>

Update packages/driver/src/cypress/stack_utils.ts

ec9dad3

Co-authored-by: Matt Schile <[email protected]>

cursor bot reviewed Nov 12, 2025

View reviewed changes

packages/driver/src/cypress/mocha.ts Show resolved Hide resolved

astone123 added 2 commits November 12, 2025 15:16

feedback

85756f0

Merge branch 'develop' into astone123/fix-itgrep-trace

1bc1d68

cursor bot reviewed Nov 12, 2025

View reviewed changes

packages/driver/src/cypress/stack_utils.ts Show resolved Hide resolved

astone123 added 3 commits November 12, 2025 15:25

changelog entry

4a53ed1

remove unnecessary test, add driver integration tests

cde2e2c

fix .only

badf190

cursor bot reviewed Nov 13, 2025

View reviewed changes

packages/app/cypress/e2e/runner/reporter.hooks.cy.ts Outdated Show resolved Hide resolved

astone123 added 2 commits November 13, 2025 09:34

update test name

d7f64f0

Merge branch 'develop' into astone123/fix-itgrep-trace

76e2319

cursor bot reviewed Nov 13, 2025

View reviewed changes

fix types

31af0fb

astone123 requested a review from mschile November 13, 2025 16:22

mschile reviewed Nov 13, 2025

View reviewed changes

astone123 added 5 commits November 14, 2025 10:33

add integration test

dbe8c6f

logic and test updates

e9d72cf

Merge branch 'develop' into astone123/fix-itgrep-trace

4df6616

fix types

1dfe50c

fix test

6fa202f

astone123 requested a review from mschile November 14, 2025 20:02

only modify stacks for e2e tests

50418a8

cursor bot reviewed Nov 14, 2025

View reviewed changes

fix: normalize test body invocationDetails from stack traces #32699

Are you sure you want to change the base?

fix: normalize test body invocationDetails from stack traces #32699

Conversation

astone123 commented Oct 10, 2025 • edited by cursor bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Additional details

Steps to test

How has the user experience changed?

PR Tasks

Uh oh!

cypress bot commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

astone123 commented Oct 28, 2025

Uh oh!

mabela416 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

astone123 commented Nov 3, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor bot Nov 13, 2025

Choose a reason for hiding this comment

Bug: Flawed Stack Trimming Breaks Test Debugging

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cursor bot Nov 14, 2025

Choose a reason for hiding this comment

Bug: Prevent errors from missing Cypress object.

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

fix: normalize test body `invocationDetails` from stack traces #32699

fix: normalize test body `invocationDetails` from stack traces #32699

astone123 commented Oct 10, 2025 •

edited by cursor bot

Loading

cypress bot commented Oct 10, 2025 •

edited

Loading