Repository navigation
test(e2e): add insomnia-test Playwright framework, CI workflow and skill [1/6] - #10594
bobqiu-kong wants to merge 18 commits into
Conversation
|
✅ KSAI Review: Finished
Run report (federated) · 1 paid run · $0.8311 total
Reviewing cost $0.8305; deciding how to run it cost $0.0006 |
There was a problem hiding this comment.
| Check | Result |
|---|---|
| Scope | New packages/insomnia-test Playwright E2E suite (flows, pages, models, mock servers), insomnia-test CI workflow, eslint/package.json wiring, plus small insomnia-data and electron-builder.config.js edits. |
| Mandate | typescript-code-reviewer |
| Findings | 1 High, 1 Medium, 5 Low |
| Findings audit | Ran kreview:findings-auditor. All 7 findings upheld; 2 line locations corrected against the files on disk. |
The E2E framework itself is solid, but it cannot run: echo-server reads a fixture file this PR never commits, so the webServer never starts and every Playwright run aborts. Most flows also target mock servers absent from the repo, and the CI comment about build failures not blocking the PR is false. Fix these before merging.
Additional Risk pass: no hardcoded secrets found; mock keys are generated per process. Unauthenticated _admin mock endpoints listen on all interfaces, and per-test data directories are never deleted. Both reported as findings.
This review read b94e532. The pull request has moved to 86da056 since, so a comment on a line changed after that commit shows as outdated.
32503d2 to
386cee6
Compare
There was a problem hiding this comment.
🟡 Changes recommended
The framework currently has type-check, dependency-lock, lifecycle-cleanup, listener, connection, error-handling, and workflow-rollout defects.
7 open findings
Rename conflicting authentication helper and delegate non-OAuth variants · New Regenerate lockfile to match pinned Playwright version · New Install database change listener only once per window · New Handle gRPC error dialogs while waiting for response · New Align build failure behavior with documented rollout settings · New Track relaunched app and window handles for fixture cleanup · New Wait for connection state after clicking connect · New
What changed in this PR
Introduces a layered Playwright/Electron E2E framework, supporting mock services and diagnostics, plus sharded GitHub Actions execution.
Changes:
- Adds reusable models, pages, flows, fixtures, mock servers, and an HTTP smoke test.
- Adds build caching, test sharding, coverage merging, and report publication.
- Adds test-package configuration, documentation, and source-map packaging support.
| File | Description |
|---|---|
.claude/skills/write-insomnia-test/SKILL.md |
Defines E2E authoring skill. |
.claude/skills/write-insomnia-test/reference.md |
Documents framework APIs. |
.github/workflows/insomnia-test.yml |
Adds sharded E2E workflow. |
.gitignore |
Ignores generated test artifacts. |
eslint.config.mjs |
Configures test-package linting. |
package-lock.json |
Locks new workspace dependencies. |
package.json |
Registers the test workspace. |
packages/insomnia-data/package.json |
Adds CommonJS export conditions. |
packages/insomnia-data/src/models/types.ts |
Exports gRPC credential type. |
packages/insomnia-test/enums/api-spec-export-format.ts |
Defines API export formats. |
packages/insomnia-test/enums/auth-type.ts |
Defines authentication types. |
packages/insomnia-test/enums/collaborator-role.ts |
Defines collaborator roles. |
packages/insomnia-test/enums/content-type.ts |
Defines request content types. |
packages/insomnia-test/enums/context-menu-items.ts |
Defines context-menu actions. |
packages/insomnia-test/enums/delete-mode.ts |
Defines deletion modes. |
packages/insomnia-test/enums/export-format.ts |
Defines export formats. |
packages/insomnia-test/enums/http-method.ts |
Defines HTTP methods. |
packages/insomnia-test/enums/import-sources.ts |
Defines import sources. |
packages/insomnia-test/enums/lint-severity.ts |
Defines lint severities. |
packages/insomnia-test/enums/mock-body-type.ts |
Defines mock body types. |
packages/insomnia-test/enums/project-types.ts |
Defines project types. |
packages/insomnia-test/enums/request-state.ts |
Defines request states. |
packages/insomnia-test/enums/ruleset-type.ts |
Defines ruleset types. |
packages/insomnia-test/enums/script-sandbox-rule-group.ts |
Defines sandbox rule groups. |
packages/insomnia-test/enums/script-tab.ts |
Defines script tabs. |
packages/insomnia-test/enums/spec-format.ts |
Defines specification formats. |
packages/insomnia-test/enums/sync-status.ts |
Defines sync states. |
packages/insomnia-test/enums/tree-node-types.ts |
Defines workspace node types. |
packages/insomnia-test/flows/app.flow.ts |
Orchestrates app lifecycle. |
packages/insomnia-test/flows/base.flow.ts |
Provides shared flow behavior. |
packages/insomnia-test/flows/certificates.flow.ts |
Orchestrates certificate actions. |
packages/insomnia-test/flows/cloud-sync.flow.ts |
Orchestrates cloud synchronization. |
packages/insomnia-test/flows/cookie.flow.ts |
Orchestrates cookie actions. |
packages/insomnia-test/flows/environment.flow.ts |
Orchestrates environment actions. |
packages/insomnia-test/flows/event-stream-request.flow.ts |
Orchestrates SSE requests. |
packages/insomnia-test/flows/export.flow.ts |
Orchestrates data exports. |
packages/insomnia-test/flows/flow-context.ts |
Defines shared flow context. |
packages/insomnia-test/flows/flow-manager.ts |
Registers flow instances. |
packages/insomnia-test/flows/folder.flow.ts |
Orchestrates folder actions. |
packages/insomnia-test/flows/git-sync.flow.ts |
Orchestrates Git synchronization. |
packages/insomnia-test/flows/graphql-request.flow.ts |
Orchestrates GraphQL requests. |
packages/insomnia-test/flows/grpc-request.flow.ts |
Orchestrates gRPC requests. |
packages/insomnia-test/flows/http-request.flow.ts |
Orchestrates HTTP requests. |
packages/insomnia-test/flows/import.flow.ts |
Orchestrates data imports. |
packages/insomnia-test/flows/mcp-client.flow.ts |
Orchestrates MCP clients. |
packages/insomnia-test/flows/mock-server.flow.ts |
Orchestrates mock servers. |
packages/insomnia-test/flows/organization.flow.ts |
Orchestrates organization actions. |
packages/insomnia-test/flows/preferences.flow.ts |
Orchestrates preference changes. |
packages/insomnia-test/flows/socket-io-request.flow.ts |
Orchestrates Socket.IO requests. |
packages/insomnia-test/flows/template-tag.flow.ts |
Orchestrates template tags. |
packages/insomnia-test/flows/web-socket-request.flow.ts |
Orchestrates WebSocket requests. |
packages/insomnia-test/flows/workspace.flow.ts |
Orchestrates workspace actions. |
packages/insomnia-test/misc/certs.js |
Generates TLS test certificates. |
packages/insomnia-test/misc/constants.ts |
Defines shared test constants. |
packages/insomnia-test/misc/coverage-global-setup.ts |
Initializes coverage collection. |
packages/insomnia-test/misc/coverage-global-teardown.ts |
Finalizes coverage collection. |
packages/insomnia-test/misc/coverage.ts |
Collects V8 coverage. |
packages/insomnia-test/misc/decorators.ts |
Adds dialog-error handling. |
packages/insomnia-test/misc/echo-server.js |
Provides HTTP echo service. |
packages/insomnia-test/misc/event-stream-server.js |
Provides SSE test service. |
packages/insomnia-test/misc/export-file.ts |
Supports export assertions. |
packages/insomnia-test/misc/fixtures.ts |
Defines Electron test fixtures. |
packages/insomnia-test/misc/fixtures/petstore-swagger.json |
Supplies OpenAPI fixture data. |
packages/insomnia-test/misc/fixtures/upload-sample.txt |
Supplies upload fixture data. |
packages/insomnia-test/misc/git-fixtures.ts |
Defines Git test fixtures. |
packages/insomnia-test/misc/git-server.js |
Provides Git HTTP service. |
packages/insomnia-test/misc/grpc-server.js |
Provides gRPC test service. |
packages/insomnia-test/misc/launch.ts |
Launches tested Electron builds. |
packages/insomnia-test/misc/mcp-server.js |
Provides MCP test service. |
packages/insomnia-test/misc/merge-coverage-union.js |
Unions shard coverage. |
packages/insomnia-test/misc/merge-coverage.js |
Generates merged coverage reports. |
packages/insomnia-test/misc/mock-api.js |
Mocks Insomnia APIs. |
packages/insomnia-test/misc/mock-bin-server.js |
Provides mock-bin service. |
packages/insomnia-test/misc/oauth2-server.js |
Provides OAuth2 service. |
packages/insomnia-test/misc/protos/hello.proto |
Defines test gRPC service. |
packages/insomnia-test/misc/socket-server.js |
Provides Socket.IO service. |
packages/insomnia-test/misc/step-instrumentation.ts |
Instruments page and flow steps. |
packages/insomnia-test/misc/ws-echo-server.js |
Provides WebSocket echo service. |
packages/insomnia-test/models/certificate.ts |
Models certificates. |
packages/insomnia-test/models/collection.ts |
Models collections. |
packages/insomnia-test/models/commit.ts |
Models Git commits. |
packages/insomnia-test/models/cookie.ts |
Models cookies. |
packages/insomnia-test/models/curl-command.ts |
Models cURL commands. |
packages/insomnia-test/models/environment.ts |
Models environments. |
packages/insomnia-test/models/event-stream-request.ts |
Models SSE requests. |
packages/insomnia-test/models/folder.ts |
Models folders. |
packages/insomnia-test/models/git-credential.ts |
Models Git credentials. |
packages/insomnia-test/models/git-repo-connection.ts |
Models Git connections. |
packages/insomnia-test/models/graphql-request.ts |
Models GraphQL requests. |
packages/insomnia-test/models/grpc-request.ts |
Models gRPC requests. |
packages/insomnia-test/models/http-request.ts |
Models HTTP requests. |
packages/insomnia-test/models/mcp-client.ts |
Models MCP clients. |
packages/insomnia-test/models/mock-route.ts |
Models mock routes. |
packages/insomnia-test/models/mock-server.ts |
Models mock servers. |
packages/insomnia-test/models/project.ts |
Models projects. |
packages/insomnia-test/models/response.ts |
Models response data. |
packages/insomnia-test/models/runner.ts |
Models runner results. |
packages/insomnia-test/models/settings.ts |
Models application settings. |
packages/insomnia-test/models/socket-io-request.ts |
Models Socket.IO requests. |
packages/insomnia-test/models/template-tag.ts |
Models template tags. |
packages/insomnia-test/models/websocket-request.ts |
Models WebSocket requests. |
packages/insomnia-test/package.json |
Configures the test package. |
packages/insomnia-test/pages/auth-tab.page.ts |
Automates authentication fields. |
packages/insomnia-test/pages/base.page.ts |
Provides shared page behavior. |
packages/insomnia-test/pages/certificates.page.ts |
Automates certificate settings. |
packages/insomnia-test/pages/cloud-sync.page.ts |
Automates cloud sync. |
packages/insomnia-test/pages/command-palette.page.ts |
Automates command palette actions. |
packages/insomnia-test/pages/cookie.page.ts |
Automates cookie management. |
packages/insomnia-test/pages/environment.page.ts |
Automates environment editing. |
packages/insomnia-test/pages/event-stream-request.page.ts |
Automates SSE request UI. |
packages/insomnia-test/pages/export.page.ts |
Automates export UI. |
packages/insomnia-test/pages/folder.page.ts |
Automates folder UI. |
packages/insomnia-test/pages/git-sync.page.ts |
Automates Git sync UI. |
packages/insomnia-test/pages/graphql-request.page.ts |
Automates GraphQL request UI. |
packages/insomnia-test/pages/grpc-request.page.ts |
Automates gRPC request UI. |
packages/insomnia-test/pages/http-request.page.ts |
Automates HTTP request UI. |
packages/insomnia-test/pages/import.page.ts |
Automates import UI. |
packages/insomnia-test/pages/invite.page.ts |
Automates collaborator invites. |
packages/insomnia-test/pages/konnect.page.ts |
Automates Konnect UI. |
packages/insomnia-test/pages/mcp-client.page.ts |
Automates MCP client UI. |
packages/insomnia-test/pages/mock-server.page.ts |
Automates mock-server UI. |
packages/insomnia-test/pages/page-manager.ts |
Registers page objects. |
packages/insomnia-test/pages/preferences.page.ts |
Automates preferences UI. |
packages/insomnia-test/pages/project-settings.page.ts |
Automates project settings. |
packages/insomnia-test/pages/request.page.ts |
Provides shared request actions. |
packages/insomnia-test/pages/response.page.ts |
Reads response state. |
packages/insomnia-test/pages/runner.page.ts |
Automates collection runner UI. |
packages/insomnia-test/pages/script-tab.page.ts |
Automates script editors. |
packages/insomnia-test/pages/socket-io-request.page.ts |
Automates Socket.IO UI. |
packages/insomnia-test/pages/tab-panel.page.ts |
Provides shared tab behavior. |
packages/insomnia-test/pages/template-tag.page.ts |
Automates template-tag UI. |
packages/insomnia-test/pages/web-socket-request.page.ts |
Automates WebSocket UI. |
packages/insomnia-test/pages/workspace.page.ts |
Automates workspace navigation. |
packages/insomnia-test/playwright.config.ts |
Configures Playwright and services. |
packages/insomnia-test/tests/http-request/happy-path.spec.ts |
Adds end-to-end HTTP smoke coverage. |
packages/insomnia/electron-builder.config.js |
Retains source maps for coverage. |
🧠 Review effort: Balanced
e0cb034 to
5c7ea23
Compare
2c352ce to
ccad0e3
Compare
…ite-insomnia-test skill Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… at startup misc/echo-server.js reads misc/fixtures/petstore-swagger.json when it starts, so the fixture must land together with the server; otherwise every shard's webServer exits with ENOENT. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- bind mock-api and echo-server to 127.0.0.1 - remove per-test dataPath after use - throw when setCodeMirrorValue does not hold after retries - drop continue-on-error from the build job - shorten cookie waitForPersisted docstring Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Adds the event-stream, socket.io, ws-echo, mcp, grpc, git, oauth2 and mock-bin servers (bound to 127.0.0.1), their proto/upload fixtures, npm scripts and Playwright webServer entries, so flows that reference them no longer fail with connection refused. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… nosniff Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…response Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
WorkspaceFlow now takes a structural RequestFlows dependency instead of importing FlowManager, which removes the 3 new circular references. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The type represents the response pane (response data plus console, stream events and script test results), not the network response, and clashed with the insomnia-data Response name. Update flows, pages, specs, JSDoc and the write-insomnia-test skill accordingly. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ones WorkspaceFlow.create/duplicate/rename, ExportFlow.export and EnvironmentFlow.create dispatched on argument types via typeof/instanceof and casts, so a call site did not show which flow it triggered. Replace them with explicitly named methods (createProject, openLocalGitFolder, cloneGitProject, createCollection, duplicateRequest, renameCollection, exportProject, createBaseEnvironment, ...) and update the write-insomnia-test skill docs. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Drop the pass-through re-exports of expect from misc/fixtures.ts and misc/git-fixtures.ts, import it from @playwright/test in specs, and update the write-insomnia-test skill accordingly. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…onfig for insomnia-test
ccad0e3 to
c4c142e
Compare


Summary
Introduces
packages/insomnia-test, a Playwright + Electron E2E framework with a strict layered architecture, plus the shardedInsomnia TestGitHub Actions workflow and thewrite-insomnia-testClaude skill. Ships with a single smoke spec (http-request/happy-path) proving the stack end to end; all other specs follow in the stacked PRs below.insomnia-smoke-testis not modified.Stack
Part of a stacked E2E suite. Merge in order:
fix/project-modal-close-on-redirect— app fix (independent, based ondevelop)test/e2e-1-framework— framework + CI + skilltest/e2e-2-http-cookie-env— HTTP, cookie, environment, template-tagtest/e2e-3-protocols— GraphQL, gRPC, WebSocket, Socket.IO, SSE, MCPtest/e2e-4-workspace— workspace, document, runner, mock-server, app, preferencestest/e2e-5-sync— git-sync, cloud-sync, organization, konnecttest/e2e-6-data— data, migration, pluginWhy
The existing smoke-test package is hard to scale: specs are coupled to selectors, helpers are ad hoc, and CI runs are slow and hard to debug. This builds a replacement foundation that is easier to write, review and scale horizontally. The workflow is not yet a required check, so it can stabilise alongside the existing suite before any cut-over.
What's changed
1. Layered package
packages/insomnia-testtests/flows/flow-manager.tspages/page-manager.tsmodels/,enums/misc/throwOnDialogdecorator, TLS cert generation, base mock serversAll flows/pages/models land here (not split per feature) because they are mutually dependent — e.g.
WorkspaceFlowcreates requests of every protocol — so they can't be separated without breaking type-checks. The follow-up PRs are specs + their mock servers only.Mock servers in this PR: Insomnia mock API (
:4010) and HTTP echo (:4060, which loadsmisc/fixtures/petstore-swagger.jsonat startup, so that fixture lives here too). Others are added by the PRs that need them.Launch modes (
misc/fixtures.ts,playwright.config.ts): dev mode (default locally) launches Electron frompackages/insomniaagainst the Vite dev server (:3334); settingINSOMNIA_BINARY(CI always does) tests the packaged build;INSOMNIA_DEV_MODE=falseuses the installed app.Coverage & diagnostics: opt-in V8 coverage (
COVERAGE=true) viamonocart-coverage-reportswith per-shard raw data and merge scripts; trace/screenshot/video retained on failure.2. CI:
.github/workflows/insomnia-test.ymlRuns on pushes to
developand PRs, with per-ref concurrency cancellation.build— builds the Linux unpacked binary once, cached by commit SHA. A build failure fails the run (shards need the cached binary).parallel-test— sharded matrix, each shard restores the cached binary and runs underxvfb(retries: 2on CI); uploads raw coverage, HTML and blob reports.insomnia-test-result— merges coverage and blob reports intocoverage-report/playwright-report; every step iscontinue-on-error.3. Supporting changes
packages/insomnia-data: addrequireconditions toexports(CommonJS test package imports it); exportGCPCredentialtype.packages/insomnia/electron-builder.config.js: keep.mapfiles whenKEEP_SOURCEMAPS=trueso coverage maps to source.package.json/package-lock.json: register the workspace.eslint.config.mjs: Playwright rules for the package, Node globals for standalone CJS mock servers..gitignore: ignore reports/generated fixtures..claude/skills/write-insomnia-test/: skill for writing new specs from plain-language scenarios using the layers above.Test plan
npx playwright test --listinpackages/insomnia-testlists the smoke spec (verified locally)Insomnia Testworkflow:buildsucceeds and the shard runninghappy-pathpassesInsomnia Test Resultpublishes merged report artifactsNotes for reviewers
misc/fixtures.ts+launch.ts→flows/base.flow.ts/pages/base.page.ts→ CI workflow.insomnia-smoke-testis a follow-up.