Skip to content

feat(validation): implement incremental validation (#17)#283

Open
ntoulasm wants to merge 8 commits intohyperjump-io:mainfrom
ntoulasm:feat/only-rebuild-dependent-schemas
Open

feat(validation): implement incremental validation (#17)#283
ntoulasm wants to merge 8 commits intohyperjump-io:mainfrom
ntoulasm:feat/only-rebuild-dependent-schemas

Conversation

@ntoulasm
Copy link
Copy Markdown

@ntoulasm ntoulasm commented Apr 21, 2026

Description

Closes #17
This PR implements incremental validation. The server now tracks the dependencies of the workspace and revalidates only the set of schemas affected by a change.

Key Changes

  1. Implemented the Dependencies service, which is responsible for tracking the dependencies of the workspace and computing which schemas are affected by each change.
  2. Updated validate-workspace.js to pass all changes to the Dependencies service and revalidate only the returned schemas instead of all the schemas in the workspace.
  3. Updated build-server.js to instantiate the Dependencies service and pass it to the ValidateWorkspace feature.
  4. Updated existing test expectations to consider the incremental validation.
  5. Added validate-workspace.test.ts and implemented some integration tests to verify the incremental validation functionality.

Implementation Details

The Dependencies service stores a DependencyRecord for each schema that exists in the workspace. The DependencyRecord stores a Set of all its dependencies and a Set of all of its definitions (uri and the provided $ids). The Dependencies service also stores a dependents Map to easily find the dependents of a file when it is edited. Although a DependencyRecord indicates that a schema exists in the workspace, this is not true for the dependents Map because an existing schema may reference a schema that does not exist in the workspace (yet).

When the workspace is changed, the ValidateWorkspaceFeature calls Dependencies.sync and passes the changes. Then the Dependencies service gets updated and the affected schemas are returned to be validated. If the changes are empty, it is considered a full dependencies discovery, so the Dependencies service clears its internal state and starts from scratch returning all the schemas in the workspace. Otherwise it gets updated with the changes and returns only the affected schemas.

To find the affected schemas for a change, first we need to retrieve the DependencyRecord for the changed schema, which is easy because the dependency records are stored in a Map with their uris as keys. From the DependencyRecord we can get all its handles from the definitions Set. Using these handles as indexes to the dependents Map we can get all its direct dependents. The Dependencies service uses this logic recursively, while checking if a node is already visited to avoid infinite loops. Also, the uri of the changed file itself is added to the Set of the affected files. When a change occurs, the affected files actually need to be computed twice, once with the old workspace state and once with the current one. This is necessary because of 2 edge cases:

  1. The introduced changes cause a schema to stop providing a definition. This can happen when a file is deleted so its uri and its $ids no longer exist. Or when it is changed and its provided $ids changed. If someone depended on this schema and the affected schemas were computed with the new workspace image, we would miss that.
  2. The introduced changes cause a schema to provide a definition that was previously missing. This can happen when a file is added so its uri and its $ids become available. Or when it's changed and its provided $ids changed. If a schema depended on the previously missing schema and the affected schemas were computed based on the previous image we would miss that.

The Dependencies service is also incrementally updated:

  1. If a schema is deleted, its DependencyRecord is removed from the records and its uri is removed from the dependents values.
  2. If a schema is added, a DependencyRecord for this schema is created and its uri is added to the dependents values for each of its dependencies.
  3. If a schema is changed, it is considered that the schema was deleted and added.

@ntoulasm ntoulasm marked this pull request as draft April 21, 2026 14:37
@ntoulasm ntoulasm marked this pull request as ready for review April 25, 2026 15:54
@ntoulasm
Copy link
Copy Markdown
Author

Hi @jdesrosiers, this PR is ready for review. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Server - Only rebuild dependent schemas

1 participant