Skip to content

Use pytest+jupytext for notebook testing#1838

Open
mpharrigan wants to merge 8 commits intoquantumlib:mainfrom
mpharrigan:2026-04/jupytext
Open

Use pytest+jupytext for notebook testing#1838
mpharrigan wants to merge 8 commits intoquantumlib:mainfrom
mpharrigan:2026-04/jupytext

Conversation

@mpharrigan
Copy link
Copy Markdown
Collaborator

Replace the kernel-based notebook execution in execute_notebook() with jupytext-based in-memory conversion and exec(). This avoids parallel kernel startup overhead and eliminates flaky kernel failures

Changes:

  • qualtran/testing.py: Rewrite execute_notebook() to use jupytext to convert notebooks to Python scripts in memory, then execute them in a temporary directory.
  • dev_tools/check-notebook-tests.py: New CI script that verifies every committed .ipynb (in qualtran/ and tutorials/, excluding dev_tools/) has a corresponding @pytest.mark.notebook test calling execute_notebook(). Prints actionable error messages on failure.
  • check/pytest-notebook: New check script that runs the coverage check then pytest -m notebook with MPLBACKEND=agg to prevent interactive matplotlib windows.
  • .github/workflows/pr.yaml: The notebooks CI job now runs check/pytest-notebook instead of the old execute-notebooks.py script. The docs build workflow (execute-notebooks.py, notebook_execution.py) is unchanged.
  • pyproject.toml: Add jupytext to test dependencies.
  • Added @pytest.mark.notebook tests for 61 notebooks that were missing them, plus 3 tutorial notebooks in tutorials/quantum-programming/.

@review-notebook-app
Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request replaces the kernel-based Jupyter notebook testing with a faster in-process execution method using jupytext. It adds a verification script to ensure all committed notebooks have corresponding tests and updates numerous test files to include notebook execution checks. Feedback highlights the need for recursive path matching in the notebook discovery script and suggests improvements to the execute_notebook function to handle sys.modules isolation, working directory consistency, and IPython magics.

mpharrigan and others added 4 commits April 3, 2026 16:39
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@mpharrigan mpharrigan added the devops Involves build systems, Make files, Bazel files, continuous integration, and or other DevOps topics label Apr 3, 2026
@mpharrigan
Copy link
Copy Markdown
Collaborator Author

xref #1745
xref #1807

@mpharrigan mpharrigan mentioned this pull request Apr 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

devops Involves build systems, Make files, Bazel files, continuous integration, and or other DevOps topics

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant