Adds axom module to python interface with axom.sidre by kennyweiss · Pull Request #1896 · llnl/axom

kennyweiss · 2026-06-26T03:54:17Z

Summary

This PR improves how our python bindings are installed and exposed
It provides Sidre's Python module under axom.sidre instead of pysidre
It adds the AXOM_PYTHON_MODULE_INSTALL_PREFIX CMake variable for controlling where Axom's python modules get installed, and updates spack to have axom extend python for improved access within spack environments.
- The run_axom_with_sidre script now allows relative paths to the Axom module in support of this.
It also paves the way to being able to install axom's python modules through pip/uv wheels
And, it improves the user docs about how to set up and use Axom's python bindings

kennyweiss · 2026-06-26T03:57:34Z

Question for reviewers -- especially @bmhan12 -- do you think we need this pysidre shim?
I'm not aware of anyone that's currently using this, and the minimal fix in a script that uses this is very straightforward:

- import pysidre + import axom.sidre as pysidre

do you think we need this pysidre shim?

I think we can get rid of the shim.
Thinking of packages like matplotlib, where it's standard practice to just do: import matplotlib.pyplot as plt as the recommended path (https://matplotlib.org/stable/tutorials/pyplot.html).

kennyweiss · 2026-06-26T03:59:55Z


 if(NANOBIND_FOUND)
-    nanobind_add_module(pysidre nanobind_sidre.cpp)
+    # Python bindings for Sidre.


I think most of this logic belongs in the parent directory's CMakeLists.txt,
but I held off on moving it up for now since we only currently have one python submodule (axom.sidre)
Once we have another one, we'll likely have a better sense of what the general pattern should be.

On the other hand, I'm not against refactoring it in this PR.

Leave it for now and refactor when we have another Python module to avoid churn?

kennyweiss · 2026-06-26T04:02:12Z


-    # gen python helper to build directory
-    set(_PYEXT_DIR ${PROJECT_BINARY_DIR}/lib)
+    # gen python helper to build directory.


Similarly here -- the installation logic for Axom's python modules probably doesn't belong in src/tools/CMakeLists.txt

kennyweiss · 2026-06-26T04:03:49Z

 # SPDX-License-Identifier: (BSD-3-Clause)

-import pysidre
+import axom.sidre as pysidre


To minimize the changes in this PR, I only applied

- import pysidre + import axom.sidre as pysidre

Should we use the full axom.sidre or perhaps something else instead of pysidre in our examples/tests for sidre's python interface?

Is there an issue with using pysidre?

The only thing I can think of is that this PR is changing the (sub)module name to axom.sidre but our tests are still using pysidre --

I don't think it's an actual issue, but was wondering if we might want the python to just call it sidre,

bmhan12 · 2026-06-26T16:46:30Z

+        if spec.satisfies("+python"):
+            # Install Axom's Python package(s) so a spack environment view merges them into
+            # a single site-packages and `import axom.sidre` works without updating PYTHONPATH
+            entries.append(
+                cmake_cache_path(
+                    "AXOM_PYTHON_MODULE_INSTALL_PREFIX",
+                    spec["python"].package.platlib,
+                )
+            )


Does this PR require updating the host-configs?

Good call -- ~~I'll do that.~~
Edit: On second thought, I think I'll add this in the follow-up PR (unless you feel strongly about it for this one).

It won't cause our CI to fail since it's related to how others import axom.sidre.
I think we should add an installation test for this, but will save it for a follow-up PR.

I created an issue for this: #1900

bmhan12 · 2026-06-26T16:57:08Z

   parallel_io_concepts
   sidre_conduit
   mfem_sidre_datacollection
+   python_interface


Link to relevant ReadTheDocs for this PR:

https://axom.readthedocs.io/en/feature-kweiss-improve-python-sidre/axom/sidre/docs/sphinx/python_interface.html

https://axom.readthedocs.io/en/feature-kweiss-improve-python-sidre/docs/sphinx/dev_guide/component_org.html#python-interfaces

bmhan12 · 2026-06-26T17:12:33Z

Interesting:
https://safjan.com/the-importance-of-adding-py-typed-file-to-your-typed-package/

* Renames the extension module from `pysidre` to `_sidre` so can import `axom.sidre` * Adds python package scaffolding to `src/python` * Introduces `AXOM_PYTHON_MODULE_INSTALL_PREFIX` CMake cache variable * Update pysidre usage to `import axom.sidre as pysidre` and adds a shim to warning users that importing `pysidre` is deprecated.

This allows spack environment views to place Axom's installed Python packages onto the interpreter path automatically. This commit also emits `AXOM_PYTHON_MODULE_INSTALL_PREFIX` in the generated host-config.

One can call the macro with either a COMMAND or a SOURCE file.

…ack env

Misc: Fixes typos and whitespace issues

…he DataStore

This allows type checkers to see the sidre Python API

The tests are in our github-ci tests as well as for spack-based installations

This will allow the types from all of the Axom python modules to better interoperate.

kennyweiss · 2026-07-15T21:09:21Z

            example()
            make("clean")
+
+    @run_after("install", when="+examples+python+tools components=sidre")


I tested this with:

./scripts/uberenv/uberenv.py --install --run_tests --spec "+python+devtools+hdf5+mfem+c2c+adiak+caliper+opencascade %clang_19"

Here's the relevant part of the installation test:

==> [2026-07-02-16:02:24.101027] '.../llvm-19.1.3/axom-develop-adk5fhlf3iangdg5wkn4kmd4mpsn7bvi/bin/run_python_with_axom.sh' '.../llvm-19.1.3/axom-develop-adk5fhlf3iangdg5wkn4kmd4mpsn7bvi/examples/axom/using-with-python/example.py' Using installed axom.sidre v0.14.0 ==> [2026-07-02-16:02:24.464562] '.../python-3.13.11/bin/python3' '-c' 'import axom.sidre as s; import numpy; print('"'"'axom.sidre'"'"', s.__version__)' axom.sidre v0.14.0

Co-authored-by: Chris White <white238@llnl.gov>

… in CI

kennyweiss · 2026-07-16T02:08:53Z

+    NUMPROCS=$(python3 -c 'import os; print(os.cpu_count())')
+    NUM_BUILD_PROCS=$(python3 -c 'import os; print(max(2, os.cpu_count() * 8 // 10))')

    echo "~~~~~~ RUNNING CMAKE ~~~~~~~~"
    or_die python3 ./config-build.py -bp builddir -hc ./host-configs/docker/${HOST_CONFIG} -bt ${BUILD_TYPE} -DENABLE_GTEST_DEATH_TESTS=ON ${CMAKE_EXTRA_FLAGS}
    or_die cd builddir

    echo "~~~~~~ BUILDING ~~~~~~~~"
    if [[ ${CMAKE_EXTRA_FLAGS} == *COVERAGE* ]] ; then
-        or_die make -j $NUM_BUILD_PROCS
+        or_die make -j ${NUM_BUILD_PROCS}
    else
-        or_die make -j $NUM_BUILD_PROCS VERBOSE=1
+        or_die make -j ${NUM_BUILD_PROCS} VERBOSE=1
    fi

    echo "~~~~~~ RUNNING TESTS ~~~~~~~~"
-    make CTEST_OUTPUT_ON_FAILURE=1 test ARGS='-T Test --output-on-failure -j$NUM_BUILD_PROCS'
+    or_die make CTEST_OUTPUT_ON_FAILURE=1 test ARGS="-T Test --output-on-failure -j${NUM_BUILD_PROCS}"


We weren't passing in -j flag properly:

2026-07-16T00:56:04.1126650Z ~~~~~~ FIND NUMPROCS ~~~~~~~~ 2026-07-16T00:56:04.1127335Z + echo '~~~~~~~~~~~~~~~~~~~~~~' 2026-07-16T00:56:04.1127649Z + export BUILD_TYPE=Debug 2026-07-16T00:56:04.1127886Z + BUILD_TYPE=Debug 2026-07-16T00:56:04.1128090Z + [[ yes == \y\e\s ]] 2026-07-16T00:56:04.1128312Z + echo '~~~~~~ FIND NUMPROCS ~~~~~~~~' 2026-07-16T00:56:04.1128658Z ++ python3 -c 'import os; print(f'\''{os.cpu_count()}'\'')' 2026-07-16T00:56:04.1254132Z + NUMPROCS=4 2026-07-16T00:56:04.1256781Z ++ python3 -c 'import os; print(f'\''{max(2, os.cpu_count() * 8 // 10)}'\'')' 2026-07-16T00:56:04.1373763Z + NUM_BUILD_PROCS=3 ... 2026-07-16T00:56:16.6263485Z + echo '~~~~~~ BUILDING ~~~~~~~~' 2026-07-16T00:56:16.6264412Z + [[ -DBUILD_SHARED_LIBS=ON -DBLT_CXX_STD=c++20 -DENABLE_BENCHMARKS:BOOL=ON -DAXOM_QUEST_ENABLE_EXTRA_REGRESSION_TESTS:BOOL=OFF == *COVERAGE* ]] 2026-07-16T00:56:16.6265219Z ~~~~~~ BUILDING ~~~~~~~~ 2026-07-16T00:56:16.6265519Z + or_die make -j 3 VERBOSE=1 2026-07-16T00:56:16.6280465Z + make -j 3 VERBOSE=1 ... 2026-07-16T01:38:02.1051791Z ~~~~~~ RUNNING TESTS ~~~~~~~~ 2026-07-16T01:38:02.1052665Z + [[ 0 != 0 ]] 2026-07-16T01:38:02.1053239Z + echo '~~~~~~ RUNNING TESTS ~~~~~~~~' 2026-07-16T01:38:02.1054316Z + or_die make CTEST_OUTPUT_ON_FAILURE=1 test 'ARGS=-T Test --output-on-failure -j$NUM_BUILD_PROCS' 2026-07-16T01:38:02.1055900Z + make CTEST_OUTPUT_ON_FAILURE=1 test 'ARGS=-T Test --output-on-failure -j$NUM_BUILD_PROCS'

(the last line should end with -j3)

kennyweiss requested review from Arlie-Capps, BradWhitlock, bmhan12, jcs15c, nselliott, publixsubfan, rhornung67 and white238 June 26, 2026 03:54

kennyweiss self-assigned this Jun 26, 2026

kennyweiss added Sidre Issues related to Axom's 'sidre' component Build system Issues related to Axom's build system usability Related to code usability Python labels Jun 26, 2026

kennyweiss force-pushed the feature/kweiss/improve-python-sidre branch from 08306f7 to 06e053d Compare June 26, 2026 03:55

kennyweiss commented Jun 26, 2026

View reviewed changes

bmhan12 approved these changes Jun 29, 2026

View reviewed changes

rhornung67 approved these changes Jun 29, 2026

View reviewed changes

kennyweiss mentioned this pull request Jul 1, 2026

Add an installation test for Axom's python interface #1900

Open

kennyweiss force-pushed the feature/kweiss/improve-python-sidre branch from 06e053d to 5114bc4 Compare July 2, 2026 06:35

kennyweiss added 8 commits July 15, 2026 13:59

Spack: Make Axom a Python extension for the bindings

bc208f0

This allows spack environment views to place Axom's installed Python packages onto the interpreter path automatically. This commit also emits `AXOM_PYTHON_MODULE_INSTALL_PREFIX` in the generated host-config.

Python: Tests now run direction through python (without wrapper)

9317cc2

Sidre: Update Python docs

ba142d6

Python: Allows default Python install path to be relative

aa6f6da

sidre: Improves ImportError checks in axom.sidre

c4196e0

Improves axom_add_python_test CMake macro to take a COMMAND keyword

83292d0

One can call the macro with either a COMMAND or a SOURCE file.

Fixes python shim test

6c1ffa6

kennyweiss added 10 commits July 15, 2026 13:59

python: Updates docs to note limitations of running python through sp…

158da35

…ack env

Updates RELEASE-NOTES

8c69385

python: Fixes installation of run_axom_with_python script

073407d

Misc: Fixes typos and whitespace issues

sidre: In python interface, pinned views need to be associated with t…

1fecaf3

…he DataStore

sidre: Removes unreachable Python binding for setExternal

52ef44f

sidre: Improves some python docs

71e8428

sidre: configure and install sidre Python stubs

6264f0b

This allows type checkers to see the sidre Python API

python: Adds installation tests for Axom Python bindings

61a0792

The tests are in our github-ci tests as well as for spack-based installations

python: Registers axom as the NB_DOMAIN for axom.sidre

46c633d

This will allow the types from all of the Axom python modules to better interoperate.

Adds a spack installation test for Python (when enabled)

c18ab0b

kennyweiss force-pushed the feature/kweiss/improve-python-sidre branch from 5114bc4 to c18ab0b Compare July 15, 2026 21:02

kennyweiss commented Jul 15, 2026

View reviewed changes

white238 reviewed Jul 15, 2026

View reviewed changes

Comment thread scripts/github-actions/linux-build_and_test.sh

white238 reviewed Jul 15, 2026

View reviewed changes

Comment thread src/axom/sidre/docs/sphinx/python_interface.rst Outdated

white238 approved these changes Jul 15, 2026

View reviewed changes

kennyweiss and others added 6 commits July 15, 2026 14:53

Apply suggestions from code review

691471b

Co-authored-by: Chris White <white238@llnl.gov>

sidre: Improve docs about pinned Views in python interface

23c93eb

sidre: Removes pysidre shim in favor of axom.sidre in Python interface

77f6f49

sidre: Use sidre instead of pysidre in Python unit tests and example

71fd96a

Updates RELEASE-NOTES

279e34b

Bugfix: We weren't passing in the right parallel flag for make test…

16c6a1a

… in CI

kennyweiss commented Jul 16, 2026

View reviewed changes

kennyweiss mentioned this pull request Jul 16, 2026

Bugfix: CI jobs should fail if any unit test fails #1920

Draft

4 tasks

kennyweiss commented Jul 16, 2026

View reviewed changes

Comment thread scripts/github-actions/linux-build_and_test.sh Outdated

Removes or_die bugfix from this branch to handle in a dedicated branch

9d4b138

kennyweiss merged commit 4839900 into develop Jul 16, 2026
15 checks passed

kennyweiss deleted the feature/kweiss/improve-python-sidre branch July 16, 2026 09:04

bmhan12 mentioned this pull request Jul 29, 2026

axom: Add cxxstd variant for C++20, python variant improvements spack/spack-packages#5791

Open

Uh oh!

Conversation

kennyweiss commented Jun 26, 2026

Summary

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kennyweiss Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kennyweiss Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

kennyweiss Jul 1, 2026 •

edited

Loading

kennyweiss Jul 1, 2026 •

edited

Loading