MPI shared memory socket scope by kab163 · Pull Request #1072 · llnl/Umpire

kab163 · 2026-05-04T22:20:11Z

No description provided.

…ocket-scope

adayton1 · 2026-06-03T22:50:09Z

Would it be worth considering adding hwloc as a dependency to get the socket info?

kab163 · 2026-06-04T14:25:24Z

Would it be worth considering adding hwloc as a dependency to get the socket info?

I think it would be worth considering... I think for now we should merge this and see how people like it. If there's a need/desire to add hwloc later, we can. (Maybe as an optional dependency - like use hwloc if detected, otherwise use the linux system calls/numa like the branch does now.)

…ocket-scope

This reverts commit c97fd78.

kab163 · 2026-06-09T23:23:15Z

Update: Hubcast doesn't understand jobs that are allowed to fail. We can either fix the job failure or try to fix within Hubcast somehow.. stay tuned

adayton1 · 2026-06-09T23:35:10Z

Update: Hubcast doesn't understand jobs that are allowed to fail. We can either fix the job failure or try to fix within Hubcast somehow.. stay tuned

You'll need to update branch protections under settings -> branches -> develop and list the CI jobs that are required to pass (instead of ci/gitlab.lc.llnl.gov).

kab163 · 2026-06-10T00:00:01Z

Update: Hubcast doesn't understand jobs that are allowed to fail. We can either fix the job failure or try to fix within Hubcast somehow.. stay tuned

You'll need to update branch protections under settings -> branches -> develop and list the CI jobs that are required to pass (instead of ci/gitlab.lc.llnl.gov).

I don't see where I'd be able to do this in the develop branch's settings... (I'm getting copilot to help me...)

…ocket-scope

…nto task/add-mpi3-socket-scope

…ocket-scope

kab163 · 2026-07-09T22:29:54Z

@lc-hubcast approve

lc-hubcast · 2026-07-09T22:29:59Z

To approve syncing this PR, please use the GitHub review comment feature to submit an approval. This ensures approval is tied to a specific commit to avoid the sync of malicious data.

kab163

@lc-hubcast approve

adayton1 · 2026-07-11T02:29:10Z

+ *
+ * \return true when the affinity mask is non-empty and maps to a single socket.
+ */
+bool affinity_maps_to_single_socket(std::string& reason);


Is there a better place for this method?

I made some changes for this...

adayton1 · 2026-07-11T02:39:40Z

+               "shared_scope::socket for MPI3 shared memory requires Linux CPU affinity information");
+    }
+
+    const int color = get_socket_color_from_affinity();


I don't think this will compile on platforms other than linux. Won't the socket scope code require a macro guard (e.g. #if defined(__linux__)) rather than just an if constexpr?

I made some changes for this...

adayton1 · 2026-07-11T02:41:47Z

+
+} // end anonymous namespace
+
+bool affinity_maps_to_single_socket(std::string& reason)


Isn't this also defined in umpire.hpp and umpire.cpp?

I made some changes for this...

…nto task/add-mpi3-socket-scope

adayton1 · 2026-07-17T16:48:39Z

+#if defined(__linux__)
+namespace {
+
+bool all_ranks_affinity_maps_to_single_socket(std::string& reason)


This function appears to be duplicated a couple of times.

adayton1 · 2026-07-17T16:49:17Z

 #include "umpire/strategy/NamedAllocationStrategy.hpp"
 #include "umpire/util/MemoryResourceTraits.hpp"

+bool all_ranks_affinity_maps_to_single_socket(std::string& reason)


This is duplicated a bunch.

adayton1 · 2026-07-17T16:51:15Z

 }
 #endif

+bool affinity_maps_to_single_socket(std::string& reason)


Is this wrapper function actually needed?

adayton1 · 2026-07-17T16:52:05Z

+#if defined(__linux__) && defined(UMPIRE_ENABLE_MPI3_SHARED_MEMORY)
+namespace {
+
+bool all_ranks_affinity_maps_to_single_socket(std::string& reason)


This is the other duplicate.

adayton1 · 2026-07-17T16:53:12Z

+    GTEST_SKIP() << "Socket-scoped MPI3 shared memory requires ranks bound to a single socket (" << e.what() << ")";
+  }
+
+  auto comm = umpire::get_communicator_for_allocator(allocator, MPI_COMM_WORLD);


Don't these communicators need to be cleaned up at the end of the test?

Amazing, thank you for catching that. Yes they do

adayton1 · 2026-07-17T16:54:00Z

+  ASSERT_EQ(socket_data[0], 42);
+  ASSERT_EQ(socket_data[1], nranks);
+
+  allocator.deallocate(socket_data);


Clean up cached communicators?

kab163 added 4 commits May 4, 2026 15:04

initial changes for mpi shared memory socket scope

a988a34

revising socket color impl

ec20a7a

fix typo

fb24207

cleaning up and adding docs

498b2b6

kab163 marked this pull request as ready for review May 11, 2026 22:55

kab163 requested a review from davidbeckingsale May 12, 2026 17:36

kab163 requested review from adayton1 and rhornung67 June 1, 2026 21:33

adayton1 reviewed Jun 1, 2026

View reviewed changes

Comment thread src/umpire/resource/HostMpi3SharedMemoryResource.cpp

adayton1 reviewed Jun 1, 2026

View reviewed changes

Comment thread src/umpire/resource/HostMpi3SharedMemoryResource.cpp Outdated

adayton1 reviewed Jun 1, 2026

View reviewed changes

Comment thread src/umpire/resource/HostMpi3SharedMemoryResource.cpp Outdated

kab163 added 2 commits June 2, 2026 16:49

Merge branch 'develop' of github.com:llnl/Umpire into task/add-mpi3-s…

08e5199

…ocket-scope

adding some docs and error checking

071eb71

adayton1 reviewed Jun 3, 2026

View reviewed changes

Comment thread src/umpire/resource/HostMpi3SharedMemoryResource.cpp Outdated

adayton1 previously approved these changes Jun 3, 2026

View reviewed changes

rearranging helper functions and linux define statement

6e9735b

kab163 dismissed adayton1’s stale review via 6e9735b June 9, 2026 19:17

adayton1 reviewed Jun 9, 2026

View reviewed changes

Comment thread src/umpire/resource/HostMpi3SharedMemoryResource.cpp Outdated

adayton1 reviewed Jun 9, 2026

View reviewed changes

Comment thread src/umpire/resource/HostMpi3SharedMemoryResource.cpp Outdated

kab163 added 4 commits June 9, 2026 13:05

using constexpr

3cc3d52

Merge branch 'develop' of github.com:llnl/Umpire into task/add-mpi3-s…

3e34c1f

…ocket-scope

update radiuss-spack-configs

c97fd78

Revert "update radiuss-spack-configs"

3b8e9fd

This reverts commit c97fd78.

attempting to fix the hubcast reporting

24b2a1c

trigger ci

63f4308

adayton1 reviewed Jul 7, 2026

View reviewed changes

Comment thread .github/hubcast.yml

adayton1 reviewed Jul 7, 2026

View reviewed changes

Comment thread src/umpire/Umpire.hpp Outdated

adayton1 reviewed Jul 7, 2026

View reviewed changes

Comment thread examples/mpi3_shared_memory.cpp

adayton1 reviewed Jul 7, 2026

View reviewed changes

Comment thread examples/mpi3_shared_memory.cpp

adayton1 reviewed Jul 7, 2026

View reviewed changes

Comment thread examples/mpi3_shared_memory.cpp Outdated

kab163 added 4 commits July 7, 2026 14:30

Merge branch 'develop' of github.com:llnl/Umpire into task/add-mpi3-s…

1ce4f75

…ocket-scope

Merge branch 'task/add-mpi3-socket-scope' of github.com:llnl/Umpire i…

da4aaf2

…nto task/add-mpi3-socket-scope

fixing indentations

2375e2d

Merge branch 'develop' of github.com:llnl/Umpire into task/add-mpi3-s…

f9f3fd2

…ocket-scope

kab163 requested a review from adayton1 July 9, 2026 17:06

kab163 commented Jul 9, 2026

View reviewed changes

adayton1 reviewed Jul 11, 2026

View reviewed changes

Comment thread src/umpire/resource/HostMpi3SharedMemoryResource.cpp Outdated

adayton1 reviewed Jul 11, 2026

View reviewed changes

kab163 added 5 commits July 13, 2026 13:47

merging with develop

36abbce

Merge branch 'task/add-mpi3-socket-scope' of github.com:llnl/Umpire i…

a425ef6

…nto task/add-mpi3-socket-scope

fix indentation in source file

f669e76

fixing the cpu affinity mapping

635d304

update docs

5a55283

kab163 requested a review from MrBurmark July 13, 2026 22:43

adayton1 reviewed Jul 17, 2026

View reviewed changes


		} // end anonymous namespace

		bool affinity_maps_to_single_socket(std::string& reason)

Uh oh!

Conversation

kab163 commented May 4, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

adayton1 commented Jun 3, 2026

Uh oh!

kab163 commented Jun 4, 2026

Uh oh!

Uh oh!

Uh oh!

kab163 commented Jun 9, 2026

Uh oh!

adayton1 commented Jun 9, 2026

Uh oh!

kab163 commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kab163 commented Jul 9, 2026

Uh oh!

lc-hubcast Bot commented Jul 9, 2026

Uh oh!

kab163 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kab163 commented Jun 10, 2026 •

edited

Loading