Cannot initialize second experiment from library #932
Description
-
Tracking in Enable Initializing Second Experiment From Library #931
-
There was an issue discovered in Automatically Enumerate Dryruns and MpiOnlyExperiment #904, where trying to initialize an experiment other than
saxpy(which is used in every current pytest), will take the form:
$ benchpark unit-test
_________________________________________________________________________________ test_programming_model_checks __________________________________________________________________________________
def test_programming_model_checks():
# babelstream mpi-only not valid
with pytest.raises(NotImplementedError, match="cannot run with MPI only"):
spec = benchpark.spec.ExperimentSpec("babelstream").concretize()
> experiment = spec.experiment # noqa: F841
lib/benchpark/test/experiment_errors.py:15:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
lib/benchpark/spec.py:380: in experiment
return self.experiment_class(self)
lib/benchpark/experiment.py:225: in __init__
super().__init__()
lib/benchpark/cuda.py:16: in __init__
super().__init__()
lib/benchpark/rocm.py:16: in __init__
super().__init__()
lib/benchpark/openmp.py:17: in __init__
if self.spec.variants["openmp"][0]:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <benchpark.spec.ConcreteVariantMap object at 0x14a4e9590>, key = 'openmp'
def __getitem__(self, key):
> return self.dict[key]
E KeyError: 'openmp'
../../../.benchpark/ramble/lib/ramble/llnl/util/lang.py:495: KeyError
==================================================================================== short test summary info =====================================================================================
FAILED lib/benchpark/test/experiment_errors.py::test_programming_model_checks - KeyError: 'openmp'
We know the issue is with multiple initialization because running the test on its own works, i.e. $ benchpark unit-test lib/benchpark/test/experiment_errors.py
The issue seems to stem from the variant directive not working correctly after the first initialization. This behavior does not affect normal benchpark use, as initializing experiments from the command line will never cause this issue.
Metadata
Metadata
Assignees
Labels
No labels