feat: Remove global (cross-worker) stopping criteria #252
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR restructures how stopping conditions are handled in NEPS by removing the old global, cross-worker logic and shifting all controlling parameters to the level of individual workers.
definition: A worker_id represents a single execution of neps.run. Running NEPS again—sequentially or in parallel—creates a new worker_id. Stopping parameters are scoped to that worker rather than shared across all active workers.
Previously, fidelity usage and total evaluations were tracked also in memory of the worker, and trial files on disk were not checked for a single worker. For avoiding inconsistency and having one single source of truth, this PR moves to a disk-based ground truth and removes the now-invalid parameters from the worker settings, e.g. max_evaluation_for_worker, max_cost_for_worker,..
The earlier design also treated fidelity_to_spend as a global limit across workers, which made it impossible to define per-worker limits, especially in multi-fidelity setups where evaluations_to_spend could not be combined with that system. cost_to_spend and evaluations_to_spend were effectively the same logic pretending to be different knobs.
Now each worker has its own independent stopping parameters, and multiple stopping criteria are supported instead of being restricted to just one.
One trade-off: when users pass something like evaluations_to_spend, they must be aware of how much has already been consumed by the worker and adjust the value accordingly after failures or restarts. This will be addressed by adding a new summary report—total evaluations, total evaluation time, total cost, total fidelity usage.
This brings the API closer to predictable, per-worker resource accounting while cleaning up several historical inconsistencies.