Skip to content

[FEATURE ADD] Redesign Pool class and Pool group architecture #6

Description

@HunterAP23

Describe the feature that is being tested/implemented
Redesign the architecture for creating individual pools and pool groups.
Current implementation is confusing and hard to work with.

Miscellaneous changes required:

  • Pools should have the same inputs & functionality as the underlying ThreadPoolExecutor and ProcessPoolExecutor classes, with the added ability to start with / change CPU core affinity.
  • Pools should have the following two arguments:
    • Iterable core_iter listing the individual core numbers that the pool can use. IF not definied then assume all available cores that the process can use.
    • Integer core_limit to limit how many CPU cores can be utilized at any given time. If not defined then assume all provided cores can be utilized at once.
  • Pool groups should just handle holding onto pools with an overall affinity
  • Pools inside pool groups should be given a name and/or a unique ID
  • Pool groups can use either a dictionary key or a namespace for accessing individual pools within the pool group
  • Telling a pool group to run a function without giving a name of a specific pool should run the function across all pools within the pool group.
  • Pools within pool groups can have their own affinities based on the pool group's affinity

List any relations to other issues in this repository or in other repositories
Relates to #5

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions