Skip to content

Implement submitted jobs pipeline#3670

Open
r4victor wants to merge 18 commits intomasterfrom
issue_3551_submitted_jobs_pipeline
Open

Implement submitted jobs pipeline#3670
r4victor wants to merge 18 commits intomasterfrom
issue_3551_submitted_jobs_pipeline

Conversation

@r4victor
Copy link
Collaborator

Part of #3551

This PR migrates submitted_jobs background task to pipelines. The main behavioral change is that old task locked all project's instances before selecting fleets and held the lock during the entire instance assignment duration that includes long get offers calls. Now the fleet is selected without instance lock and instance lock is only acquired for the chosen fleet to select existing instance offer, which is quick. Otherwise it's mostly one-to-one migration with pipeline specifics such separate processing/apply phases.

Known submitted_jobs issues like #3665, #3663, #3603 are NOT fixed. I decided to fix them for pipelines only after dropping scheduled tasks altogether since they require changes across background tasks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant