worker_active logic can be replaced with a break instead

[notirl]

In the workers for running regular measurements we have a worker_active variable. This isn't ever useful afaict as as soon as it's set to false and if statement causes the loop to end anyway.

There's a wait at the end of the loop which doesn't seem to serve any purpose. Maybe we can just remove this.

[irl]

Looking at this more, it's actually more complicated. Probably we should draw out what this state machine looks like before messing with it.

[mirjak]

I think the wait is there to make sure that no new jobs come in. Currently I think we load all jobs at the beginning and then never add any additional ones later but as far as I understood this was originally designed to also be able to stream input...

[irl]

You can feed jobs in until you send the shutdown sentinel. The issue here is that it is sleeping after the shutdown sentinel has been received before terminating the thread. There's not really anything else that the worker can do because after the sleep it will just die anyway.

I think this is a left-over from the synchronised spider where we wanted to keep all workers alive until the end to be able to spin the semaphores, but I can't be sure yet.

[mirjak]

Okay. Was just guess which wait you meant...