Make the Overlay Algorithm Multithreadable#12
Make the Overlay Algorithm Multithreadable#12samf25 wants to merge 4 commits intoMuonColliderSoft:mainfrom
Conversation
|
The main issue was something with ROOT file In theory, there will be some waiting early on, but due to this waiting the overlay threads should start to stagger their requests. It depends on the file opening time. There will be an upper limit on the number of effective threads since the bottleneck will always be the I/O thread. I did consider implementing a caching feature, but with the amount of BIB samples, that's a high memory cost. If we were okay with this memory cost, then things would dramatically speed up as we lazy-load the BIB files. A different speed-up could be gained if we combined BIB sample files. Right now the bottleneck is file opening. So if there were fewer files to open, we could implement a cache because there would be more frequent requests for the same files. As it stands, with 6666 files in each group, repeated requests are unlikely. |
|
This error with the refitter as exists outside of the multithreading framework. It comes up when there's BIB which I didn't test at first. Basically, if there are tracks that aren't linked to any MCParticles, it will look somewhere that doesn't exist in the map. Now it checks to make sure it exists first. |
This is the last piece of the multithreading story. I've tested this and it's good to go 👍 . Now all of digi and reco can be multithreaded over events