When processing multiple seed URLs, work should be split across them to reduce hammering on a single box.