Add more timings with tracings

[GeertvanHorrik]

While trying to improve the caching mechanism, I've tested several timings when working on #4471

While trying the preview, it's still not giving us the improvements we hoped for. We think it's also caused by the initialization (installing / extracting all the addins / tools), but we don't see timings in the final report so it's just guessing.

I am happy to work on a PR that adds more timings so we can figure out what is taking the most time when bootstrapping the scripts.

Ultimately, it could even be considered to cache the dependencies / addins next to the script cache so they don't need to be extracted on every build. I think this will greatly benefit performance of bootstrapping on a build agent.

Pinging @gep13 and @devlead since they recently reviewed my PR for the script cache.

[gep13]

The addins/tools/modules folder are all configurable within the cake.config flie. Normally, these folders exist within the root of the location where the entry .cake file lives, but there is no reason that these can't exist in another location on the build agent, which would mean that they don't need to be populated each build.

Does that help?

Having additional metrics to indicate how long each part of the process is taking is a good idea though, I would have no objections to getting these added if you wanted to put in a PR.

[GeertvanHorrik]

Thanks for your reply @gep13, it definitely helps!

Do you know (from the top of your head, no need to do a full investigation) if this can safely be combined / re-used among multiple builds / scripts, or will cake blindly load any assembly from the Addins / Tools directories or will it know which version to load based on the version from NuGet? Otherwise I can find it out myself in the code, but if you just happen to know it, it would be awesome :)

Will check if sharing the tools / addins will improve performance. If not, I will try to add the timings in a PR.

[gep13]

It is reasonably "safe" to share the addins folder between different builds, as the addins folder becomes the equivlent of the global nuget cache folder, and the required assembly will be found.

I have seen issues with sharing the tools folder between different builds, especially if those builds use different versions of tools. The way that tools are located is typically via a top level globbing pattern, which means that it simply uses the first version that is located in the tools folder. If your builds are using the same versions of tools, then I don't see there being an issue to move to using a shared tools folder between different builds.

[gep13]

Out of curiousity, what sort of timings are you looking to improve on here? Are we talking in the orders of seconds, 10 of seconds, minutes?

[GeertvanHorrik]

We split up in stages (initialize, build, test, (stop if ci), package, deploy, finalize). Each stage seems to add ~30 seconds, so it would "greatly" improve the build throughput.

I have seen issues with sharing the tools folder between different builds, especially if those builds use different versions of tools. The way that tools are located is typically via a top level globbing pattern, which means that it simply uses the first version that is located in the tools folder. If your builds are using the same versions of tools, then I don't see there being an issue to move to using a shared tools folder between different builds.

That could be an issue if we hotfix an old version (e.g. master is a few patches away from develop). Will check the code to see if I might be able to implement version specific loading if a version is pinned.