Repository navigation
Benchmark visualizer: Venz #784
Description
Hi there!
First of all, thanks a lot for building and maintaining hyperfine, it's a great and omnipresent tool.
Since I had a desire to visualize a range of benchmarks over time in various situations, I thought it might be useful to render charts in the browser for nice "dataviz". So one thing led to another, and an early "local-only" version to try is here: https://try.venz.dev. No data leaves the browser.
Use the flow/form to manage commands and provide titles and colors. Or drop JSON output directly onto https://try.venz.dev/chart (either drop the file, or paste JSON as text). Todo:
- Render "summary" in chart (to compare benchmarks and see which one is fastest etc)
- Support other output formats (eg CSV), maybe even also the CLI output itself
- Improve UX/accessbility
Feel free to provide any kind of feedback or suggest ideas. Better chart types, etc. etc. Anything.
Plenty more ideas I have. Already have a "remote storage" version working (local = IndexedDB, remote = SQLite), and I'm looking for some kind of "on-demand bare metal" cloud solutions to run benchmarks in an isolated and stable environment. End-users would then initiate such runs from the Venz UI, while also allowing more "prepare" and "build" commands, etcetera. I might also start support for other benchmark running tools.
For now, at least we have a public visualizer for hyperfine output. Here's hoping to make running benchmarks more fun and accessible to more people.
By the way, the name Venz is a portmanteau(?) of "bENchmark VisualiZer".
Activity
webpro commentedon Dec 19, 2024
For anyone interested, I'm building this in public. Follow me on Bluesky for progress updates: https://bsky.app/profile/webpro.nl
sharkdp commentedon Dec 21, 2024
Wow — this is really cool, thank you for sharing it here! Great idea and very nice execution.
Is the project open source too?
Just to be sure: you have seen our visualization scripts at https://github.com/sharkdp/hyperfine/tree/master/scripts?
Personally, I don't see the need for this. JSON seems like the best fit here?
I think the charts could use a bit more polish. I'd like to see
axis labels(wow, I just noticed that it does have labels, but the contrast of the font is so low, I didn't see them), a legend (in the chart, not below it), etc.One thing I would find very valuable personally would be to edit some of the chart properties interactively. This is something we can't easily do with the Python scripts. For example, I often run a benchmark, store the results, but forget to set a nice label. Being able to adjust the command names would be really neat. And it would be really cool to change colors in the chart view, not before.
Another thing that would be useful is support for parametrized benchmarks (see
--parameter-scanand--parameter-list). These benchmarks often lend themselves well to interesting plotting opportunities.webpro commentedon Dec 22, 2024
Thanks! Much appreciated.
Not yet, but it will be.
I glanced over them, but didn't actually try since I figured there wouldn't be too much interactivity and I wanted to play with Solid JS and D3. Also I'm not doing much Python so didn't fancy going through installing things. But I might still try, let me know if there's something of interest.
Yeah, no priority in supporting more than JSON. Was thinking it could be neat to pipe hyperfine results to clipboard so one can then paste into browser without even adding
--export-json [file]but it would obviously miss out on individual data points.Do you mean in the browser or in the generated image? In the browser contrast should be very high. Added a screenshot below just in case. The images aren't really looked after yet, they've a transparent background, I figured for now that's flexible. There's also a simple legend you can toggle, but not too fond of it.
That has now been added. And an option to save that test as a new item. So you can approach in two ways.
Great. I already had something for that so let's just say it's on the TODO list (that's why the strategy only has a "default" option, there'll be another option for this)
sharkdp commentedon Dec 27, 2024
I was not using dark mode. Anyway, contrast is much better now after the update. Thanks. There is a new problem where the y axis labels seem to have another label behind them in gray?
Very nice
👍
One additional feedback point: the toggle buttons look very nice, but I find the UX not really great. I think I'd prefer something were I could immediately select the option I want. And immediately understand what I'm changing without having to interpret the icons.