Skip to content

Commit 8577d17

Browse files
committed
Update README with benchmark information
1 parent 201f40a commit 8577d17

File tree

1 file changed

+96
-16
lines changed

1 file changed

+96
-16
lines changed

README.md

Lines changed: 96 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,29 @@
11
Come out and Clar
22
=================
33

4-
In Catalan, "clar" means clear, easy to perceive. Using clar will make it
5-
easy to test and make clear the quality of your code.
4+
Clar is a minimal C unit testing and benchmarking framework. It
5+
provides a simple mechanism for writing tests and asserting
6+
postconditions, while providing support for TAP and JUnit style
7+
outputs.
68

7-
> _Historical note_
8-
>
9-
> Originally the clar project was named "clay" because the word "test" has its
10-
> roots in the latin word *"testum"*, meaning "earthen pot", and *"testa"*,
11-
> meaning "piece of burned clay"?
12-
>
13-
> This is because historically, testing implied melting metal in a pot to
14-
> check its quality. Clay is what tests are made of.
9+
In Catalan, "clar" means clear, easy to perceive. Using Clar will make it
10+
easy to test and make clear the quality of your code.
1511

1612
## Quick Usage Overview
1713

1814
Clar is a minimal C unit testing framework. It's been written to replace the
1915
old framework in [libgit2][libgit2], but it's both very versatile and
2016
straightforward to use.
2117

22-
Can you count to funk?
23-
24-
- **Zero: Initialize test directory**
18+
1. **Initialize test directory**
2519

2620
~~~~ sh
2721
$ mkdir tests
2822
$ cp -r $CLAR_ROOT/clar* tests
2923
$ cp $CLAR_ROOT/example/*.c tests
3024
~~~~
3125

32-
- **One: Write some tests**
26+
2. **Write some tests**
3327

3428
File: tests/adding.c:
3529

@@ -59,7 +53,7 @@ Can you count to funk?
5953
}
6054
~~~~~
6155

62-
- **Two: Build the test executable**
56+
3. **Build the test executable**
6357

6458
~~~~ sh
6559
$ cd tests
@@ -68,7 +62,7 @@ Can you count to funk?
6862
$ gcc -I. clar.c main.c adding.c -o testit
6963
~~~~
7064

71-
- **Funk: Funk it.**
65+
4. **Run the tests**
7266

7367
~~~~ sh
7468
$ ./testit
@@ -319,6 +313,92 @@ void test_example__a_test_with_auxiliary_methods(void)
319313
}
320314
~~~~
321315
316+
## Benchmarks
317+
318+
The clar mixer (`generate.py`) and runner can also be used to support
319+
simple benchmark capabilities. When running in benchmark mode, Clar
320+
will run each test multiple times in succession, using a high-resolution
321+
platform timer to measure the elapsed time of each run.
322+
323+
By default, Clar will run each test repeatedly for 3 seconds (with
324+
a minimum of 10 runs), but you can define the explicit number of
325+
runs for each test in the definition.
326+
327+
By default, Clar will run the initialization and cleanup functions
328+
before _each_ test run. This allows for consistent setup and teardown
329+
behavior, and predictability with existing test setups. However, you
330+
can avoid this additional overhead by defining a _reset_ function.
331+
This will be called between test runs instead of the cleanup and
332+
re-initialization; in this case, initialization will occur only
333+
before all test runs, and cleanup will be performed only when all
334+
test runs are complete.
335+
336+
To configure a benchmark application instead of a test application:
337+
338+
1. **Set clar into benchmark mode in your main function**
339+
340+
~~~~ c
341+
int main(int argc, char *argv[])
342+
{
343+
clar_test_set_mode(CL_TEST_BENCHMARK);
344+
clar_test_init(argc, argv);
345+
res = clar_test_run();
346+
clar_test_shutdown();
347+
return res;
348+
}
349+
~~~~
350+
351+
2. **Optionally, set up your initialization, cleanup, and reset
352+
functions**
353+
354+
~~~~ c
355+
void test_foo__initialize(void)
356+
{
357+
global_data = malloc(1024 * 1024 * 1024);
358+
memset(global_data, 0, 1024 * 1024 * 1024);
359+
}
360+
361+
void test_foo__reset(void)
362+
{
363+
memset(global_data, 0, 1024 * 1024 * 1024);
364+
}
365+
366+
void test_foo__cleanup(void)
367+
{
368+
global_data = malloc(1024 * 1024 * 1024);
369+
}
370+
~~~~
371+
372+
3. **Optionally, configure tests with a specific run number**
373+
374+
~~~~ c
375+
/* Run this test 500 times */
376+
void test_foo__bar(void)
377+
/* [clar]:runs=500 */
378+
{
379+
bar();
380+
}
381+
~~~~
382+
383+
3. **Run the benchmarks**
384+
385+
When running in benchmark mode, you'll see timings output; if you
386+
write a summary file, it will be a JSON file that contains the
387+
time information.
388+
389+
~~~~ sh
390+
$ ./benchmarks -r/path/to/results.json
391+
Started benchmarks (mean time ± stddev / min time … max time):
392+
393+
foo::bar: 24.75 ms ± 1.214 ms / range: 24.41 ms … 38.06 ms (500 runs)
394+
foo::baz: 24.67 ms ± 248.2 μs / range: 24.41 ms … 25.41 ms (478 runs)
395+
foo::qux: 25.98 ms ± 333.0 μs / range: 25.64 ms … 26.82 ms (112 runs)
396+
~~~~
397+
398+
Note: you can change the prefix of the test function names from `test_`
399+
to something of your choice by using the `--prefix=...` option for
400+
the `generate.py` mixer script.
401+
322402
About Clar
323403
==========
324404

0 commit comments

Comments
 (0)