Skip to content

Commit 0290399

Browse files
committed
Update README with benchmark information
1 parent 6f3260b commit 0290399

File tree

1 file changed

+105
-17
lines changed

1 file changed

+105
-17
lines changed

README.md

Lines changed: 105 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,29 @@
11
Come out and Clar
22
=================
33

4-
In Catalan, "clar" means clear, easy to perceive. Using clar will make it
5-
easy to test and make clear the quality of your code.
4+
Clar is a minimal C unit testing and benchmarking framework. It
5+
provides a simple mechanism for writing tests and asserting
6+
postconditions, while providing support for TAP and JUnit style
7+
outputs.
68

7-
> _Historical note_
8-
>
9-
> Originally the clar project was named "clay" because the word "test" has its
10-
> roots in the latin word *"testum"*, meaning "earthen pot", and *"testa"*,
11-
> meaning "piece of burned clay"?
12-
>
13-
> This is because historically, testing implied melting metal in a pot to
14-
> check its quality. Clay is what tests are made of.
9+
In Catalan, "clar" means clear, easy to perceive. Using Clar will make it
10+
easy to test and make clear the quality of your code.
1511

1612
## Quick Usage Overview
1713

1814
Clar is a minimal C unit testing framework. It's been written to replace the
1915
old framework in [libgit2][libgit2], but it's both very versatile and
2016
straightforward to use.
2117

22-
Can you count to funk?
23-
24-
- **Zero: Initialize test directory**
18+
1. **Initialize test directory**
2519

2620
~~~~ sh
2721
$ mkdir tests
2822
$ cp -r $CLAR_ROOT/clar* tests
2923
$ cp $CLAR_ROOT/example/*.c tests
3024
~~~~
3125

32-
- **One: Write some tests**
26+
2. **Write some tests**
3327

3428
File: tests/adding.c:
3529

@@ -59,7 +53,7 @@ Can you count to funk?
5953
}
6054
~~~~~
6155

62-
- **Two: Build the test executable**
56+
3. **Build the test executable**
6357

6458
~~~~ sh
6559
$ cd tests
@@ -68,7 +62,7 @@ Can you count to funk?
6862
$ gcc -I. clar.c main.c adding.c -o testit
6963
~~~~
7064

71-
- **Funk: Funk it.**
65+
4. **Run the tests**
7266

7367
~~~~ sh
7468
$ ./testit
@@ -314,15 +308,109 @@ void test_example__a_test_with_auxiliary_methods(void)
314308
}
315309
~~~~
316310
311+
## Benchmarks
312+
313+
The clar mixer (`generate.py`) and runner can also be used to support
314+
simple benchmark capabilities. When running in benchmark mode, Clar
315+
will run each test multiple times in succession, using a high-resolution
316+
platform timer to measure the elapsed time of each run.
317+
318+
By default, Clar will run each test repeatedly for 3 seconds (with
319+
a minimum of 10 runs), but you can define the explicit number of
320+
runs for each test in the definition.
321+
322+
By default, Clar will run the initialization and cleanup functions
323+
before _each_ test run. This allows for consistent setup and teardown
324+
behavior, and predictability with existing test setups. However, you
325+
can avoid this additional overhead by defining a _reset_ function.
326+
This will be called between test runs instead of the cleanup and
327+
re-initialization; in this case, initialization will occur only
328+
before all test runs, and cleanup will be performed only when all
329+
test runs are complete.
330+
331+
To configure a benchmark application instead of a test application:
332+
333+
1. **Set clar into benchmark mode in your main function**
334+
335+
~~~~ c
336+
int main(int argc, char *argv[])
337+
{
338+
clar_test_set_mode(CL_TEST_BENCHMARK);
339+
clar_test_init(argc, argv);
340+
res = clar_test_run();
341+
clar_test_shutdown();
342+
return res;
343+
}
344+
~~~~
345+
346+
2. **Optionally, set up your initialization, cleanup, and reset
347+
functions**
348+
349+
~~~~ c
350+
void test_foo__initialize(void)
351+
{
352+
global_data = malloc(1024 * 1024 * 1024);
353+
memset(global_data, 0, 1024 * 1024 * 1024);
354+
}
355+
356+
void test_foo__reset(void)
357+
{
358+
memset(global_data, 0, 1024 * 1024 * 1024);
359+
}
360+
361+
void test_foo__cleanup(void)
362+
{
363+
global_data = malloc(1024 * 1024 * 1024);
364+
}
365+
~~~~
366+
367+
3. **Optionally, configure tests with a specific run number**
368+
369+
~~~~ c
370+
/* Run this test 500 times */
371+
void test_foo__bar(void)
372+
/* [clar]:runs=500 */
373+
{
374+
bar();
375+
}
376+
~~~~
377+
378+
3. **Run the benchmarks**
379+
380+
When running in benchmark mode, you'll see timings output; if you
381+
write a summary file, it will be a JSON file that contains the
382+
time information.
383+
384+
~~~~ sh
385+
$ ./benchmarks -r/path/to/results.json
386+
Started benchmarks (mean time ± stddev / min time … max time):
387+
388+
foo::bar: 24.75 ms ± 1.214 ms / range: 24.41 ms … 38.06 ms (500 runs)
389+
foo::baz: 24.67 ms ± 248.2 μs / range: 24.41 ms … 25.41 ms (478 runs)
390+
foo::qux: 25.98 ms ± 333.0 μs / range: 25.64 ms … 26.82 ms (112 runs)
391+
~~~~
392+
393+
Note: you can change the prefix of the test function names from `test_`
394+
to something of your choice by using the `--prefix=...` option for
395+
the `generate.py` mixer script.
396+
317397
About Clar
318398
==========
319399

320400
Clar has been written from scratch by [Vicent Martí](https://github.com/vmg),
321401
to replace the old testing framework in [libgit2][libgit2].
322402

403+
> _Historical note_
404+
>
405+
> Originally the clar project was named "clay" because the word "test" has its
406+
> roots in the latin word *"testum"*, meaning "earthen pot", and *"testa"*,
407+
> meaning "piece of burned clay"?
408+
>
409+
> This is because historically, testing implied melting metal in a pot to
410+
> check its quality. Clay is what tests are made of.
411+
323412
Do you know what languages are *in* on the SF startup scene? Node.js *and*
324413
Latin. Follow [@vmg](https://www.twitter.com/vmg) on Twitter to
325414
receive more lessons on word etymology. You can be hip too.
326415

327-
328416
[libgit2]: https://github.com/libgit2/libgit2

0 commit comments

Comments
 (0)