Track performance and memory use over time

[bnoordhuis]

Scrounged from #917 (comment):

1. Add benchmarks for things we're interested in (startup time, codegen performance, etc.)

2. Run valgrind's callgrind and massif on said benchmarks for every new commit to master

3. Extract the instruction count and (min/max/avg/std?) memory use

4. Store the results somewhere; e.g., commit from CI to an orphan branch in this repo

5. Crunch numbers and plot pretty graphs (TBD how and what with)

Optional but very desirable: run (2) for existing commits

[bnoordhuis]

Add benchmarks for things we're interested in (startup time, codegen performance, etc.)

Note to self: web-tooling-benchmark would be an obvious choice but it:

1. would need to be split into standalone benchmarks (it's a whole suite running inside a single process now), and it

2. mostly exercises runtime performance, not (for example) time to first executed bytecode

[saghul]

What's the CLI incantation to use callgrind and massif? I've never used them before 😅

[bnoordhuis]

$ valgrind --tool=callgrind --callgrind-out-file=callgrind.out --dump-instr=yes <program>
$ grep -w summary callgrind.out | awk '{print $2}' # instruction count

$ valgrind --tool=massif --massif-out-file=massif.out <program>
$ grep -w mem_heap_B massif.out | awk -F= '{print $2}' | sort -n | tail -1 # peak memory

Add --instr-atstart=no when using the toggle on/off macros from <valgrind/callgrind.h>

[gengjiawen]

Can quickjs-ng track the peak memory usage number now ?

[bnoordhuis]

No (as in: it could but it currently doesn't.) Why the question?