aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorFlorian Fischer <florian.fl.fischer@fau.de>2019-11-27 14:18:23 +0100
committerFlorian Fischer <florian.fl.fischer@fau.de>2019-11-27 14:18:23 +0100
commit704bd23780ea5922d852abf9638b07e215118625 (patch)
tree0e1c00672ee06e0ed357ba306f87725b7651a988
parent2a27c8ef4d9b65390edd45c6d3b912d99a8f2ac9 (diff)
downloadallocbench-704bd23780ea5922d852abf9638b07e215118625.tar.gz
allocbench-704bd23780ea5922d852abf9638b07e215118625.zip
Update Readme
Mention the seperation into our three scripts. Add some requirements.
-rw-r--r--Readme.md87
1 files changed, 66 insertions, 21 deletions
diff --git a/Readme.md b/Readme.md
index 545d460..a06b190 100644
--- a/Readme.md
+++ b/Readme.md
@@ -1,6 +1,8 @@
# allocbench - benchmark tool for POSIX memory allocators
-To download allocbench run
+allocbench is a POSIX memory allocator benchmarking framework and tooling.
+
+To obtain allocbench run
```shell
git clone https://muhq.space/software/allocbench.git
@@ -8,58 +10,100 @@ git clone https://muhq.space/software/allocbench.git
## Requirements
-* python3
+* python >= 3.6
* make, find, gcc (build dependencies)
* perf (`perf stat -d` is the default command to measure benchmark results)
* util-linux (`whereis` is used to find system installed allocators)
-* (git to clone allocators in `src/allocators/*.py`)
+* git, tar to handle external artifacts
+* numpy and matplotlib to summarize results and generate plots
## Usage
+allocbench consists of three small utilities: `bench.py`, `summarize.py` and `merge.py`.
+`bench.py` is used to prepare, analyze and run benchmarks.
- usage: bench.py [-h] [-ds, --dont-save] [-l LOAD] [--analyse] [-r RUNS] [-v]
- [-vdebug] [-b BENCHMARKS [BENCHMARKS ...]]
+ usage: bench.py [-h] [--analyze] [-r RUNS] [-v]
+ [-b BENCHMARKS [BENCHMARKS ...]]
[-xb EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...]]
- [-a ALLOCATORS [ALLOCATORS ...]] [-ns] [-rd RESULTDIR]
- [--license]
+ [-a ALLOCATORS [ALLOCATORS ...]] [-rd RESULTDIR] [--license]
+ [--version]
benchmark memory allocators
optional arguments:
-h, --help show this help message and exit
- -ds, --dont-save don't save benchmark results in RESULTDIR
- -l LOAD, --load LOAD load benchmark results from directory
- --analyse analyse benchmark behaviour using malt
+ --analyze analyze benchmark behavior using malt
-r RUNS, --runs RUNS how often the benchmarks run
-v, --verbose more output
- -vdebug, --verbose-debug
- debug output
-b BENCHMARKS [BENCHMARKS ...], --benchmarks BENCHMARKS [BENCHMARKS ...]
benchmarks to run
-xb EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...], --exclude-benchmarks EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...]
explicitly excluded benchmarks
-a ALLOCATORS [ALLOCATORS ...], --allocators ALLOCATORS [ALLOCATORS ...]
allocators to test
- -ns, --nosum don't produce plots
-rd RESULTDIR, --resultdir RESULTDIR
directory where all results go
--license print license info and exit
+ --version print version info and exit
+
+`./summarize.py` is used to summarize results created with bench.py.
+It groups the included allocators into categories to produce readable and not extremely noisy plots.
+
+ usage: summarize.py [-h] [--license] [--version]
+ [-b BENCHMARKS [BENCHMARKS ...]]
+ [-x EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...]]
+ results
+
+ Summarize allocbench results in allocator sets
+
+ positional arguments:
+ results path to results
+
+ optional arguments:
+ -h, --help show this help message and exit
+ --license print license info and exit
+ --version print version info and exit
+ -b BENCHMARKS [BENCHMARKS ...], --benchmarks BENCHMARKS [BENCHMARKS ...]
+ benchmarks to summarize
+ -x EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...], --exclude-benchmarks EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...]
+ benchmarks to exclude
+
+`./merge.py` can combine the results of different benchmark runs.
+
+ usage: merge.py [-h] [--license] [--version] [-b BENCHMARKS [BENCHMARKS ...]]
+ [-x EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...]]
+ src dest
+
+ Merge to allocbench results
+
+ positional arguments:
+ src results which should be merged into dest
+ dest results in which src should be merged
+
+ optional arguments:
+ -h, --help show this help message and exit
+ --license print license info and exit
+ --version print version info and exit
+ -b BENCHMARKS [BENCHMARKS ...], --benchmarks BENCHMARKS [BENCHMARKS ...]
+ benchmarks to summarize
+ -x EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...], --exclude-benchmarks EXCLUDE_BENCHMARKS [EXCLUDE_BENCHMARKS ...]
+ benchmarks to exclude
### Examples
./bench.py -b loop
-runs only the loop benchmark for some installed allocators and will put its
-results in `$PWD/results/$HOSTNAME/<time>/loop`
+runs only the loop benchmark for all included allocators and will put its
+results in `$PWD/results/$HOSTNAME/<time>/loop`.
./bench.py -a BA_allocators
builds all allocators used in Florian Fischer's [BA thesis](https://muhq.space/ba.html)
-and runs all benchmarks
+and runs all benchmarks.
- ./bench.py -r 0 -l <path/to/saved/results>
+ ./summarize.py <path/to/saved/results>
-doesn't run any benchmark just summarizes the loaded results
+summarizes the previously created results.
## Benchmarks
@@ -68,10 +112,11 @@ have a look at [doc/Benchmarks.md](doc/Benchmarks.md).
## Allocators
-By default tcmalloc, jemalloc, Hoard and your libc's allocator will be used
-if found and the `-a` option is not used.
+By default all included allocators will be build and measured. For more precise control
+about used allocators use the `-a` option.
-For more control about used allocators have a look at [doc/Allocators.md](doc/Allocators.md).
+For more details about what allocators are available, how they are used and how to
+include new once have a look at [doc/Allocators.md](doc/Allocators.md).
## License