keras-benchmarks.rst

Keras Benchmarks

TODO: some of this belongs to test/.

Run the Keras benchmarks under hpvm/hpvm/test/dnn_benchmarks/keras

Download CNN Model Files

Prior to running the benchmarks, ensure you download the CNN model data (inputs and weights) if not done in automatic build script.

wget https://databank.illinois.edu/datafiles/o3izd/download -O model_params.tar.gz
tar -xf  model_params.tar.gz

Move extracted model_params directory to /test/dnn_benchmarks/model_params (Benchmarks expect data at this location)

Running Benchmaks

List of benchmarks and the expected accuracies:

Benchmark	Accuracy
alexnet.py	79.28
alexnet2.py	84.98
alexnet_imagenet.py	56.30
lenet.py	98.70
mobilenet_cifar10.py	84.42
resnet18_cifar10.py	89.56
resnet50_imagenet.py	75.10
vgg16_cifar10.py	89.96
vgg16_cifar100.py	66.50
vgg16_imagenet.py	69.46

Synopsis

python3 ${BENCH_NAME}.py  [hpvm_reload|keras_reload]  [frontend] [compile]

Command-line Parameters

hpvm_reload : Reloads HPVM weights (.bin binary format used by HPVM weights - present in model_params download directory) from directory path specified in the reload_dir parameter set in code - this is described in "Parameters to Change in Code" (below).

keras_reload: Alternatively, reload weights in Keras .h5 file format with path to file specified in keras_model_file described in "Parameters to Change in Code" (below).

frontend: Invokes the HPVM frontend and dumps weights (in HPVM .bin format) in the output directory specified. The parameters that control where data and source files are dumped are specified by parameters data_dir and src_dir, respectively. These are described below.

compile: Optional Parameter. When specified, it compiles the HPVM-C code generated by the frontend into an HPVM binary under the directory specified by src_dir (described below). If src_dir path exists, a unique directory (which appends a unique ID) is created. The binary is built with the name HPVM_binary.