Autotuning with Error-predictive Proxy

Performs autotuning on program approximation knobs using an error-predictive proxy in place of the original program, to greatly speedup autotuning while getting results comparable in quality.

Work in progress.

Getting Started

After finishing this readme, go to ./proxy_tuner.py to try tuning one model. Use this set of arguments for a start:

python proxy_tuner.py --test-limit 1000 --accuracy-drop 1.5 --accuracy-slack 2.1 \
-o tuner_output alexnet2 autotuner/data/alexnet2

Supported Programs & Approximations

Programs

Currently DNN only. Support for several image processing benchmarks are in progress.

Supported DNNs:

• LeNet @ MNIST

• AlexNet @ CIFAR-10

• AlexNet2 @ CIFAR-10

• VGG16 @ CIFAR-10

• ResNet18 @ CIFAR-10

• MobileNet @ CIFAR-10

• VGG16 @ CIFAR-100

• VGG16 @ ImageNet

• ResNet50 @ ImageNet

Approximations

Currently hardware-independent approximations only. Hardware-reliant approximations are in progress.

Approximations: (output) perforation for convolution, kernel sampling for convolution.

Proxy Model

TODO: add working principle of proxy modeling.

Autotuner

We use opentuner for autontuning tasks.

Project Structure

Library

• models: PyTorch definition for DNN models

  • models/dataset: Dataset loaders for both HPVM and PyTorch-standard DNN models

  • models/hpvm: Definition for HPVM-ported models, with customized convolution layers

• toolkit: core code of project, including DNN indexing / transformations / approximations. See the code for details.

Entry Point

• ./proxy_tuner.py: perform autotuning for a given model, accuracy threshold, and a number of iterations, using a proxy model that predicts the accuracy of approximated DNN (instead of running an inference, which can be slow).

• ./run_proxy_tuner.py: run autotuning for all models defined in utils/tuner_postprocess/benchmarks.py on a set of 3 accuracy thresholds, and perform postprocessing such as computing pareto curve.

  This is the right end-to-end script to use for obtaining a comprehensive set of autotuner results.

Other Code

• tests: runnable scripts that can be used as tests (and other actual functionalities)

• utils: helper functions for library and autotuner that are generally standalone, except

  • utils/utils.py contains some convenient wrapper for model training, etc. that depends on the library.

Data

• autotuner/data: descriptions for each DNN model, such as listing of layers, tunable knobs, etc.