AMC: AutoML for Model Compression and Acceleration on Mobile Devices
This is a loose implementation of AMC, that allows you to change models and agents and use Jupyter notebooks to examine the learning process in detail. However details may note exactly match the implementation from the original paper.
Caveats:
- This is still a WiP and some information is still missing, such as setup instruction and further documentation. These will be added with time.
- Some Jupyter examples may contain mistakes in the example command-lines. Please let us know of any mistakes and bugs you find.
Notebook examples of AMC applied to compress CNNs:
These notebooks will help visualize and review the results of recreating the results of AMC (resource-constrained compression.).
- AMC worksheet: work through visualizations of different aspects of AMC learning
- Random
- Plain20
- Resnet20
- Resnet 56
- Mobilenet v1
- Mobilenet v2
At a glance
| Model | FLOPs | Baseline Top1 | AMC paper (Top1 after FT) |
Baseline Top1 | AMC Distiller |
|---|---|---|---|---|---|
| Plain20 | 50% | 90.5 | 90.2 (-0.3) | 90.55 | 90.04 (-0.4) |
| Resnet20 | 50% | 91.78 | - | ||
| Resnet56 | 50% | 92.8 | 91.9 (-0.9) | ||
| Mobilenet v1 | 50% | 70.6 | 70.2 (-0.4) | ||
| Mobilenet v2 | 50% | 71.8 | 70.8 (-1.0) |
AMC [1] trains a Deep Deterministic Policy Gradient (DDPG) RL agent to compress DNNs. For the implementation of DDPG we're using Coach or Prof. Han's team's implementation . We wrap Distiller in an OpenAI Gym environment API so that it presents a standard RL environment to the agent. We execute Distiller's sample application amc.py with special arguments telling it how to invoke AMC. This creates a DistillerWrapperEnvironment environment and a DDPG agent and starts the training cycle. For the exact technical details of doing this yourself, see the notebooks (better instructions will be added in the future).
We thank Prof. Song Han and his team for their help with certain critical parts of this implementation. However, all bugs in interpretation and/or implementation are ours ;-).
[1] AMC: AutoML for Model Compression and Acceleration on Mobile Devices.
Yihui He, Ji Lin, Zhijian Liu, Hanrui Wang, Li-Jia Li, Song Han.
In Proceedings of the European Conference on Computer Vision (ECCV), 2018.
arXiv:1802.03494