-
- Downloads
Post-Train Quantization: BN folding and "net-aware quantization" (#313)
* "Net-aware quantization" - using the term coined in https://arxiv.org/abs/1811.09886. (section 3.2.2). Refers to considering sequences of modules when quantizing. This isn't exactly layer fusion - we modify activation stats prior to setting quantization parameters, to make sure that when a module is followed by certain activation functions, only the relevant ranges are quantized. We do this for: * ReLU - Clip all negative values * Tanh / Sigmoid - Clip according to the (approximated) saturation values for these functions. We use [-4, 4] for tanh and [-6, 6] for sigmoid. * Perform batch-norm folding before post-training quantization. Batch-norm parameters are folded into the parameters of the previous layer and the BN layer is replaced with an identity module. * Both BN folding and "net-aware" are now automatically executed in PostTrainLinearQuantizer (details of this change below) * BN folding enabled by new generic mechanism to "fuse" module sequences (at the Python API level) * First module in sequence is replaced/modified by a user-provided function, rest of moudles replaced with nn.Identity * Quantizer changes: * Optionally create adjacency map during prepare_model * Subclasses may enforce adjacency map creation * Refatcoring: Replace _prepare_model_impl with pre and post override-able "callbacks", so core functionality is always executed * PostTrainLinearQuantizer Changes: * Enforce creation of adjacency map. This means users must now pass a dummy input to PostTrainLinearQuantizer.prepare_model * Before module replacement - Apply BN folding and stats updates according to net-aware quantization * Updated the language model quantization tutorial to reflect the new functionality * Updated the image classification post-train quantization samples (command line and YAML) * Other changes: * Distller LSTM implementation: Replace the ModuleList for cells with a plain list. The PyTorch trace mechanism doesn't "see" ModuleList objects, it only sees the contained modules. This means that the "scopeName" of these modules isn't complete, which makes it impossible to match op names in SummaryGraph to modules in the Python model. * ActivationStatsCollector: Ignore nn.Identity modules
Showing
- README.md 1 addition, 0 deletionsREADME.md
- distiller/data_loggers/collector.py 6 additions, 4 deletionsdistiller/data_loggers/collector.py
- distiller/model_transforms.py 141 additions, 0 deletionsdistiller/model_transforms.py
- distiller/modules/rnn.py 30 additions, 17 deletionsdistiller/modules/rnn.py
- distiller/quantization/quantizer.py 31 additions, 15 deletionsdistiller/quantization/quantizer.py
- distiller/quantization/range_linear.py 109 additions, 5 deletionsdistiller/quantization/range_linear.py
- docs-src/docs/tutorial-lang_model_quant.md 1 addition, 0 deletionsdocs-src/docs/tutorial-lang_model_quant.md
- docs/index.html 1 addition, 1 deletiondocs/index.html
- docs/search/search_index.json 1 addition, 1 deletiondocs/search/search_index.json
- docs/sitemap.xml.gz 0 additions, 0 deletionsdocs/sitemap.xml.gz
- docs/tutorial-lang_model_quant.html 1 addition, 0 deletionsdocs/tutorial-lang_model_quant.html
- examples/classifier_compression/compress_classifier.py 1 addition, 1 deletionexamples/classifier_compression/compress_classifier.py
- examples/quantization/post_train_quant/command_line.md 69 additions, 34 deletionsexamples/quantization/post_train_quant/command_line.md
- examples/quantization/post_train_quant/resnet18_imagenet_post_train.yaml 64 additions, 25 deletions...zation/post_train_quant/resnet18_imagenet_post_train.yaml
- examples/quantization/post_train_quant/stats/resnet18_quant_stats.yaml 1 addition, 1 deletion...tization/post_train_quant/stats/resnet18_quant_stats.yaml
- examples/quantization/post_train_quant/stats/resnet50_quant_stats.yaml 3271 additions, 0 deletions...tization/post_train_quant/stats/resnet50_quant_stats.yaml
- examples/word_language_model/manual_lstm_pretrained_stats.yaml 24 additions, 24 deletions...les/word_language_model/manual_lstm_pretrained_stats.yaml
- examples/word_language_model/manual_lstm_pretrained_stats_new.yaml 0 additions, 566 deletions...word_language_model/manual_lstm_pretrained_stats_new.yaml
- examples/word_language_model/quantize_lstm.ipynb 263 additions, 110 deletionsexamples/word_language_model/quantize_lstm.ipynb
- tests/common.py 10 additions, 1 deletiontests/common.py
Loading
Please register or sign in to comment