-
- Downloads
Post-Train Quant: Greedy Search + Proper mixed-settings handling (#402)
* Greedy search script for post-training quantization settings * Iterates over each layer in the model in order. For each layer, checks a user-defined set of quantization settings and chooses the best one based on validation accuracy * Provided sample that searches for best activations-clipping mode per layer, on image classification models * Proper handling of mixed-quantization settings in post-train quant: * By default, the quantization settings for each layer apply only to output quantization * Propagate quantization settings for activations tensors through the model during execution * For non-quantized inputs to layers that require quantized inputs, fall-back to quantizing according to the settings used for the output * In addition, provide mechanism to override inputs quantization settings via the YAML configuration file * By default all modules are quantized now. For module types that don't have a dedicated quantized implementation, "fake" quantization is performed * Misc. Changes * Fuse ReLU/ReLU6 to predecessor during post-training quantization * Fixes to ACIQ clipping in the half-range case Co-authored-by:Lev Zlotnik <lev.zlotnik@intel.com> Co-authored-by:
Guy Jacob <guy.jacob@intel.com>
Showing
- distiller/data_loggers/collector.py 15 additions, 6 deletionsdistiller/data_loggers/collector.py
- distiller/model_transforms.py 8 additions, 4 deletionsdistiller/model_transforms.py
- distiller/models/__init__.py 35 additions, 8 deletionsdistiller/models/__init__.py
- distiller/models/cifar10/resnet_cifar.py 3 additions, 1 deletiondistiller/models/cifar10/resnet_cifar.py
- distiller/modules/__init__.py 2 additions, 2 deletionsdistiller/modules/__init__.py
- distiller/modules/aggregate.py 10 additions, 0 deletionsdistiller/modules/aggregate.py
- distiller/quantization/ptq_greedy_search.py 488 additions, 0 deletionsdistiller/quantization/ptq_greedy_search.py
- distiller/quantization/quantizer.py 28 additions, 16 deletionsdistiller/quantization/quantizer.py
- distiller/quantization/range_linear.py 525 additions, 297 deletionsdistiller/quantization/range_linear.py
- distiller/summary_graph.py 98 additions, 12 deletionsdistiller/summary_graph.py
- distiller/utils.py 32 additions, 0 deletionsdistiller/utils.py
- examples/quantization/post_train_quant/resnet18_imagenet_post_train_input_overrides.yaml 75 additions, 0 deletions...n_quant/resnet18_imagenet_post_train_input_overrides.yaml
- tests/full_flow_tests.py 1 addition, 1 deletiontests/full_flow_tests.py
- tests/test_model_transforms.py 2 additions, 2 deletionstests/test_model_transforms.py
- tests/test_post_train_quant.py 46 additions, 14 deletionstests/test_post_train_quant.py
- tests/test_quantizer.py 0 additions, 7 deletionstests/test_quantizer.py
- tests/test_summarygraph.py 2 additions, 2 deletionstests/test_summarygraph.py
Loading
Please register or sign in to comment