-
- Downloads
Refine pruning logic (#222)
Add finer control over the pruning logic, to accommodate more pruning use-cases. The full description of the new logic is available in the updated [documentation of the CompressionScheduler](https://nervanasystems.github.io/distiller/schedule.html#pruning-fine-control), which is also part of this PR. In this PR: * Added a new callback to the CompressionScheduler: compression_scheduler.before_parameter_optimization which is invoked after the gradients are are computed, but before the weights are updated by the optimizer. * We provide an option to mask the gradients, before the weights are updated by the optimizer. We register to the parameter backward hook in order to mask the gradients. This gives us finer control over the parameter updates. * Added several DropFilter schedules. DropFilter is a method to regularize networks, and it can also be used to "prepare" a network for permanent filter pruning. *Added documentation of pruning fine-control
Showing
- distiller/policy.py 68 additions, 37 deletionsdistiller/policy.py
- distiller/pruning/ranked_structures_pruner.py 0 additions, 1 deletiondistiller/pruning/ranked_structures_pruner.py
- distiller/scheduler.py 36 additions, 19 deletionsdistiller/scheduler.py
- distiller/sensitivity.py 1 addition, 1 deletiondistiller/sensitivity.py
- docs-src/README.md 1 addition, 1 deletiondocs-src/README.md
- docs-src/docs/earlyexit.md 0 additions, 45 deletionsdocs-src/docs/earlyexit.md
- docs-src/docs/imgs/pruning_algorithm_pseudo_code.png 0 additions, 0 deletionsdocs-src/docs/imgs/pruning_algorithm_pseudo_code.png
- docs-src/docs/imgs/pruning_masking.png 0 additions, 0 deletionsdocs-src/docs/imgs/pruning_masking.png
- docs-src/docs/schedule.md 122 additions, 0 deletionsdocs-src/docs/schedule.md
- docs-src/mkdocs.yml 1 addition, 1 deletiondocs-src/mkdocs.yml
- docs/earlyexit.html 0 additions, 253 deletionsdocs/earlyexit.html
- docs/index.html 1 addition, 1 deletiondocs/index.html
- docs/schedule.html 123 additions, 0 deletionsdocs/schedule.html
- docs/search/search_index.json 1 addition, 1 deletiondocs/search/search_index.json
- docs/sitemap.xml 17 additions, 17 deletionsdocs/sitemap.xml
- docs/sitemap.xml.gz 0 additions, 0 deletionsdocs/sitemap.xml.gz
- docs/tutorial-lang_model.html 2 additions, 2 deletionsdocs/tutorial-lang_model.html
- examples/classifier_compression/compress_classifier.py 3 additions, 0 deletionsexamples/classifier_compression/compress_classifier.py
- examples/drop_filter/plain20_cifar_dropfilter_training.yaml 71 additions, 0 deletionsexamples/drop_filter/plain20_cifar_dropfilter_training.yaml
- examples/drop_filter/plain20_cifar_dropfilter_training_regularization.yaml 71 additions, 0 deletions...ter/plain20_cifar_dropfilter_training_regularization.yaml
Loading
Please register or sign in to comment