- Apr 08, 2019
-
-
Lev Zlotnik authored
-
- Dec 01, 2018
-
-
Neta Zmora authored
This commit contains the main fix for issue #85. It contains a couple of changes to the YAML structure pruning API, with examples. I urge you to read the documentation in the Wiki (https://github.com/NervanaSystems/distiller/wiki/Pruning-Filters-&-Channels). New syntax for defining Structured AGP. I tried to make the syntax similar to fine-grained (i.e. element-wise) pruning. All you need to do is add: ```group_type: Filters```. ``` low_pruner: class: L1RankedStructureParameterPruner_AGP initial_sparsity : 0.10 final_sparsity: 0.50 group_type: Filters weights: [module.layer3.0.conv2.weight, module.layer3.0.downsample.0.weight, module.layer3.1.conv2.weight, module.layer3.2.conv2.weight] ``` If you want to define “leader-based” pruning dependencies, add ```group_dependency: Leader```: ``` low_pruner: class: L1RankedStructureParameterPruner_AGP initial_sparsity : 0.10 final_sparsity: 0.50 group_type: Filters group_dependency: Leader weights: [module.layer3.0.conv2.weight, module.layer3.0.downsample.0.weight, module.layer3.1.conv2.weight, module.layer3.2.conv2.weight] ``` Retired the old ```reg_regims``` API for describing one-shot structured-pruning. The new YAML API is very similar to AGP structured-pruning, which is much better than before. The new API also allows us to describe data-dependencies when doing one-shot structure pruning, just like AGP structured-pruning. This commit also includes further code refactoring. Old API: ``` filter_pruner: class: 'L1RankedStructureParameterPruner' reg_regims: 'module.layer1.0.conv1.weight': [0.6, '3D'] 'module.layer1.1.conv1.weight': [0.6, '3D'] ``` New API: ``` filter_pruner: class: 'L1RankedStructureParameterPruner' group_type: Filters desired_sparsity: 0.6 weights: [ module.layer1.0.conv1.weight, module.layer1.1.conv1.weight] ``` thresholding.py – separate the generation of the binary_map from the pruning_mask so that we can cache the binary map and share it between several modules. pruning/automated_gradual_pruner.py – major refactoring to supported “leader-based” sub-graph pruning dependencies. The concept is explained in issue #85 agp-pruning/resnet20_filters.schedule_agp.yaml agp-pruning/resnet20_filters.schedule_agp_2.yaml agp-pruning/resnet20_filters.schedule_agp_3.yaml network_trimming/resnet56_cifar_activation_apoz.yaml network_trimming/resnet56_cifar_activation_apoz_v2.yaml
-
- Jul 13, 2018
-
-
Neta Zmora authored
This is a merge of the ADC branch and master. ADC (using a DDPG RL agent to compress image classifiers) is still WiP and requires An unreleased version of Coach (https://github.com/NervanaSystems/coach). Small features in this commit: -Added model_find_module() - find module object given its name - Add channel ranking and pruning: pruning/ranked_structures_pruner.py - Add a CIFAR10 VGG16 model: models/cifar10/vgg_cifar.py - Thinning: change the level of some log messages – some of the messages were moved to ‘debug’ level because they are not usually interesting. - Add a function to print nicely formatted integers - distiller/utils.py - Sensitivity analysis for channels-removal - compress_classifier.py – handle keyboard interrupts - compress_classifier.py – fix re-raise of exceptions, so they maintain call-stack -Added tests: -- test_summarygraph.py: test_simplenet() - Added a regression test to target a bug that occurs when taking the predecessor of the first node in a graph -- test_ranking.py - test_ch_ranking, test_ranked_channel_pruning -- test_model_summary.py - test_png_generation, test_summary (sparsity/ compute/model/modules) - Bug fixes in this commit: -- Thinning bug fix: handle zero-sized 'indices' tensor During the thinning process, the 'indices' tensor can become zero-sized, and will have an undefiend length. Therefore, we need to check for this situation when assessing the number of elements in 'indices' -- Language model: adjust main.py to new distiller.model_summary API
-