-
- Downloads
Important changes to pruning channels and filters (#93)
This commit contains the main fix for issue #85. It contains a couple of changes to the YAML structure pruning API, with examples. I urge you to read the documentation in the Wiki (https://github.com/NervanaSystems/distiller/wiki/Pruning-Filters-&-Channels). New syntax for defining Structured AGP. I tried to make the syntax similar to fine-grained (i.e. element-wise) pruning. All you need to do is add: ```group_type: Filters```. ``` low_pruner: class: L1RankedStructureParameterPruner_AGP initial_sparsity : 0.10 final_sparsity: 0.50 group_type: Filters weights: [module.layer3.0.conv2.weight, module.layer3.0.downsample.0.weight, module.layer3.1.conv2.weight, module.layer3.2.conv2.weight] ``` If you want to define “leader-based” pruning dependencies, add ```group_dependency: Leader```: ``` low_pruner: class: L1RankedStructureParameterPruner_AGP initial_sparsity : 0.10 final_sparsity: 0.50 group_type: Filters group_dependency: Leader weights: [module.layer3.0.conv2.weight, module.layer3.0.downsample.0.weight, module.layer3.1.conv2.weight, module.layer3.2.conv2.weight] ``` Retired the old ```reg_regims``` API for describing one-shot structured-pruning. The new YAML API is very similar to AGP structured-pruning, which is much better than before. The new API also allows us to describe data-dependencies when doing one-shot structure pruning, just like AGP structured-pruning. This commit also includes further code refactoring. Old API: ``` filter_pruner: class: 'L1RankedStructureParameterPruner' reg_regims: 'module.layer1.0.conv1.weight': [0.6, '3D'] 'module.layer1.1.conv1.weight': [0.6, '3D'] ``` New API: ``` filter_pruner: class: 'L1RankedStructureParameterPruner' group_type: Filters desired_sparsity: 0.6 weights: [ module.layer1.0.conv1.weight, module.layer1.1.conv1.weight] ``` thresholding.py – separate the generation of the binary_map from the pruning_mask so that we can cache the binary map and share it between several modules. pruning/automated_gradual_pruner.py – major refactoring to supported “leader-based” sub-graph pruning dependencies. The concept is explained in issue #85 agp-pruning/resnet20_filters.schedule_agp.yaml agp-pruning/resnet20_filters.schedule_agp_2.yaml agp-pruning/resnet20_filters.schedule_agp_3.yaml network_trimming/resnet56_cifar_activation_apoz.yaml network_trimming/resnet56_cifar_activation_apoz_v2.yaml
Showing
- distiller/pruning/automated_gradual_pruner.py 70 additions, 57 deletionsdistiller/pruning/automated_gradual_pruner.py
- distiller/pruning/ranked_structures_pruner.py 183 additions, 122 deletionsdistiller/pruning/ranked_structures_pruner.py
- distiller/sensitivity.py 10 additions, 5 deletionsdistiller/sensitivity.py
- distiller/thinning.py 59 additions, 31 deletionsdistiller/thinning.py
- distiller/thresholding.py 64 additions, 10 deletionsdistiller/thresholding.py
- examples/agp-pruning/resnet20_filters.schedule_agp.yaml 8 additions, 27 deletionsexamples/agp-pruning/resnet20_filters.schedule_agp.yaml
- examples/agp-pruning/resnet20_filters.schedule_agp_2.yaml 10 additions, 16 deletionsexamples/agp-pruning/resnet20_filters.schedule_agp_2.yaml
- examples/agp-pruning/resnet20_filters.schedule_agp_3.yaml 10 additions, 16 deletionsexamples/agp-pruning/resnet20_filters.schedule_agp_3.yaml
- examples/agp-pruning/resnet20_filters.schedule_agp_4.yaml 154 additions, 0 deletionsexamples/agp-pruning/resnet20_filters.schedule_agp_4.yaml
- examples/classifier_compression/logging.conf 6 additions, 6 deletionsexamples/classifier_compression/logging.conf
- examples/network_trimming/resnet56_cifar_activation_apoz.yaml 30 additions, 41 deletions...ples/network_trimming/resnet56_cifar_activation_apoz.yaml
- examples/network_trimming/resnet56_cifar_activation_apoz_v2.yaml 29 additions, 26 deletions...s/network_trimming/resnet56_cifar_activation_apoz_v2.yaml
- examples/pruning_filters_for_efficient_convnets/resnet56_cifar_channel_rank.yaml 189 additions, 0 deletions...s_for_efficient_convnets/resnet56_cifar_channel_rank.yaml
- examples/pruning_filters_for_efficient_convnets/resnet56_cifar_filter_rank.yaml 54 additions, 60 deletions...rs_for_efficient_convnets/resnet56_cifar_filter_rank.yaml
- examples/pruning_filters_for_efficient_convnets/resnet56_cifar_filter_rank_v2.yaml 54 additions, 60 deletions...for_efficient_convnets/resnet56_cifar_filter_rank_v2.yaml
- examples/pruning_filters_for_efficient_convnets/vgg19.schedule_filter_rank.yaml 21 additions, 18 deletions...rs_for_efficient_convnets/vgg19.schedule_filter_rank.yaml
- imgs/pruning_structs_ex1.png 0 additions, 0 deletionsimgs/pruning_structs_ex1.png
- imgs/pruning_structs_ex2.png 0 additions, 0 deletionsimgs/pruning_structs_ex2.png
- imgs/pruning_structs_ex3.png 0 additions, 0 deletionsimgs/pruning_structs_ex3.png
- imgs/pruning_structs_ex4.png 0 additions, 0 deletionsimgs/pruning_structs_ex4.png
Loading
Please register or sign in to comment