-
- Downloads
"git@gitlab.engr.illinois.edu:cs525-sp18-g07/spark.git" did not exist on "d12330bd2cd354919c414ee7ef367bc28c0eb7c9"
Pruning with virtual Batch-norm statistics folding (#415)
* pruning: add an option to virtually fold BN into Conv2D for ranking PruningPolicy can be configured using a new control argument fold_batchnorm: when set to `True`, the weights of BatchNorm modules are folded into the weights of Conv-2D modules (if Conv2D->BN edges exist in the model graph). Each weights filter is attenuated using a different pair of (gamma, beta) coefficients, so `fold_batchnorm` is relevant for fine-grained and filter-ranking pruning methods. We attenuate using the running values of the mean and variance, as is done in quantization. This control argument is only supported for Conv-2D modules (i.e. other convolution operation variants and Linear operations are not supported). e.g.: policies: - pruner: instance_name : low_pruner args: fold_batchnorm: True starting_epoch: 0 ending_epoch: 30 frequency: 2 * AGP: non-functional refactoring distiller/pruning/automated_gradual_prun...
Showing
- distiller/norms.py 13 additions, 13 deletionsdistiller/norms.py
- distiller/policy.py 69 additions, 5 deletionsdistiller/policy.py
- distiller/pruning/automated_gradual_pruner.py 6 additions, 5 deletionsdistiller/pruning/automated_gradual_pruner.py
- distiller/pruning/ranked_structures_pruner.py 7 additions, 57 deletionsdistiller/pruning/ranked_structures_pruner.py
- distiller/thresholding.py 6 additions, 3 deletionsdistiller/thresholding.py
- examples/agp-pruning/resnet20_filters.bn_fold.schedule_agp.yaml 140 additions, 0 deletions...es/agp-pruning/resnet20_filters.bn_fold.schedule_agp.yaml
- examples/agp-pruning/resnet20_filters.schedule_agp.yaml 53 additions, 51 deletionsexamples/agp-pruning/resnet20_filters.schedule_agp.yaml
- examples/agp-pruning/resnet20_filters.schedule_agp_2.yaml 44 additions, 45 deletionsexamples/agp-pruning/resnet20_filters.schedule_agp_2.yaml
- examples/agp-pruning/resnet20_filters.schedule_agp_3.yaml 42 additions, 44 deletionsexamples/agp-pruning/resnet20_filters.schedule_agp_3.yaml
- examples/agp-pruning/resnet20_filters.schedule_agp_4.yaml 45 additions, 40 deletionsexamples/agp-pruning/resnet20_filters.schedule_agp_4.yaml
- tests/test_pruning.py 2 additions, 2 deletionstests/test_pruning.py
Loading
Please register or sign in to comment