-
Neta Zmora authored
Group-wise convolutions with num_groups == num_in_channels, as configured in MobileNet for example, create attribute and shape dependency chains that are longer than convolutions with num_groups == 1. For example in the sequence below, changing the number of filters of the weights of Conv1, triggers changes in BN1, Conv2, BN2, and Conv3 (g indicates the number of groups): Conv1(g=1) => BN1 ==> Conv2(g=32) ==> BN2 ==> Conv3(g=1) Changing the number of filters used in Conv1 affects the parameters and attributes of BN1 and Conv2 - The number of input channels of Conv2 is changed.as explained in https://nervanasystems.github.io/distiller/tutorial-struct_pruning.html. However, since Conv2 has num_groups == num_in_channels, we have to change num_groups, which triggers a change in num_out_channels. This is akin to changing the number of filters of Conv2, which triggers a change in BN2 and Conv3. models/mobilenet.py: Changed the code that flattens the output of the features-extractors and prepares it as input to the classifier. The code was written using hard-coded shape values, which made it impossible to use in thinned models (where dimensions are changed). tests/test_pruning.py: Added test for thinning MobileNet (grouped convolutions)
Neta Zmora authoredGroup-wise convolutions with num_groups == num_in_channels, as configured in MobileNet for example, create attribute and shape dependency chains that are longer than convolutions with num_groups == 1. For example in the sequence below, changing the number of filters of the weights of Conv1, triggers changes in BN1, Conv2, BN2, and Conv3 (g indicates the number of groups): Conv1(g=1) => BN1 ==> Conv2(g=32) ==> BN2 ==> Conv3(g=1) Changing the number of filters used in Conv1 affects the parameters and attributes of BN1 and Conv2 - The number of input channels of Conv2 is changed.as explained in https://nervanasystems.github.io/distiller/tutorial-struct_pruning.html. However, since Conv2 has num_groups == num_in_channels, we have to change num_groups, which triggers a change in num_out_channels. This is akin to changing the number of filters of Conv2, which triggers a change in BN2 and Conv3. models/mobilenet.py: Changed the code that flattens the output of the features-extractors and prepares it as input to the classifier. The code was written using hard-coded shape values, which made it impossible to use in thinned models (where dimensions are changed). tests/test_pruning.py: Added test for thinning MobileNet (grouped convolutions)