Automated Gradual Pruner (AGP) Pruning Examples

Introduction

In To prune, or not to prune: exploring the efficacy of pruning for model compression, authors Michael Zhu and Suyog Gupta provide an algorithm to schedule iterative level pruning.

We introduce a new automated gradual pruning algorithm in which the sparsity is increased from an initial sparsity value (usually 0) to a final sparsity value over a span of n pruning steps. The intuition behind this sparsity function in equation (1) is to prune the network rapidly in the initial phase when the redundant connections are abundant and gradually reduce the number of weights being pruned each time as there are fewer and fewer weights remaining in the network.

The authors describe AGP:

• Our automated gradual pruning algorithm prunes the smallest magnitude weights to achieve a preset level of network sparsity.
• Doesn't require much hyper-parameter tuning
• Shown to perform well across different models
• Does not make any assumptions about the structure of the network or its constituent layers, and is therefore more generally applicable.

Distiller

• The original AGP paper described the application of AGP for fine-grained pruning, and in Distiller we also implemented AGP for structured-pruning.
• We also provide examples of applying AGP for pruning language models. The results and methodology are discussed at length in the documentation

Examples

The tables below provide the results of the experimental pruning schedules that appear in this directory. Each example YAML schedule-file contains the command-line used to execute the experiment, and further details.

Element-wise sparsity

Block sparsity

Filter pruning with thinning

Our objective here is to minimize compute but performing thinning. Therefore, sparsity is often at 0%, but the number of parameters is reduced as filters are removed.

In this table we seek to see a lower value for Parameters Kept (%) and, more importantely, Compute Kept (%).