Skip to content
Snippets Groups Projects
user avatar
Neta Zmora authored
A recent commit changed the sorting of the best performing training
epochs to be based on the sparsity level of the model, then its
Top1 and Top5 scores.
When we create thinned models, the sparsity remains low (even zero),
while the physical size of the network is smaller.
This commit changes the sorting criteria to be based on the count
of non-zero (NNZ) parameters.  This captures both sparsity and
parameter size objectives:
- When sparsity is high, the number of NNZ params is low
(params_nnz_cnt = sparsity * params_cnt).
- When we remove structures (thinnning), the sparsity may remain
constant, but the count of params (params_cnt) is lower, and therefore,
once again params_nnz_cnt is lower.

Therefore, params_nnz_cnt is a good proxy to capture a sparsity
objective and/or a thinning objective.
9cb0dd68
History