Changed the description of the feature-set.
Updating the README a little bit since a lot has changed since we released 6 months ago.  Still a lot to add/remove/change.

* Added command line arguments for this and other post-training
  quantization settings in image classification sample.

Small improvement in the results

fix issue #65: Missing link in the README file

This short notebook performs a forward-pass on a single ImageNet image,
and saves the intermediate results to a file for later inspection

A notebook to visualize the data dependencies in non-trivial networks
such as ResNet.
This is meant to help in planning filter/channel pruning of such networks.

After commit f396c34a362731c765370d368877c2ca367ad651, we
Now always apply a pruning mask at the end of mini-batches (this is
because we understand that weights may be updated by SGD+momentum,
even for weights that are masked. Therefore, there is no need for masking
the gradients: we always mask the weights at the end of the mini-batch.

See issue #53 for more details.

This notebook provides visualizations of ResNet50 models pay attention
to when they classify. In other words, what are the models looking at.
This information can be used for localization, but here it is provided
merely to build our intuition.

Based on:
B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba. Learning
Deep Features for Discriminative Localization. CVPR'16
(arXiv:1512.04150, 2015).

https://alexisbcook.github.io/2017/global-average-pooling-layers-for-object-localization/

This is a simple application of Truncated SVD, just to get a feeling of
what happens to the accuracy if we use TruncatedSVD w/o fine-tuning.

We apply Truncated SVD on the linear layer found at the end of ResNet50,
and run a test over the validation dataset to measure the impact on the
classification accuracy.

The problem reported in issue #60 occurs when a user downloads an
archive of Distiller instead of performing "git clone".
When downloading an archive there is no ".git" directory and we
need to handle this gracefully.
For more details, see the issue itself.

Activation statistics can be leveraged to make pruning and quantization decisions, and so
We added support to collect these data.
- Two types of activation statistics are supported: summary statistics, and detailed records 
per activation.
Currently we support the following summaries: 
- Average activation sparsity, per layer
- Average L1-norm for each activation channel, per layer
- Average sparsity for each activation channel, per layer

For the detailed records we collect some statistics per activation and store it in a record.  
Using this collection method generates more detailed data, but consumes more time, so
Beware.

* You can collect activation data for the different training phases: training/validation/test.
* You can access the data directly from each module that you chose to collect stats for.  
* You can also create an Excel workbook with the stats.

To demonstrate use of activation collection we added a sample schedule which prunes 
weight filters by the activation APoZ according to:
"Network Trimming: A Data-Driven Neuron Pruning Approach towards 
Efficient Deep Architectures",
Hengyuan Hu, Rui Peng, Yu-Wing Tai, Chi-Keung Tang, ICLR 2016
https://arxiv.org/abs/1607.03250

We also refactored the AGP code (AutomatedGradualPruner) to support structure pruning,
and specifically we separated the AGP schedule from the filter pruning criterion.  We added
examples of ranking filter importance based on activation APoZ (ActivationAPoZRankedFilterPruner),
random (RandomRankedFilterPruner), filter gradients (GradientRankedFilterPruner), 
and filter L1-norm (L1RankedStructureParameterPruner)

ONNX export in PyTorch doesn't know how to handle DataParallel
layers, so we need to make sure that we remove all instances
of nn.DataParallel from the model before exporting it.

The previous ONNX implementation forgot to deal with the case
of DataParallel layers that do not wrap the entire model (as
in VGG, where only the feature-extractor layers are data-parallel).

We should only add softmax when we explicitly require it (as when
exporting to ONNX), because CrossEntropyLoss implicitly computes
softmax on the logits it receives as input.

This cade was left there by mistake and should have never been
pushed to git.

Using the automatic gradual pruning for structured-pruning is very
simple to use and produces good results.
It is implemented in Google's TensorFlow framework.

When running inference in ONNX, we often want to add a softmax
layer to TorchVision's models.

When using a schedule with epochs that have nothing scheduled for them, apply_mask() is not invoked at the end of mini-batches, and pruned weights might be unmasked by the optimizer weight updates.

See explanation in issue #53 discussion

Temporary fix for dependency on distiller class hierarchy when
serializing a model that contains a thinning recipe.

See: https://github.com/NervanaSystems/distiller/wiki/Distiller-Summary-Reports

Latest versions of Jupyter notebooks have a different syntax for
launching the server such that it listens on oll network interfaces
(this is useful if you are running the Jupyter server on one machine,
and connect to it from a browser on a different machine).

So:
	jupyter-notebook --ip=* --no-browser

is replaced by:
	jupyter-notebook --ip=0.0.0.0 --no-browser

Remove function to_var() which is not used by any code.

We need AverageValueMeter's support for numpy arrays.

Showing various details about the performance of ResNet50

Also show fitting of the histograms to the Gaussian and Laplacian
distributions.

Somehow 4 copies of the same test were pasted into this file: removed 3 instances.

* Added GSS ("Attention-Based Guided Structured Sparsity of Deep Neural Networks") and an example for ResNet20 channel pruning.
    - The idea is to regularize the variance of the distribution of the parameter structures. Some structures will zero completely and the rest should have a high value leading to a high variance.
    - A new regularizer class, GroupVarianceRegularizer, is used to regularize the group variance (effectively rewarding the loss function for high variance between the groups).
    - When tested on ResNet 20 GSS did not show any improvement over SSL

* Added sample of filter pruning for ResNet20 CIFAR using SSL (Learning Structured Sparsity in Deep Neural Networks)

* Added an example of pruning 45% of the compute (1.8x MAC reduction), while suffering 0.8% accuracy loss on ResNet20 CIFAR

* Added a ResNet50 ImageNet example of L1-Magnitude fine-grained pruning, using an AGP schedule: 46% sparsity with a 0.6% accuracy increase. This is an example of using pruning used as a regularizer.

clean-up a bit of code

In this experiment we increase the regulization_strength of some of the
channel regularization terms.
We want to increase the compute compression, while allowing some reduction
in the accuracy performance.

This schedule demonstrates low-rate pruning (26% sparsity) acting as a
regularizer to reduce the generalization error of ResNet50 using the
ImageNet dataset.

We improve the ResNet50 Top1 test error by 0.4% (23.462 vs 23.85).
Top5 is improved as well: 6.82 error vs. 7.13 error in the baseline

* Clean up PyTorch 0.3 compatibility code
We don't need this anymore and PyTorch 1.0 is just around the corner.

* explicitly place the inputs tensor on the GPU(s)

* A temporary fix for issue 36

The thinning code assumes that the sgraph it is using
is not data-parallel, because it (currently) accesses the
layer-name keys using a "normalized" name ("module." is removed).

The bug is that in thinning.py#L73 we create a data_parallel=True
model; and then give it to sgraph.
But in other places thinning code uses "normalized" keys.  For
example in thinning.py#L264.

The temporary fix configures data_parallel=False in thinning.py#L73.

A long term solution should have SummaryGraph know how to handle
both parallel and not-parallel models.  This can be done by having
SummaryGraph convert layer-names it receives in the API to
data_parallel=False using normalize_layer_name.  When returning
results, use the de-normalized format.

* Fix the documentation error from issue 36
* Move some logs to debug and show in logging.conf how to enable DEBUG logs.