Activation statistics: improve documentation

docs-src/docs/usage.md

@@ -238,22 +238,34 @@ The ```test``` parameter indicates that, in this example, we want to collect act
 An instance of a subclass of ```ActivationStatsCollector``` can be used to collect activation statistics.  Currently, ```ActivationStatsCollector``` has two types of subclasses: ```SummaryActivationStatsCollector``` and ```RecordsActivationStatsCollector```.<br>
 Instances of ```SummaryActivationStatsCollector``` compute the mean of some statistic of the activation.  It is rather
 light-weight and quicker than collecting a record per activation.  The statistic function is configured in the constructor.<br>
-In ```compress_classifier.py``` we create a dictionary of collectors.  For example, this collector collects statistics that is stored in each relevant module, in a variable named ```sparsity```.  The lambda expression is invoked per activation encountered during forward passes, and the value it returns (in this case, the sparsity of the activation tensors) is stored in ```module.sparsity``` (*"sparsity"* is this collector's name).
+In the sample compression application, ```compress_classifier.py```, we create a dictionary of collectors.  For example:
 ```
 SummaryActivationStatsCollector(model,
                                 "sparsity",
                                 lambda t: 100 * distiller.utils.sparsity(t))
 ```
-To access the statistics, you can invoke ```collector.value()```, or you can access each module's data directly.<br>
-You can add other statistics collectors and use a different function to compute your new statistic.<br>
+The lambda expression is invoked per activation encountered during forward passes, and the value it returns (in this case, the sparsity of the activation tensors, multiplied by 100) is stored in ```module.sparsity``` (*"sparsity"* is this collector's name).  To access the statistics, you can invoke ```collector.value()```, or you can access each module's data directly.
+
 Another type of collector is ```RecordsActivationStatsCollector``` which computes a hard-coded set of activations statistics and collects a
 *record per activation*.  For obvious reasons, this is slower than instances of ```SummaryActivationStatsCollector```.<br>```ActivationStatsCollector``` default to collecting activations statistics only on the output activations of ReLU layers, but we can choose any layer type we want.  In the example below we collect statistics from outputs of ```torch.nn.Conv2d``` layers.
 ```  
 RecordsActivationStatsCollector(model, classes=[torch.nn.Conv2d])
 ```
 
-Collectors can write their data to Excel workbooks, by invoking ```collector.to_xlsx(path_to_workbook)```, which are named using the collector's name.<br>
-You can use a utility function, ```distiller.log_activation_statsitics```, to log the data of an ```ActivationStatsCollector``` instance to one of the loggers.  For an example, the code below logs the *"sparsity"* collector to a TensorBoard log file.
+Collectors can write their data to Excel workbooks (which are named using the collector's name), by invoking ```collector.to_xlsx(path_to_workbook)```.  In ```compress_classifier.py``` we currently create four different collectors which you can selectively disable.  You can also add other statistics collectors and use a different function to compute your new statistic.
+```
+collectors = missingdict({
+    "sparsity":      SummaryActivationStatsCollector(model, "sparsity",
+                                                     lambda t: 100 * distiller.utils.sparsity(t)),
+    "l1_channels":   SummaryActivationStatsCollector(model, "l1_channels",
+                                                     distiller.utils.activation_channels_l1),
+    "apoz_channels": SummaryActivationStatsCollector(model, "apoz_channels",
+                                                     distiller.utils.activation_channels_apoz),
+    "records":       RecordsActivationStatsCollector(model, classes=[torch.nn.Conv2d])})
+```
+By default, these Collectors write their data to files in the active log directory.
+
+You can use a utility function, ```distiller.log_activation_statsitics```, to log the data of an ```ActivationStatsCollector``` instance to one of the backend-loggers.  For an example, the code below logs the *"sparsity"* collector to a TensorBoard log file.
 ```
 distiller.log_activation_statsitics(epoch, "train", loggers=[tflogger],
                                     collector=collectors["sparsity"])

docs/index.html

@@ -258,5 +258,5 @@ And of course, if we used a sparse or compressed representation, then we are red
 
 <!--
 MkDocs version : 0.17.2
-Build Date UTC : 2018-11-24 09:47:02
+Build Date UTC : 2018-11-25 09:04:15
 -->

docs/sitemap.xml

@@ -4,7 +4,7 @@
     
     <url>
      <loc>/index.html</loc>
-     <lastmod>2018-11-24</lastmod>
+     <lastmod>2018-11-25</lastmod>
      <changefreq>daily</changefreq>
     </url>
     
@@ -12,7 +12,7 @@
     
     <url>
      <loc>/install/index.html</loc>
-     <lastmod>2018-11-24</lastmod>
+     <lastmod>2018-11-25</lastmod>
      <changefreq>daily</changefreq>
     </url>
     
@@ -20,7 +20,7 @@
     
     <url>
      <loc>/usage/index.html</loc>
-     <lastmod>2018-11-24</lastmod>
+     <lastmod>2018-11-25</lastmod>
      <changefreq>daily</changefreq>
     </url>
     
@@ -28,7 +28,7 @@
     
     <url>
      <loc>/schedule/index.html</loc>
-     <lastmod>2018-11-24</lastmod>
+     <lastmod>2018-11-25</lastmod>
      <changefreq>daily</changefreq>
     </url>
     
@@ -37,31 +37,31 @@
         
     <url>
      <loc>/pruning/index.html</loc>
-     <lastmod>2018-11-24</lastmod>
+     <lastmod>2018-11-25</lastmod>
      <changefreq>daily</changefreq>
     </url>
         
     <url>
      <loc>/regularization/index.html</loc>
-     <lastmod>2018-11-24</lastmod>
+     <lastmod>2018-11-25</lastmod>
      <changefreq>daily</changefreq>
     </url>
         
     <url>
      <loc>/quantization/index.html</loc>
-     <lastmod>2018-11-24</lastmod>
+     <lastmod>2018-11-25</lastmod>
      <changefreq>daily</changefreq>
     </url>
         
     <url>
      <loc>/knowledge_distillation/index.html</loc>
-     <lastmod>2018-11-24</lastmod>
+     <lastmod>2018-11-25</lastmod>
      <changefreq>daily</changefreq>
     </url>
         
     <url>
      <loc>/conditional_computation/index.html</loc>
-     <lastmod>2018-11-24</lastmod>
+     <lastmod>2018-11-25</lastmod>
      <changefreq>daily</changefreq>
     </url>
         
@@ -71,19 +71,19 @@
         
     <url>
      <loc>/algo_pruning/index.html</loc>
-     <lastmod>2018-11-24</lastmod>
+     <lastmod>2018-11-25</lastmod>
      <changefreq>daily</changefreq>
     </url>
         
     <url>
      <loc>/algo_quantization/index.html</loc>
-     <lastmod>2018-11-24</lastmod>
+     <lastmod>2018-11-25</lastmod>
      <changefreq>daily</changefreq>
     </url>
         
     <url>
      <loc>/algo_earlyexit/index.html</loc>
-     <lastmod>2018-11-24</lastmod>
+     <lastmod>2018-11-25</lastmod>
      <changefreq>daily</changefreq>
     </url>
         
@@ -92,7 +92,7 @@
     
     <url>
      <loc>/model_zoo/index.html</loc>
-     <lastmod>2018-11-24</lastmod>
+     <lastmod>2018-11-25</lastmod>
      <changefreq>daily</changefreq>
     </url>
     
@@ -100,7 +100,7 @@
     
     <url>
      <loc>/jupyter/index.html</loc>
-     <lastmod>2018-11-24</lastmod>
+     <lastmod>2018-11-25</lastmod>
      <changefreq>daily</changefreq>
     </url>
     
@@ -108,7 +108,7 @@
     
     <url>
      <loc>/design/index.html</loc>
-     <lastmod>2018-11-24</lastmod>
+     <lastmod>2018-11-25</lastmod>
      <changefreq>daily</changefreq>
     </url>
     

docs/usage/index.html

@@ -424,21 +424,31 @@ For example:</p>
 <p>An instance of a subclass of <code>ActivationStatsCollector</code> can be used to collect activation statistics.  Currently, <code>ActivationStatsCollector</code> has two types of subclasses: <code>SummaryActivationStatsCollector</code> and <code>RecordsActivationStatsCollector</code>.<br>
 Instances of <code>SummaryActivationStatsCollector</code> compute the mean of some statistic of the activation.  It is rather
 light-weight and quicker than collecting a record per activation.  The statistic function is configured in the constructor.<br>
-In <code>compress_classifier.py</code> we create a dictionary of collectors.  For example, this collector collects statistics that is stored in each relevant module, in a variable named <code>sparsity</code>.  The lambda expression is invoked per activation encountered during forward passes, and the value it returns (in this case, the sparsity of the activation tensors) is stored in <code>module.sparsity</code> (<em>"sparsity"</em> is this collector's name).</p>
+In the sample compression application, <code>compress_classifier.py</code>, we create a dictionary of collectors.  For example:</p>
 <pre><code>SummaryActivationStatsCollector(model,
                                 &quot;sparsity&quot;,
                                 lambda t: 100 * distiller.utils.sparsity(t))
 </code></pre>
 
-<p>To access the statistics, you can invoke <code>collector.value()</code>, or you can access each module's data directly.<br>
-You can add other statistics collectors and use a different function to compute your new statistic.<br>
-Another type of collector is <code>RecordsActivationStatsCollector</code> which computes a hard-coded set of activations statistics and collects a
+<p>The lambda expression is invoked per activation encountered during forward passes, and the value it returns (in this case, the sparsity of the activation tensors, multiplied by 100) is stored in <code>module.sparsity</code> (<em>"sparsity"</em> is this collector's name).  To access the statistics, you can invoke <code>collector.value()</code>, or you can access each module's data directly.</p>
+<p>Another type of collector is <code>RecordsActivationStatsCollector</code> which computes a hard-coded set of activations statistics and collects a
 <em>record per activation</em>.  For obvious reasons, this is slower than instances of <code>SummaryActivationStatsCollector</code>.<br><code>ActivationStatsCollector</code> default to collecting activations statistics only on the output activations of ReLU layers, but we can choose any layer type we want.  In the example below we collect statistics from outputs of <code>torch.nn.Conv2d</code> layers.</p>
 <pre><code>RecordsActivationStatsCollector(model, classes=[torch.nn.Conv2d])
 </code></pre>
 
-<p>Collectors can write their data to Excel workbooks, by invoking <code>collector.to_xlsx(path_to_workbook)</code>, which are named using the collector's name.<br>
-You can use a utility function, <code>distiller.log_activation_statsitics</code>, to log the data of an <code>ActivationStatsCollector</code> instance to one of the loggers.  For an example, the code below logs the <em>"sparsity"</em> collector to a TensorBoard log file.</p>
+<p>Collectors can write their data to Excel workbooks (which are named using the collector's name), by invoking <code>collector.to_xlsx(path_to_workbook)</code>.  In <code>compress_classifier.py</code> we currently create four different collectors which you can selectively disable.  You can also add other statistics collectors and use a different function to compute your new statistic.</p>
+<pre><code>collectors = missingdict({
+    &quot;sparsity&quot;:      SummaryActivationStatsCollector(model, &quot;sparsity&quot;,
+                                                     lambda t: 100 * distiller.utils.sparsity(t)),
+    &quot;l1_channels&quot;:   SummaryActivationStatsCollector(model, &quot;l1_channels&quot;,
+                                                     distiller.utils.activation_channels_l1),
+    &quot;apoz_channels&quot;: SummaryActivationStatsCollector(model, &quot;apoz_channels&quot;,
+                                                     distiller.utils.activation_channels_apoz),
+    &quot;records&quot;:       RecordsActivationStatsCollector(model, classes=[torch.nn.Conv2d])})
+</code></pre>
+
+<p>By default, these Collectors write their data to files in the active log directory.</p>
+<p>You can use a utility function, <code>distiller.log_activation_statsitics</code>, to log the data of an <code>ActivationStatsCollector</code> instance to one of the backend-loggers.  For an example, the code below logs the <em>"sparsity"</em> collector to a TensorBoard log file.</p>
 <pre><code>distiller.log_activation_statsitics(epoch, &quot;train&quot;, loggers=[tflogger],
                                     collector=collectors[&quot;sparsity&quot;])
 </code></pre>