Skip to content
Snippets Groups Projects
  1. Apr 08, 2019
    • Neta Zmora's avatar
      Refine pruning logic (#222) · 816a943d
      Neta Zmora authored
      Add finer control over the pruning logic, to accommodate more pruning
      use-cases.
      The full description of the new logic is available in the updated [documentation
      of the CompressionScheduler](https://nervanasystems.github.io/distiller/schedule.html#pruning-fine-control), which is also part of this PR.
      
      In this PR:
      
      * Added a new callback to the CompressionScheduler:
      compression_scheduler.before_parameter_optimization which is invoked
      after the gradients are are computed, but before the weights are updated
      by the optimizer.
      
      * We provide an option to mask the gradients, before the weights are updated by the optimizer. 
      We register to the parameter backward hook in order to mask the gradients.
      This gives us finer control over the parameter updates.
      
      * Added several DropFilter schedules.
      DropFilter is a method to regularize networks, and it can also be
      used to "prepare" a network for permanent filter pruning.
      
      *Added documentation of pruning fine-control
      Unverified
      816a943d
  2. Apr 01, 2019
    • Lev Zlotnik's avatar
      Quantizer: Specify # bias bits + custom overrides (BREAKING) (#178) · 5271625a
      Lev Zlotnik authored
      * Bias handling:
        * Add 'bits_bias' parameter to explicitly specify # of bits for bias,
          similar to weights and activations.
        * BREAKING: Remove the now redundant 'quantize_bias' boolean parameter
      * Custom overrides:
        * Expand the semantics of the overrides dict to allow overriding of
          other parameters in addition to bit-widths
        * Functions registered in the quantizer's 'replacement_factory' can
          define keyword arguments. Non bit-width entries in the overrides
          dict will be checked against the function signature and passed
        * BREAKING:
          * Changed the name of 'bits_overrides' to simply 'overrides'
          * Bit-width overrides must now be defined using the full parameter
            names - 'bits_activations/weights/bias' instead of the short-hands
            'acts' and 'wts' which were used so far.
        * Added/updated relevant tests
        * Modified all quantization YAMLs under 'examples' to reflect 
          these changes
        * Updated docs
      5271625a
  3. Mar 29, 2019
  4. Feb 26, 2019
  5. Feb 11, 2019
    • Guy Jacob's avatar
      Post-train quant based on stats + additional modules quantized (#136) · 28a8ee18
      Guy Jacob authored
      Summary of changes:
      (1) Post-train quantization based on pre-collected statistics
      (2) Quantized concat, element-wise addition / multiplication and embeddings
      (3) Move post-train quantization command line args out of sample code
      (4) Configure post-train quantization from YAML for more fine-grained control
      
      (See PR #136 for more detailed changes descriptions)
      Unverified
      28a8ee18
  6. Dec 11, 2018
  7. Dec 09, 2018
  8. Dec 06, 2018
    • Neta Zmora's avatar
      Documentation refactoring · 178c8c49
      Neta Zmora authored
      - Moved the Language model and struct pruning tutorials from the Wiki to
      the HTML documentation.  Love the ease of Wiki, but GitHub doesn't let
      Google crawl these pages, and users can't open PRs on Wiki pages.
      
      - Updated the pruning algorithms documentation
      178c8c49
  9. Dec 04, 2018
    • Guy Jacob's avatar
      Range-Based Linear Quantization Features (#95) · 907a6f04
      Guy Jacob authored
      * Asymmetric post-training quantization (only symmetric supported so until now)
      * Quantization aware training for range-based (min-max) symmetric and asymmetric quantization
      * Per-channel quantization support in both training and post-training
      * Added tests and examples
      * Updated documentation
      Unverified
      907a6f04
  10. Nov 25, 2018
  11. Nov 24, 2018
    • Neta Zmora's avatar
      Fix activation stats for Linear layers · 22e3ea8b
      Neta Zmora authored
      Thanks to Dan Alistarh for bringing this issue to my attention.
      The activations of Linear layers have shape (batch_size, output_size) and those
      of Convolution layers have shape (batch_size, num_channels, width, height) and
      this distinction in shape was not correctly handled.
      
      This commit also fixes sparsity computation for very large activations, as seen
      in VGG16, which leads to memory exhaustion.  One solution is to use smaller
      batch sizes, but this commit uses a different solution, which counts zeros “manually”,
      and using less space.
      
      Also in this commit:
      - Added a “caveats” section to the documentation.
      - Added more tests.
      22e3ea8b
  12. Nov 21, 2018
  13. Nov 08, 2018
    • Haim Barad's avatar
      Early Exit docs (#75) · 470209b9
      Haim Barad authored
      * Updated stats computation - fixes issues with validation stats
      
      * Clarification of output (docs)
      
      * Update
      
      * Moved validation stats to separate function
      470209b9
  14. Nov 07, 2018
  15. Nov 06, 2018
  16. Nov 04, 2018
  17. Oct 03, 2018
    • Neta Zmora's avatar
      documentation: update syntax of launching jupyter notebook · 5902146a
      Neta Zmora authored
      Latest versions of Jupyter notebooks have a different syntax for
      launching the server such that it listens on oll network interfaces
      (this is useful if you are running the Jupyter server on one machine,
      and connect to it from a browser on a different machine).
      
      So:
      	jupyter-notebook --ip=* --no-browser
      
      is replaced by:
      	jupyter-notebook --ip=0.0.0.0 --no-browser
      5902146a
  18. Sep 16, 2018
    • Neta Zmora's avatar
      A temporary fix for issue #36 (#48) · 5d3d6d8d
      Neta Zmora authored
      * A temporary fix for issue 36
      
      The thinning code assumes that the sgraph it is using
      is not data-parallel, because it (currently) accesses the
      layer-name keys using a "normalized" name ("module." is removed).
      
      The bug is that in thinning.py#L73 we create a data_parallel=True
      model; and then give it to sgraph.
      But in other places thinning code uses "normalized" keys.  For
      example in thinning.py#L264.
      
      The temporary fix configures data_parallel=False in thinning.py#L73.
      
      A long term solution should have SummaryGraph know how to handle
      both parallel and not-parallel models.  This can be done by having
      SummaryGraph convert layer-names it receives in the API to
      data_parallel=False using normalize_layer_name.  When returning
      results, use the de-normalized format.
      
      * Fix the documentation error from issue 36
      * Move some logs to debug and show in logging.conf how to enable DEBUG logs.
      Unverified
      5d3d6d8d
  19. Sep 03, 2018
  20. Jul 31, 2018
  21. Jul 22, 2018
    • Gal Novik's avatar
      PACT quantizer (#30) · df9a00ce
      Gal Novik authored
      * Adding PACT quantization method
      * Move logic modifying the optimizer due to changes the quantizer makes into the Quantizer itself
      * Updated documentation and tests
      df9a00ce
  22. Jul 17, 2018
    • Guy Jacob's avatar
      Quantizer tests, fixes and docs update · 6b166cec
      Guy Jacob authored
      * Add Quantizer unit tests
      * Require 'bits_overrides' to be OrderedDict to support overlapping
        patterns in a predictable manner + update documentation to reflect this
      * Quantizer class cleanup
        * Use "public" nn.Module APIs instead of protected attributes
        * Call the builtins set/get/delattr instead of the class special methods
          (__***__)
        * Fix issues reported in #24
      * Bug in RangeLinearQuantParamLayerWrapper - add explicit override of
        pre_quantized_forward accpeting single input (#15)
      * Add DoReFa test to full_flow_tests
      6b166cec
  23. Jul 01, 2018
  24. Jun 22, 2018
    • Thomas Fan's avatar
      DOC: Fix (#10) · ab35fed7
      Thomas Fan authored
      Reviewed and looking good.  We have to set a convention for naming files.
      ab35fed7
  25. Jun 21, 2018
  26. Jun 14, 2018
  27. May 22, 2018
  28. May 14, 2018
  29. May 13, 2018
  30. Apr 30, 2018
  31. Apr 28, 2018
  32. Apr 24, 2018
Loading