Skip to content
Snippets Groups Projects
  1. Apr 27, 2020
  2. Oct 06, 2019
    • Neta Zmora's avatar
      Low-level pruning API refactor (#401) · 05d5592e
      Neta Zmora authored
      Some refactoring of the low-level pruning API
      
      Added distiller/norms.py - for calculating norms of various sub-tensors.
      
      ranked_structures_pruner.py:
      -Removed l1_magnitude, l2_magnitude. Use instead distiller.norms.l1_norm
      -Lots of refactoring
      -replaced LpRankedStructureParameterPruner.ch_binary_map_to_mask with
      distiller.thresholding.expand_binary_map
      -FMReconstructionChannelPruner.rank_and_prune_channels used L2-norm
      by default and now uses L1-norm (i.e.magnitude_fn=l2_magnitude was
      replaced with magnitude_fn=distiller.norms.l1_norm)
      
      thresholding.py:
      -Delegated lots of the work to the new norms.py.
      -Removed support for 4D (entire convolution layers) since that has not been
      maintained for a longtime. This may break some old scripts that remove entire
      layers.
      -added expand_binary_map() explicitly so others can use it. Might need to
      move to a different file
      -removed threshold_policy()
      
      utils.py:
      -use distiller.norms.xxx for sparsity stats
      Unverified
      05d5592e
  3. Aug 22, 2019
  4. Jul 04, 2019
    • Guy Jacob's avatar
      Switch to PyTorch 1.1.0 (#306) · 032b1f74
      Guy Jacob authored
      * PyTorch 1.1.0 now required
        - Moved other dependencies to up-to-date versions as well
      * Adapt LR scheduler to PyTorch 1.1 API changes:
        - Change lr_scheduler.step() calls to succeed validate calls,
          during training
        - Pass to lr_scheduler.step() caller both loss and top1
          (Resolves issue #240)
      * Adapt thinning for PyTorch 1.1 semantic changes
        - **KNOWN ISSUE**: When a thinning recipe is applied, in certain
          cases PyTorch displays this warning:
          "UserWarning: non-inplace resize is deprecated".
          To be fixed later
      * SummaryGraph: Workaround for new scope name issue from PyTorch 1.1.0
      * Adapt to updated PyTest version:
        - Stop using deprecated 'message' parameter of pytest.raises(),
          use pytest.fail() instead
        - Make sure only a single test case per pytest.raises context
      * Move PyTorch version check to root __init__.py 
        - This means the version each checked when Distiller is first
          imported. A RuntimeError is raised if the version is wrong.
      * Updates to parameter_histograms notebook:
        - Replace deprecated normed argument with density
        - Add sparsity rate to plot title
        - Load model in CPU
      Unverified
      032b1f74
  5. Jul 01, 2019
  6. May 06, 2019
    • Bar's avatar
      Fix broken load test (#245) · 1f48fa64
      Bar authored
      In a former commit, distiller accepts checkpoints that do not contain
      'optimizer' argument. However, this change was not reflected in the
      relevant test.
      1f48fa64
  7. Apr 01, 2019
    • Lev Zlotnik's avatar
      Quantizer: Specify # bias bits + custom overrides (BREAKING) (#178) · 5271625a
      Lev Zlotnik authored
      * Bias handling:
        * Add 'bits_bias' parameter to explicitly specify # of bits for bias,
          similar to weights and activations.
        * BREAKING: Remove the now redundant 'quantize_bias' boolean parameter
      * Custom overrides:
        * Expand the semantics of the overrides dict to allow overriding of
          other parameters in addition to bit-widths
        * Functions registered in the quantizer's 'replacement_factory' can
          define keyword arguments. Non bit-width entries in the overrides
          dict will be checked against the function signature and passed
        * BREAKING:
          * Changed the name of 'bits_overrides' to simply 'overrides'
          * Bit-width overrides must now be defined using the full parameter
            names - 'bits_activations/weights/bias' instead of the short-hands
            'acts' and 'wts' which were used so far.
        * Added/updated relevant tests
        * Modified all quantization YAMLs under 'examples' to reflect 
          these changes
        * Updated docs
      5271625a
    • Bar's avatar
      Load optimizer from checkpoint (BREAKING - see details) (#182) · 992291cf
      Bar authored
      Load optimizer from checkpoint (BREAKING - see details) (#182)
      
      * Fixes issues #70, #145 and replaces PR #74
      * checkpoint.py
        * save_checkpoint will now save the optimizer type in addition to
          its state
        * load_checkpoint will now instantiate an optimizer based on the
          saved type and load its state
      * config.py: file/dict_config now accept the resumed epoch to pass to
        LR schedulers
      * policy.py: LRPolicy now passes the current epoch to the LR scheduler
      * Classifier compression sample
        * New flag '--resume-from' for properly resuming a saved training
          session, inc. optimizer state and epoch #
        * Flag '--reset-optimizer' added to allow discarding of a loaded
          optimizer.
        * BREAKING:
          * Previous flag '--resume' is deprecated and is mapped to
            '--resume-from' + '--reset-optimizer'. 
          * But, old resuming behavior had an inconsistency where the epoch
            count would continue from the saved epoch, but the LR scheduler
            was setup as if we were starting from epoch 0.
          * Using '--resume-from' + '--reset-optimizer' now will simply
            RESET the epoch count to 0 for the whole environment.
          * This means that scheduling configurations (in YAML or code)
            which assumed use of '--resume' might need to be changed to
            reflect the fact that the epoch count now starts from 0
          * All relevant YAML files under 'examples' modified to reflect
            this change
      * Initial support for ReduceLROnPlateau (#161):
        * Allow passing **kwargs to policies via the scheduler
        * Image classification now passes the validation loss to the
          scheduler, to be used yo ReduceLROnPlateau
        * The current implementation is experimental and subject to change
      992291cf
  8. Mar 11, 2019
    • Bar's avatar
      Integrate Cadene pretrained PyTorch models [fixes #142] (#184) · 321abb61
      Bar authored
      Integrate Cadene ```pretrainedmodels``` package.
      
      This PR integrates a large set of pre-trained PyTorch image-classification and object-detection models which originate from https://github.com/Cadene/pretrained-models.pytorch.
      
      *******************************************************************************************
      PLEASE NOTE: 
      This PR adds a dependency on he ```pretrainedmodels``` package, and you will need to install it using ```pip3 install pretrainedmodels```.  For new users, we have also updated the ```requirements.txt``` file.
      *******************************************************************************************
      
      Distiller does not currently support the compression of object-detectors (a sample application is required - and the community is invited to send us a PR).
      
      Compression of some of these models may not be fully supported by Distiller due to bugs and/or missing features.  If you encounter any issues, please report to us.
      
      Whenever there is contention on the names of models passed to the ```compress_classifier.py``` sample application, it will prefer to use the Cadene models at the lowest priority (e.g. Torchvision models are used in favor of Cadene models, when the same model is supported by both packages).
      
      This PR also:
      * Adds documentation to ```create_model```
      * Adds tests for ```create_model```
      321abb61
  9. Feb 26, 2019
  10. Feb 12, 2019
    • Neta Zmora's avatar
      CPU support: fix the case of loading a thinned GPU-model on the CPU · ba05f6cf
      Neta Zmora authored
      This commit fixes (and adds a test) for the case that we with to load
      a thinned GPU checkpoint onto the CPU.
      ba05f6cf
    • Neta Zmora's avatar
      Fix issue #148 + refactor load_checkpoint.py (#153) · 1210f412
      Neta Zmora authored
      The root-cause of issue #148 is that DataParallel modules cannot execute on the CPU,
      on machines that have both CPUs and GPUs.
      Therefore, we don’t use DataParallel for models loaded for the CPUs, but we do wrap
      the models with DataParallel when loaded on the GPUs (to make them run faster).
      The names of module keys saved in a checkpoint file depend if the modules are wrapped
      by a DataParallel module or not.  So loading a checkpoint that ran on the GPU onto a
      CPU-model (and vice-versa) will fail on the keys.
      This is all PyTorch and despite the community asking for a fix -
      e.g. https://github.com/pytorch/pytorch/issues/7457 - it is still pending.
      
      This commit contains code to catch key errors when loading a GPU-generated model
      (i.e. with DataParallel) onto a CPU, and convert the names of the keys.
      
      This PR also merges refactoring to load_chackpoint.py done by @barrh, who also added
      a test to further test loading checkpoints.
      Unverified
      1210f412
  11. Jul 11, 2018
  12. Apr 25, 2018
  13. Apr 24, 2018
Loading