- Apr 27, 2020
-
-
Neta Zmora authored
-
- Oct 06, 2019
-
-
Neta Zmora authored
Some refactoring of the low-level pruning API Added distiller/norms.py - for calculating norms of various sub-tensors. ranked_structures_pruner.py: -Removed l1_magnitude, l2_magnitude. Use instead distiller.norms.l1_norm -Lots of refactoring -replaced LpRankedStructureParameterPruner.ch_binary_map_to_mask with distiller.thresholding.expand_binary_map -FMReconstructionChannelPruner.rank_and_prune_channels used L2-norm by default and now uses L1-norm (i.e.magnitude_fn=l2_magnitude was replaced with magnitude_fn=distiller.norms.l1_norm) thresholding.py: -Delegated lots of the work to the new norms.py. -Removed support for 4D (entire convolution layers) since that has not been maintained for a longtime. This may break some old scripts that remove entire layers. -added expand_binary_map() explicitly so others can use it. Might need to move to a different file -removed threshold_policy() utils.py: -use distiller.norms.xxx for sparsity stats
-
- Aug 22, 2019
-
-
Neta Zmora authored
This is inspired by @barrh’s PR https://github.com/NervanaSystems/distiller/pull/246 but it at a “slower-integration-pace” and w/o changing APIs. 1. create_model() attaches model attributes (arch, dataset, is_parallel) to created models. 2. save_checkpoint() stores the new model attributes with checkpoint metadata 3. load_checkpoint() can be invoked with model=None, in which case we attempt to create the model from the stored checkpoint metadata.
-
- Jul 04, 2019
-
-
Guy Jacob authored
* PyTorch 1.1.0 now required - Moved other dependencies to up-to-date versions as well * Adapt LR scheduler to PyTorch 1.1 API changes: - Change lr_scheduler.step() calls to succeed validate calls, during training - Pass to lr_scheduler.step() caller both loss and top1 (Resolves issue #240) * Adapt thinning for PyTorch 1.1 semantic changes - **KNOWN ISSUE**: When a thinning recipe is applied, in certain cases PyTorch displays this warning: "UserWarning: non-inplace resize is deprecated". To be fixed later * SummaryGraph: Workaround for new scope name issue from PyTorch 1.1.0 * Adapt to updated PyTest version: - Stop using deprecated 'message' parameter of pytest.raises(), use pytest.fail() instead - Make sure only a single test case per pytest.raises context * Move PyTorch version check to root __init__.py - This means the version each checked when Distiller is first imported. A RuntimeError is raised if the version is wrong. * Updates to parameter_histograms notebook: - Replace deprecated normed argument with density - Add sparsity rate to plot title - Load model in CPU
-
- Jul 01, 2019
-
-
Guy Jacob authored
-
- May 06, 2019
-
-
Bar authored
In a former commit, distiller accepts checkpoints that do not contain 'optimizer' argument. However, this change was not reflected in the relevant test.
-
- Apr 01, 2019
-
-
Lev Zlotnik authored
* Bias handling: * Add 'bits_bias' parameter to explicitly specify # of bits for bias, similar to weights and activations. * BREAKING: Remove the now redundant 'quantize_bias' boolean parameter * Custom overrides: * Expand the semantics of the overrides dict to allow overriding of other parameters in addition to bit-widths * Functions registered in the quantizer's 'replacement_factory' can define keyword arguments. Non bit-width entries in the overrides dict will be checked against the function signature and passed * BREAKING: * Changed the name of 'bits_overrides' to simply 'overrides' * Bit-width overrides must now be defined using the full parameter names - 'bits_activations/weights/bias' instead of the short-hands 'acts' and 'wts' which were used so far. * Added/updated relevant tests * Modified all quantization YAMLs under 'examples' to reflect these changes * Updated docs
-
Bar authored
Load optimizer from checkpoint (BREAKING - see details) (#182) * Fixes issues #70, #145 and replaces PR #74 * checkpoint.py * save_checkpoint will now save the optimizer type in addition to its state * load_checkpoint will now instantiate an optimizer based on the saved type and load its state * config.py: file/dict_config now accept the resumed epoch to pass to LR schedulers * policy.py: LRPolicy now passes the current epoch to the LR scheduler * Classifier compression sample * New flag '--resume-from' for properly resuming a saved training session, inc. optimizer state and epoch # * Flag '--reset-optimizer' added to allow discarding of a loaded optimizer. * BREAKING: * Previous flag '--resume' is deprecated and is mapped to '--resume-from' + '--reset-optimizer'. * But, old resuming behavior had an inconsistency where the epoch count would continue from the saved epoch, but the LR scheduler was setup as if we were starting from epoch 0. * Using '--resume-from' + '--reset-optimizer' now will simply RESET the epoch count to 0 for the whole environment. * This means that scheduling configurations (in YAML or code) which assumed use of '--resume' might need to be changed to reflect the fact that the epoch count now starts from 0 * All relevant YAML files under 'examples' modified to reflect this change * Initial support for ReduceLROnPlateau (#161): * Allow passing **kwargs to policies via the scheduler * Image classification now passes the validation loss to the scheduler, to be used yo ReduceLROnPlateau * The current implementation is experimental and subject to change
-
- Mar 11, 2019
-
-
Bar authored
Integrate Cadene ```pretrainedmodels``` package. This PR integrates a large set of pre-trained PyTorch image-classification and object-detection models which originate from https://github.com/Cadene/pretrained-models.pytorch. ******************************************************************************************* PLEASE NOTE: This PR adds a dependency on he ```pretrainedmodels``` package, and you will need to install it using ```pip3 install pretrainedmodels```. For new users, we have also updated the ```requirements.txt``` file. ******************************************************************************************* Distiller does not currently support the compression of object-detectors (a sample application is required - and the community is invited to send us a PR). Compression of some of these models may not be fully supported by Distiller due to bugs and/or missing features. If you encounter any issues, please report to us. Whenever there is contention on the names of models passed to the ```compress_classifier.py``` sample application, it will prefer to use the Cadene models at the lowest priority (e.g. Torchvision models are used in favor of Cadene models, when the same model is supported by both packages). This PR also: * Adds documentation to ```create_model``` * Adds tests for ```create_model```
-
- Feb 26, 2019
-
-
Lev Zlotnik authored
Not backward compatible - re-installation is required * Fixes for PyTorch==1.0.0 * Refactoring folder structure * Update installation section in docs
-
- Feb 12, 2019
-
-
Neta Zmora authored
This commit fixes (and adds a test) for the case that we with to load a thinned GPU checkpoint onto the CPU.
-
Neta Zmora authored
The root-cause of issue #148 is that DataParallel modules cannot execute on the CPU, on machines that have both CPUs and GPUs. Therefore, we don’t use DataParallel for models loaded for the CPUs, but we do wrap the models with DataParallel when loaded on the GPUs (to make them run faster). The names of module keys saved in a checkpoint file depend if the modules are wrapped by a DataParallel module or not. So loading a checkpoint that ran on the GPU onto a CPU-model (and vice-versa) will fail on the keys. This is all PyTorch and despite the community asking for a fix - e.g. https://github.com/pytorch/pytorch/issues/7457 - it is still pending. This commit contains code to catch key errors when loading a GPU-generated model (i.e. with DataParallel) onto a CPU, and convert the names of the keys. This PR also merges refactoring to load_chackpoint.py done by @barrh, who also added a test to further test loading checkpoints.
-
- Jul 11, 2018
-
-
Neta Zmora authored
- Raise IOError instead of crude exit() when file is not found in the file-system - Test that the correct exception is raised when opening a non-existent checkpoint file
-
- Apr 25, 2018
-
-
Neta Zmora authored
-
- Apr 24, 2018
-
-
Neta Zmora authored
-