Skip to content
Snippets Groups Projects
  • Bar's avatar
    d08c3734
    Relaxation of the SummaryGraph API (#212) · d08c3734
    Bar authored
    This commit simplifies the SummaryGraph API,
    by removing from the client to burden to handle 
    the differences between models with/without 
    DataParallel layers.
    
    DataParallel layers in PyTorch change the fully-qualified
    names (FQNs) of PyTorch modules.  A module's FQN
    unambiguously identifies a module within a model, by 
    encoding the path to the module from the root of the 
    model.  For example, ```module.layer2.1.conv1``` and 
    ```module.layer2.0.conv1``` are FQNs of two different
    modules named ```conv1``` in some module.  
    Because a module's FQN reflects the module's hierarchy,
    adding/removing a DataParallel node also changes its FQN.
    
    Distiller uses FQNs to refer to modules and parameters 
    (e.g. from YAML files), and non-functional changes to the 
    model hierarchy, such as using DataParallel modules are
    handled by converting FQNs using `
    ``utils.{de,}normalize_module_name()```.
    
    Before this commit, the SummaryGraph API assumed that 
    the API client will convert layers names using 
    ```utils.normalize_module_name()``` before invoking the API.
    This led to needlessly verbose client code, which was also
    error-prone and harder to read and maintain.
    This commit fixes these short-comings by relaxing the API, 
    and handling the FQNN naming differences internally.
    
    The thinning implementation is simplified somewhat
    by refactoring to the new APIs lenient requirements.
    
    Added named_params_layers method to SummaryGraph
    that yields a 3-tuple of: layer name, param name, and param.
    When using the new method, summary graph communicates the
    true layer name in respect to the model it was initiated with.
    d08c3734
    History
    Relaxation of the SummaryGraph API (#212)
    Bar authored
    This commit simplifies the SummaryGraph API,
    by removing from the client to burden to handle 
    the differences between models with/without 
    DataParallel layers.
    
    DataParallel layers in PyTorch change the fully-qualified
    names (FQNs) of PyTorch modules.  A module's FQN
    unambiguously identifies a module within a model, by 
    encoding the path to the module from the root of the 
    model.  For example, ```module.layer2.1.conv1``` and 
    ```module.layer2.0.conv1``` are FQNs of two different
    modules named ```conv1``` in some module.  
    Because a module's FQN reflects the module's hierarchy,
    adding/removing a DataParallel node also changes its FQN.
    
    Distiller uses FQNs to refer to modules and parameters 
    (e.g. from YAML files), and non-functional changes to the 
    model hierarchy, such as using DataParallel modules are
    handled by converting FQNs using `
    ``utils.{de,}normalize_module_name()```.
    
    Before this commit, the SummaryGraph API assumed that 
    the API client will convert layers names using 
    ```utils.normalize_module_name()``` before invoking the API.
    This led to needlessly verbose client code, which was also
    error-prone and harder to read and maintain.
    This commit fixes these short-comings by relaxing the API, 
    and handling the FQNN naming differences internally.
    
    The thinning implementation is simplified somewhat
    by refactoring to the new APIs lenient requirements.
    
    Added named_params_layers method to SummaryGraph
    that yields a 3-tuple of: layer name, param name, and param.
    When using the new method, summary graph communicates the
    true layer name in respect to the model it was initiated with.