Skip to content
Snippets Groups Projects
Unverified Commit 32a7e4bf authored by Guy Jacob's avatar Guy Jacob Committed by GitHub
Browse files

Knowledge distillation fixes (#503)

Fixed two long-standing bugs in knowledge distillation:
 * Distillation loss needs to be scaled by T^2 (#122)
 * Use tensor.clone instead of new_tensor when caching student logits (#234)
Updated example results and uploaded the script to generate them
parent 8c5b287c
No related branches found
No related tags found
No related merge requests found
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment