Cross Tokenizer Distillation

Implemented the paper on Cross Tokenizer Distillation [ULD Loss] to enhance small-scale models by leveraging larger models as supervisors.

• Trained bloomz-560m (student) using phi-3-mini-instruct (teacher) on the QED dataset, improving F1 score from 0.550 to 0.588.