Viewing a single comment thread. View all comments

sqweeeeeeeeeeeeeeeps t1_izspv5o wrote on December 11, 2022 at 3:57 PM

Reply to comment by MazenAmria in Advices for Deep Learning Research on SWIN Transformer and Knowledge Distillation by MazenAmria

Showing you can create a smaller model with the same performance means SWIN is overparameterized for that given task. Give it datasets with varying complexity, not just one single one.