Viewing a single comment thread. View all comments

IntelArtiGen t1_j5tijjx wrote

I managed to use SwAV on 1 GPU (8GB), batch size 240, 224x224 images, FP16, ResNet18.

Of course it works, the problem isn't just the batch size but the accuracy - batchsize trade-off, and the accuracy was quite bad (still usable for my task though). If 50% top5 on imagenet is ok for you, you can do it. But I'm not sure there are many tasks where it makes sense.

Perhaps contrastive learning isn't the best for single GPU. I'm not sure about the current SOTA on this task.

3

shingekichan1996 OP t1_j5tiupg wrote

For 224x224 images, sure. But for images with large sizes, for example satellite images, it is hard to get 200+ batch size for a single gpu.

1

shingekichan1996 OP t1_j5tjy44 wrote

I think single GPU for SSL contrastive learning is a research direction to pursue, I'm not sure if anyone published papers on it, but if there's none, I'm surprised.

1