Viewing a single comment thread. View all comments

JustOneAvailableName t1_jbthd11 wrote

Isn't the whole transformer revolution due to SSL which is just plain unsupervised learning?

12

onebigcat OP t1_jbtjqc4 wrote

I guess it’s a matter of how you define unsupervised, but isn’t SSL closer to supervised learning because there’s a ground-truth to compare the prediction to? Whereas if you’re just clustering some high dimensional data, you might not know what the “true” or most accurate way of clustering that information might be, especially in something like genomics where there’s a lot of information that has an unknown purpose.

−1

currentscurrents t1_jbtpv6w wrote

Run SSL to learn about the structure of the data and then just cluster the embeddings.

3

onebigcat OP t1_jbtuxhs wrote

Any papers or models you could point to using this for a specific purpose?

2

marboka t1_jbue3u3 wrote

DINO by facebook, STEGO by microsoft

5

SeucheAchat9115 t1_jbwoc1c wrote

SSL is the synonym for Semi-Supervised Learning. What you refer here is Self-Supervised Learning, which is related to unsupervised learning

0

huehue9812 t1_jbw7qjq wrote

SSL doesn't require human labels, thus it is unsupervised learning

1

[deleted] t1_jbtogrq wrote

[deleted]

−1

onebigcat OP t1_jbuj0qk wrote

I appreciate the insight! I’m new to ML (coming from the bio research side of things) and trying to keep up.

0