abstract000 t1_jbfcydi wrote on March 8, 2023 at 5:26 PM

Reply to [D] Neat project that would "fit" onto a 4090? by lifesthateasy

You will not be able to train huge models like ViT or BERT from scratch, but for fine tuning a 4090 is waaaaaaay enough. Even with my 2080Ti I achieve really fine results with a decent amount of time.

Your best friend is Huggingface, because transformer are life with a nice GPU

With your new config here would be my first try:

ViT
BERT
Donut because it's hell of a thing
Everything you can find on huggingface and makes you curious

abstract000 t1_itc3cw1 wrote on October 22, 2022 at 2:35 PM

Reply to comment by dluther93 in [D][R] Staking XGBOOST and CNN/Transformer by MichelMED10

OK I will try this next time I work on tabular data.

abstract000 t1_itc0l85 wrote on October 22, 2022 at 2:14 PM

Reply to comment by dluther93 in [D][R] Staking XGBOOST and CNN/Transformer by MichelMED10

You really had a significant improvement? I tried this and it performed poorly, but maybe it was just the dataset. BTW did you test that on a Kaggle competition?