Somewhere between a high end gaming card and 8 A100s probably. So £1k - 160k ish worth of GPUs?

I print them out and put them in piles. I write on them with pens.

It seems that contrary to conventional wisdom, models with more parameters learn more efficiently. My personal ‘hunch’ is that training large models and then some form of distillation may become the standard thing to do.

IDK did you read it?

‘Hey, I know nothing about what I’m saying but…’

I guess for it to really count as a variational autoencoder you need to be reconstructing the input

Yeah, that’s something which would be useful indeed. Don’t worry yourself though, I can put in a PR.

Looks nice. I probably wouldn’t use it for shutting down or anything, but a notification on failure might be useful!

Kaggle would like to know your location

Classes

I would just try isotonic regression or platt scaling.

Reads more like an HMM

Perhaps you can do some data augmentation and resampling proportional to the difficulty.

Also perhaps a scheme like this could be appropriate:

https://arxiv.org/abs/2206.07137

Cool just write a deep learning framework

And you feel if you wrote raw C++ it would be as fast as the PyTorch ops, or you seek to replace the Python part, or something else?

Yeah, no

I have no idea what you’re suggesting. Use C++ instead of vectorising properly and using PyTorch? Do you currently do much compute ‘outside’ PyTorch?

The GIL has nothing to do with ‘flexibility’

Even if you wrote in a language like Rust, Go, C, C++ you wouldn’t avoid ‘calling into a framework’ syndrome.

There are several reasons why things like the compute kernels available in PyTorch are fast / optimised, and it’s not as simple as they are compiled. Operations are written efficiently with hardware and memory access patterns in mind, and we also have to ensure we do things like correctly implement the API for backward mode autodiff.

If you wanted to swap over to writing all of this in bare C++ it would undoubtedly be much slower than using PyTorch. We use Python because it’s a nice convenient language for calling into the framework. The overhead from this usually isn’t particularly significant. If you have a use-case where it is significant then sure use C++.

I only buy iPhones and have never had an Android. You could argue it’s habit / inertia, but nothing has every made me feel swayed towards an Android.

Perhaps of relevance:

https://arxiv.org/abs/1902.04422

The merit in a paper isn’t in how hard it was or how long it took. It’s the novelty / value of the contribution.

Ye

As in, you care about compression for the sake of communication and not computation?

I must say, I find it really weird that someone who would ask people online to write trivially simple code for them would be this defensive. Can you not look at yourself and think, huh maybe something is wrong with my attitude?