The best analysis on this is from this blog:

https://kipp.ly/blog/transformer-inference-arithmetic/

Right now, the tech isn't there to train on a single GPU. You're gonna end up training a language model for ~1 month to do so. It is slightly more efficient, though.

Lots of people looking at running locally. In addition to everything that people have said, there's a bunch of companies that will be releasing models that can just barely fit on an A100 soon that I've heard rumours about from employees.

Sutton and Barto is obligatory if you want to learn RL, imo. Even as an experienced researcher, I read it every year or so. It's very approachable.

Have you looked at some of the architectures that get rid of BatchNorm (e.g. NFNets)? In my experience, BatchNorm tends to be quite slow, so I wonder if there's some speed to be gained there.

Become a manager, you’ll have tons of meetings with people.

Alternatively, get a social hobby. You don’t need social connections to come from work.

The main problem with tabular Q-learning (I'm assuming that by classical, you mean tabular) is that for most environments that are interesting, the state space is massive, so we can't actually store all states in memory.

In particular for lunar lander, you have a continuous observation space, so you need to apply some sort of discretization; at that point, you might as well just use tile coding or some sort of other function approximator.

Why force the network to do this instead of just sorting the predictions afterward?

Have you profiled your code? That would be the first thing I would do.

What sort of utilization of the GPU are you getting?

It's likely you're bottlenecked by feeding data in- for supervised learning, that's often the case.

I'm happy to offer suggestions for feeding data in if you're using Tensorflow/JAX.

If you do this, make sure you don’t buy the same security within 30 days or it’s a “wash-loss” and you can’t dedicate the loss from your income.

What sort of loads are you planning on running on your laptop? If you’re running LLMs the answer’s gonna be totally different than if you’re trying to train RL agents in grid worlds.