Brigham Young University might also be near the top in another kind of yield.

Then you shouldn't be training on a laptop, and should train on a remote server, in which case your laptop doesn't matter.

It looks like you don't care about throughput so it doesn't matter.

Isn't this just the result of using certain tokenizers? Using Chinese as an example, no reasonable tokenizer developed with Chinese in mind would give you 17 tokens. You'd have maybe 6 to 8:

1. 你好
2. ，
3. 我
4. 是
5. 个
6. 高个子

...depending on whether it thinks 你好 and 高个子 should be split.

2 seems doable. Not everybody has to have a GPU, but I bet lots of people, including me, would rather spin up the GPU in their personal computer for a few seconds than manually specify where skippable segments are.

The one central server thing bugs me. I'd prefer something like "query your nearest neighbors and choose the one with the most recent data." No idea how to do that though; not a systems person.

OK guys, you can chill with the downvotes. They're just asking questions.

As mentioned elsewhere, Python does not do much work; the important parts are in CUDA. So if you used some other language such as C++, you still can't write loops, and you still have to use the framework's data structures.

Can't see it. Not beautiful.

Not a fan of your color map. Because you used a diverging color map, the least saturated colors, near 50%, isn't some intuitive reference value such as 0, and look too much like NA. You should have used a sequential color map.

I looked at it. He misspoke and probably meant that batch normalization makes the preactivations closer to the domain of the activation function (tanh in this case) where the derivative is far from 0.

Also, "Gaussian" is often used to refer to the standard normal distribution.

Same kind of deal when lots of people say "convolution" when they mean "cross correlation."

To answer your question, no, it is not necessary, but good researchers often have a solid math foundation.