TheCloudTamer t1_iuloar3 wrote on November 1, 2022 at 8:16 AM

EE to ML guy here. Copying some advice I sent someone else. It’s not exactly what you asked, but might be helpful.

Spend some time at a low level: use only Numpy to write a NN including backdrop. It’s feasible to do this for reasonably complex nets. For example the DRAW network by Alex Graves. You want to read the first 1/4 of David MacKay’s book. Watch the NYU DL video series and try the notebooks. Do the same the UvA I think that reading lots of new papers is a bad idea, especially as people create elaborate explanations for things. Instead, find some people to follow on GitHub who are coding with the latest ideas. Having said that, there are quite a few ppl who you want to read/listen their every word (Yann LeCun comes to mind). Listen to Andrej Karpathy to see what DL looks like in production. Don’t shy away from working on core basics like linear algebra.

1percentof2 t1_iuloy32 wrote on November 1, 2022 at 8:27 AM

That seems like a ridiculous amount of work to get started.

TheCloudTamer t1_iulvd20 wrote on November 1, 2022 at 10:05 AM

They aren’t requirements, just some paths worth going down.

fhadley t1_iupdm7l wrote on November 2, 2022 at 1:17 AM

Writing a deep net in numpy 10 years ago continues to prove to be one of the most valuable personal development tasks I've ever taken on. It's a lot, it's totally unnecessary, but dang does it get you learned up good. For a minute there in the beginning of my career, whenever I tried a new model/algorithm, I wouldn't use it for anything serious until after I'd implemented from "scratch" (with numpy, so not technically "scratch," but you get the idea). This was an insane amount of overhead and tons of extra work that would make zero sense for a person who's well into their career. But again gosh dang did it get things to stick.

ETA: I occasionally wish that instead of diving so deep on methods and their details, I'd spent more time building my math skills. I can keep up with most ML math, but theory papers go over my head, and I not infrequently find myself wishing I had deeper linear algebra knowledge.