Writing a deep net in numpy 10 years ago continues to prove to be one of the most valuable personal development tasks I've ever taken on. It's a lot, it's totally unnecessary, but dang does it get you learned up good. For a minute there in the beginning of my career, whenever I tried a new model/algorithm, I wouldn't use it for anything serious until after I'd implemented from "scratch" (with numpy, so not technically "scratch," but you get the idea). This was an insane amount of overhead and tons of extra work that would make zero sense for a person who's well into their career. But again gosh dang did it get things to stick.

ETA: I occasionally wish that instead of diving so deep on methods and their details, I'd spent more time building my math skills. I can keep up with most ML math, but theory papers go over my head, and I not infrequently find myself wishing I had deeper linear algebra knowledge.