That's not what I meant, so no offense taken.

Huh, I could have sworn it was a lot older.

No, I mean you don't need anything special or to follow a conventional path.

I feel like Eliezer Yudkowsky proves that everyone can be Eliezer Yudkowsky, going from a crazy guy with a Harry Potter fanfic and a blog to being mentioned in your post alongside those other two names.

Of course you should go. And depending on the company and the role they actually have in mind that amount of math background could be plenty.

Unfortunately there are just too many crackpots in that space. It's like bringing up GUT in physics - worthwhile goal, but you're sharing the bus with too many crazies.

Me too. There's a lot of great people I want to hear from but only when they post about ML, not politics.

What's the current understanding of why/when batch norm works? I haven't kept up with the literature but I had the impression there was no real consensus.

Maybe it's a difference in fields. I rarely see people do meta-analysis in ML so it didn't strike me as odd. Most of the reviews are just "here's what people are trying" with some attempt at categorization. But I see what you mean now, it makes sense that having a meta-analysis is important in medical fields where you want to aggregate studies.

I have definitely seen the kind of papers you're talking about, but this one seems fine to me? Granted I skimmed it really quickly but the title says it's a review article and the abstract reflects that.

As an aside: I really like the format I see in bio fields (and maybe others, but this is where I've encountered it) of putting the results before the detailed methodology. It doesn't always make sense for a lot of CS papers where the results are the most boring part (essentially being "it works better") but where it does it leads to a much better paper in my opinion.

Thanks for that resource, I've been experimenting with the lottery ticket method but that's a lot of papers I haven't seen! Did you initialize the weights as if training from scratch, or did you do something like trying to match the variance of the old and new weights? I'm intrigued that your method didn't hurt performance - most of the things I've tested were detrimental to the network. I have seen some performance improvements under different conditions but I'm still trying to rule out any confounding factors.

Holy shit, that's incredible.

Can you elaborate on this? I'm trying something similar, so I'm curious what your results were and if you ran across any literature about this idea.

wandb is great but I had no idea it also versioned code, I'm still using git for that.

Wait, has someone actually integrated Wolfram with ChatGPT? I thought it was still in the "would be cool" stage.

What is that? I can't find a copy online.