Viewing a single comment thread. View all comments

MrAcurite t1_j2rplc1 wrote

Everyone else has addressed starting your PhD at 27, and they better be right, as I likely won't be starting my own PhD for another few years.

But, regarding the value of pure ML theory research e.g. convergence bounds, versus practical ML research e.g. quantization methods, my personal feeling for quite some time has been that purely theoretical ML research has been predominantly bunk. Machine Learning is so high dimensional that things that can't be proven universal can be nearly guaranteed probabilistically, and things that can be shown to be possible can be staggeringly unlikely; for example, just because the No Free Lunch theorem exists, doesn't mean that Adam won't work in the vast, vast majority of cases.

Someone with a PhD in pure ML theory, if they're good, is probably still perfectly capable of heading to industry and making bank, whether that's continuing to do ML theory research, moving over to applications, or just becoming a quant or something. But honestly? I just find screwing around with training models and shit to be way more fun, and you should try it, if you haven't already.

1

notyourregularnerd OP t1_j2rqq0k wrote

Hey, thanks for the inputs. Actually I did very empirical stuff for a while. Infact, I published in some computer vision venues too, eccv, cvpr workshop et al. My main issue with total empirical stuff was that I never knew why it didn't work? Did I do it correctly or was it just setup for failure. The only way was to just brute force all possible cases of implementation to hope if it improves numbers. And then, does the model with better leaderboard number actually do better in wild? It felt more alchemy than science. That doesn't discount the fact that empirical ML helps run lot of businesses and generates value, but that isn't a metric to call it science, right? This made me explore stuff which is more rigorous and probably works with (optimistic/loose) guarantee? I actually love applied DL for its potential but I would want it to be more methodical.

1

MrAcurite t1_j2rrglj wrote

I think that's a fair criticism of applied ML as a field. I've definitely described Deep Learning as alchemy to friends.

For my work, the people who are paying for the models have a... sizable interest in confirming that the models will actually work in the field, so on occasion I've been called on to modify classical methods to fit, rather than just throwing neural networks at everything. Maybe you would like that kind of thing? Or, otherwise, there are a lot of people going after interpretability and robustness, and some interesting progress has been made.

1

notyourregularnerd OP t1_j2rsjwq wrote

Yeah I am actually Interested in classic stuff implementation and deployment from industry POV. I think doing work with classical ML models in industry is better depending upon ML maturity of clients. If I were a business with critical infra and ungergoing a digital transformation I would also be scared of DL stuff.

Btw thanks for the pointer on interpretability and robustness, I actually planned to work in robustness as part of PHD if I eventually join it.

1

MrAcurite t1_j2rt62o wrote

Maybe you should check out some of the labs at the ETH Zurich? Yeah, you'd have to put up with Schweizzerdeutsch for three years, but it seems like they're doing some interesting work in the area.

2

notyourregularnerd OP t1_j2rtlxc wrote

Thanks :D I applied to ETH for this admission cycle, it's competitive so let's see what happens. Actually the Saarland advisor kind of circled me into accepting the offer. The offer was rolled out before I could see outcome of other applications 😅

1

MrAcurite t1_j2ru24j wrote

I'm planning on applying to the ETH once I finish my MS, mostly because I think the whole "ask a professor to hire you" schtick might be easier than getting in somewhere with a more formal application, given my great work experience and fucking dogshit undergraduate performance. Also, it's a three year program with no coursework and an actually decent stipend, compared to US programs that might average five years and pay barely enough to eat or pay rent.

2

notyourregularnerd OP t1_j2s3v9t wrote

Well asking a prof to hire you is the conventional way but both ETH and EPFL, along with MPI (IMPRS programs) are moving to US style of admission cycle of once a year. Especially for AI related stuff. I'm not sure about how much the culture of open hiring from a prof will continue, until you graduate. So keep an eye on admission cycle in December and plan graduation accordingly. Even good profs in other Europe academia are being on onboarded to ELLIS (a Europe wide US style admission call for AI PhD programs). You would want to check that too! whenever you apply!

1