[D] SE for machine learning reaserch

Submitted by sad_potato00 t3_zvnjox in MachineLearning

Hello everyone,

I'm trying to figure out how to apply concepts from SE into ML research.

For me it seems like I can find really good settings for my Model and dataset, and it can be reproduced. However, I think there's a better way to create code for experimenting. Fore example, creating and testing baselines, and logging test results seems to be the same between most (if not all) my experiments. I find myself copying and pasting a lot of code snippets between my projects. Yet, every time I try to set down and write a generic code for experimenting. I find that it's either too limiting or impossible for me to write it. I think if I looked into software engineering concepts and principles it might help.

I really want to know what was your experience in searching/applying SE into this field, or if you even think it's worth it/possible to.

some of my colleagues think it's a waste of time, specially considering that the model would run on completely different code.

12

Comments

You must log in or register to comment.

ThisIsMyStonerAcount t1_j1q7cy6 wrote

There are a few libraries/frameworks that one can use and allow to reuse the same code for datasets, logging, training loop etc.... . E.g. Lightning or Scenic. Maybe you can use one of these or at least get some inspiration for your own code.

2

verdinglichung t1_j1qusj0 wrote

Google the term SE4DL or Software engineering for deep learning/machine learning. There are some great survey papers.

2

yunguta t1_j1rh04v wrote

I think you are looking for this: Full Stack Deep Learning . I work in a product-focused ML team at a software company and we use many of the tools, technologies and principles from that course.

7

bubudumbdumb t1_j1rwcg8 wrote

Have you tried exploring functional programming paradigms? You don't have to learn and use a functional language to use the style

−1

choHZ t1_j1t7mk2 wrote

I believe no one would argue against applying good engineering principles to any project. But for the context of academia ML research, reproducibility is the #1 reason for being open-sourced in the first place. People really don't care if your logger or plots are beautiful or reusable; they just want your code to a) run with minimum yak shaving and b) have clear instructions of what things are — yet that's already hard enough for many repos I have came crossed :(

For researchers that are doing the development, yes, it'd be nice to have some reusable/expandable modules. Most people I know have the same utils.py as the base template for similar projects, well-maintained open-sourced projects often have nice modules that you can borrow, there are also many well-built experiment trackers that you can make use of. If you got more routine stuff, services like AWS SageMaker are out there. It is just people don't care much about them when it comes to (most) research sharing, they are just more dependencies to track. If you are in a more production-focused role that'd be a different story.

2