Hello, I'm learning as a hobbyist and want to go to production with my trained model.

I know front and back end coding but serving and scaling a model in production seems daunting. I'm looking at AWS right now but it doesn't seem like the easiest tool to deploy ML models. I thought it would be much easier to deploy a model! (real world performance is another problem I will have to study soon)

I am a senior data analyst, working in NLP domain, chatbots .

Most of our current problems are not classical NLP problems like text classification or machine translation. We will think about a business problem and really think it is possible to solve it with ML or stats, but after research, we may not be able to terrific results to convince the product manager to implement it as a feature. May be It could be our poor quality research, but the point is is there any way I could know before hand that a particular problem can be solved with ML.

Another problem is the dataset size, we have limited data set and as usual ML models need more data to give good results, and it would be great if we have a scientific way of telling that if I get n data point my algorithm would work with a particular accuracy.

I’m staff ML engineer , I am working in cyber security domain . Our product is b2b , and there are lot of Ml challenges . Primary is find potential use case where ML uniquely can solve the problem . Second is validate your solution as its b2b , we have limited number of users & getting feedback is difficult . In cyber security domain false positive & false negatives both are costly , so need to tune algo according to each customer . We have followed augmented AI approach rather than fully automated so that customer can understand why some activity is suspicious and he can take action if it’s genuine .

In the last two weeks I am working on new product , I can’t able to come up with ML use case which can convince my Manger . I am exploring on competitor products to get more exposure now .

It is a really specific thing and depends on your settings, but maybe it could help you with the last problem. You can get some theoretical results on the amount of data needed for testing (that obviously gives you only very general idea on the amount of training data) using Hoeffding inequality.

i need a tool that allows me to use less tools please

In my experience it's much easier to serve models through either AWS Lambda with containers or just upload your model in a container to an EC2 instance.

Could you elaborate on what analysis you already do to determine if ML might be useful? I’m currently at that stage in a project myself.

What tools do you use?

There aren't any with me, we can use EDA with T-sne and all, but still, I will go ahead and create simple models to try out and see the results.

i'd like to see more researchers publish their code with a setup.py

Thanks mate I will learn this then. I find AwS is always a bit long to learn, but let's go, in the long run it always pays. Didn't want to go for something more complicated then I needed, I thought easier (and more expensive) services existed.

Then you need a ML expert to code python tools for you that encapsulate many tools with your preferred settings

(Or open source and a lot of effort)

it's called "python"

Research scientist in industry but I'll talk about something that I lead outside of my job, a project in computer vision designed to basically create a good open source alternative to a bunch of bio-imaging tools behind paywalls. I maintain the code base but work with my old lab to keep expanding and developing (at this point, I mostly do software work/troubleshoot methods from old papers that have weird issues). Currently focusing on anthropology but long term I want to push into computational neuroscience because I have a bunch of problems with that field from both the perspective as and adjacent researcher, and as family of a neurology patient...

Biggest problem is by far lack of existing tools outside of MATLAB, mostly for visualization and signal processing. Python is okay but not quite there in my experience. MATLAB alternatives are more or less in the same boat. The main challenge is that "performance evaluation" is inevitably qualitative. The benchmarks used to publish results aren't quite meaningless but they're mostly for show/make ML people happy. Practitioners don't really seem to care. That leads to a situation where I need really good 3D visualization tools that are interactive. Python hasn't been good for that. Current "free" solution is to go into Javascript but other issues arise because things just aren't configured properly for the space I work in.

The other big challenge is a really high bar for data quality prior to subsequent analysis. Most of my actual work here is spent in filling in niches of computer vision where almost nothing in top conferences/journals applies. This creates another challenge, since I have to do things from scratch. Say I'm registering 3D objects, because that's a lot of what I care about. Textures look mostly okay but have a problem using state of the art method? Rejection because prior work indicates that major findings in the past have been heavily biased by tiny issues...very little margin for error.

Then there's compute. To get meaningful attention, I have to assuming the users will have mediocre compute resources, most certainly no GPUs. Also a big limitation. Can't pre-train much of anything either.

If I had a genie, I'd mostly wish for interactivity in Python to be better because that would take care of a lot of headache...or for Julia or something to be more mature. Lots of interesting research to do and can be done in a short order, bottlenecked by lack of existing tools for intuitive interface design + lack of time to do them, even if I ended up getting paid.