General-Wing-785 OP t1_jedl8nt wrote on March 31, 2023 at 6:27 AM

Reply to comment by Philpax in [D] What are your top 3 pain points as an ML developer in 2023? by General-Wing-785

Is this because of a lack of tools to deploy Python models/functions to the edge? Also when you say user’s machine, is it a browser or a native app?

Philpax t1_jee04jo wrote on March 31, 2023 at 9:59 AM

It's just difficult to wrangle all of the dependencies; I want to be able to wrap an entire model in a complete isolated black box that I can call into with a C API or similar.

That is, I'd like something like https://github.com/ggerganov/llama.cpp/blob/master/llama.h without having to rewrite the entire model.

For my use cases, native would be good, but web would be a nice to have. (With enough magic, a native solution could be potentially compiled to WebAssembly?)