Viewing a single comment thread. View all comments

konze t1_irzfxlz wrote

I contribute to our group who is working exactly on this. Currently, it is quite a mess because each HW vendor provides its own tooling for deploying on their device which leads to a lot of problems (e.g. missing support for certain layers). One of the most promising tools for edge deployment is TVM together with Network Architecture Search (NAS) where the network is tailored for a specific use case and the available resources.

5

muunbo OP t1_is163e3 wrote

Interesting, hadn't heard of TVM before! I'm wondering, did you come across cases in your work where it wasn't the model that was the worst bottleneck but the pre-processing / data pipeline that actually needed to be optimized? I had one experience like that so just wondering how common it is

1

konze t1_is2i187 wrote

No, the data already comes in the “correct” format as a data stream from a sensor. The DNN models are trained to work on that data stream, on an edge device you usually don’t have the performance to pre process anything, except for an FFT which is a fixed function accelerator that can process the data in real time.

1