[deleted]

You'll have to try it out and see what works best. By the way, PyTorch has some beta support for mobile deployment: https://pytorch.org/mobile/home/

First, PlayTorch is a thing. That could replace tflite for mobile.

Second, you can use ONNX to transfer models from pt to tf, then to tflite.

Last but not least, if you are a REAL MAN, you can try to translate the model weights to tf by hand, using numpy as an intermediary.

How do you mean - translate model weights by hand?

Load pt model with torch, get all weights with state_dict. Make them all into channels last format with permute. Load all the arrays into numpy. Load all the arrays into tensorflow. Write the network forward logic in tensorflow. Plug in weights. Run.

Thanks for clearification! Do you know if there is an tutorial/blog post for this approach?

Nope. I don't think so. If you need help and are willing to wait a bit (A bit busy right now), DM me and I'll take a look at your problem.

Not a big TF guy but thr SOTA models nowadays will usually be in Pytorch. So if you need those, then Pytorch.

Thx - appreciate it! I dont have a problem i just want to learn/try this approach because i never heard of it.

How complex is the model you want to translate?

I dont have a model to translate - i read about this approach in the comments and i want to learn about it.

Oh no! I had no idea about ONNX and just transfered a huge model from tf to pyt manually.

ONNX tracers are decent, but aren't that good, I had to rewrite Wav2vec to get both batch size and input length as dynamic axis.

I see. But can they be used as a good starting point to build upon?

I don’t know much about this topic but to me, it seems there is a difference between a SOTA model developed for a paper and a model ready for deployment. Seems torch is easier to dev while tf uses lazy evaluation and is therefore lighter. Could be wrong

The ecosystems are very dissimilar so it’s not easy to deploy a model in both

Yeah i might be wrong based on the downvotes.

;)

I don’t know much about this either but it seems you’re right since there’s not a one size fits all solution for anything. But at the same time, it’s not very helpful advice

Definitely not helpful advise. Anyway it feels like the question caters to fanboys of each enviroment.

Both pytorch and tensorflow are pretty good, though each has their own strengths.

The thing is, either you wait for the TF support or you code it yourself. Research is fast paced so researchers want a boilerplate, or just to build on top of another repo.

For deployment, there's a lot of deployment tools for Pytorch nowadays. I use BentoML for deploying compvision models. If you want a lighter model, there are libraries and repositories that support sparsification, pruning, etc. of your models. A lot of newer repos for industrial use (mmdet, YOLOv5/6/7) have sparsification and pruning support. Some of them even have distillation built-in. Again, I am not a TF guy so I havent seen such rich support for model deployment than these Pytorch-based repos.

While it is true that research and deployment are different, it is up to your team how to do MLOps on those SOTA methods. New SSL method, new loss functions, new stuff -- either you borrow it from a repo or you code it yourself. In the industry, the prototyping procedure for experimentatio is critical for a fast-moving product. You dont have time to convert codes. So you test repositories, then build on top of it if it is successful.

Also, if u want a middle ground, then go for ONNX. Just convert ur tf or pytorch model to onnx.

TensorFlow has model optimization libraries such as, but not limited to: weight clustering, pruning, and weight quantization, as well as training support for these.

[deleted]

That's great. Again, I am not used to TF. My point was that Pytorch has deployment capabilities near TF's nowadays and that most SOTA and new research are leaning towards publishing their code with Pytorch.

[deleted]

Develop in whatever framework you like, then deploy with Triton inference server

lmao

PyTorch.

Datapoint 1: it’s part of the Linux foundation

Datapoint 2: Jax

Disclosure: I haven’t used TensorFlow.

[deleted]

[deleted]

What are you talking about?

I have been trying to convert wav2vec2 model to onnx without much success

I was actually able to do this once by relying heavily on Github's Copilot, ie just giving functions very specific names and commenting things like "this function converts a pytorch model into a vector of weights" etc. It worked pretty well and was simple for basic network architectures, though I imagine it could get more complicated.

I usually do onnx or tensorRT for deployment and use whatever I want for deployment. Most libraries can serialize down to an ONNX model or a tensorRT model.

My 2 cents: I am still the stubborn minority using TF. Mostly because of switching costs in my organization. But also, honestly, because I am decently happy with it. I think the ease-of-use gap between pytorch and TF was mostly erased with the release of TF2. I was really hoping that more of the research community would come back to TF after that, but unfortunately that has not been the case. In practice, it does suck to not have access to SOTA models. Otherwise, I think they are pretty similar.

Agreed with all the folks, if your model is far beyond ordinary CNN and RNN then ONNX will give you a bunch of warnings that confuses you.

I did it once in one of my projects. It's basically hand-picking weights from Pytorch model's layers to TensorFlow model's layers, but it feels more reliable than relying on ONNX with a bunch of warnings.

Though there are not much tutorials or blog posts about this, I will try creating a github repo for this later (just examples with simple layers), so many more people will know that this technique exists.

https://github.com/lernapparat/lernapparat/blob/master/style_gan/pytorch_style_gan.ipynb

This notebook does the reverse - TF to torch for stylegan