shahaff32
shahaff32 t1_jbu0zmr wrote
In our research we ran into issues with Lightning. It is especially annoying when designing non-trivial layers or optimizers. Also, it is much harder to convert the code back to pure pytorch.
For example, in a recent peoject, Lightning caused each forward-backward to operate twice on each batch because we used a combination of two optimizers for a specific reason. And now we are working on rewriting everything without Lightning.
shahaff32 t1_ixmw448 wrote
Reply to [D] Informal meetup at NeurIPS next week by tlyleung
Will join :)
shahaff32 t1_iu0mobx wrote
Reply to comment by YonatanBitton in [R] WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models by YonatanBitton
Thank you for your answer, we will look into it :)
shahaff32 t1_itzzb72 wrote
Reply to [R] WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models by YonatanBitton
It looks interesting, but associations can have many aspects and may lead to misunderstanding. How do you deal with it?
shahaff32 OP t1_is4jcv4 wrote
Reply to comment by danny_fel in [R] Wavelet Feature Maps Compression for Image-to-Image CNNs by shahaff32
Thanks :)
In the current state the implementation is using only standard Pytorch operations, therefore it is not as optimal as it can be, and the overhead of the wavelet transforms can outweighs the speedup of the convolution.
We are currently working on a CUDA implementation to overcome that :) see Appendix H for more details
shahaff32 OP t1_is4bbd7 wrote
Reply to comment by SearchAtlantis in [R] Wavelet Feature Maps Compression for Image-to-Image CNNs by shahaff32
Haar wavelet is also very efficient, as it can be implemented using additions and subtractions (and maybe a few bit manipulations) :)
You can also see Appendix F where we tested several others :)
shahaff32 OP t1_is2o2yz wrote
Reply to comment by Ecclestoned in [R] Wavelet Feature Maps Compression for Image-to-Image CNNs by shahaff32
Thank you for your interest in our work :)
We were not aware of these recent works. Thanks for sharing :) we will definitely check those out.
shahaff32 OP t1_is1tuuf wrote
Reply to comment by hughperman in [R] Wavelet Feature Maps Compression for Image-to-Image CNNs by shahaff32
That is indeed possible, though at a computational cost. The Haar wavelet can be implemented very efficiently because of its simplicity.
Please see Appendix F, where we shortly discuss other wavelets and their added computational costs.
shahaff32 OP t1_is1dlvu wrote
Reply to comment by pm_me_your_ensembles in [R] Wavelet Feature Maps Compression for Image-to-Image CNNs by shahaff32
Thank you for your interest in our paper :)
shahaff32 OP t1_is1cvgx wrote
Reply to comment by pm_me_your_ensembles in [R] Wavelet Feature Maps Compression for Image-to-Image CNNs by shahaff32
With some modifications to the code, I believe it can :)
shahaff32 OP t1_is13c2c wrote
Reply to comment by londons_explorer in [R] Wavelet Feature Maps Compression for Image-to-Image CNNs by shahaff32
We are in fact doing the backpropagation in the wavelet domain :)
The gradient simply goes through the inverse wavelet transform
​
See WCC/util/wavelet.py in our GitHub repo, lines 52-83 define the forward/backward of WT and IWT.
shahaff32 OP t1_is0ths5 wrote
Reply to comment by londons_explorer in [R] Wavelet Feature Maps Compression for Image-to-Image CNNs by shahaff32
This is aimed mostly at edge devices, where an accelerator is not available (e.g. mobile phones), or you want to design a cheaper chip for a product that requires running such networks (e.g. autonomous vehicles)
This work was, in fact, partially supported by AVATAR consortium, aimed at smart vehicles. https://avatar.org.il/
Submitted by shahaff32 t3_y22rk0 in MachineLearning
shahaff32 t1_jbxy6vj wrote
Reply to comment by ghostfuckbuddy in [D] Is Pytorch Lightning + Wandb a good combination for research? by gokulPRO
It might be a bug, and it might also be the case that we are the "idiot users" that don't get how to use the package correctly.
Either way, we decided to continue without Lightning (because of that reason and a few others), and therefore we need the code in pure PyTorch for future works that rely on it.