RichardBJ1
RichardBJ1 t1_jcxcu7e wrote
Probably need an answer from someone who has both and has benchmarked some examples. (EDIT: and I do not!) Personally I find a lot of “law of diminishing(Edit) returns” with this type of thing. Also for me, since I spend 100x more time coding and testing will dummy sets… the actual speed of run is not as critical as people would expect…
RichardBJ1 t1_izlbf63 wrote
Looks great, defo like to have a go with this.Perhaps epoch n/total epochs too though?
RichardBJ1 t1_iy7xvbr wrote
Was interested when I first head about this concept. People seemed to respond with either thinking it was ground shaking, …..or alternatively that it stood to reason that given enough splits it would be the case! Do you think though, that from a practical usage perspective this doesn’t help much because there are so many decisions…. Article has a lot more than just that though and a nice provocative title.
RichardBJ1 t1_ixtsp1x wrote
Reply to comment by ryuks_apple in Is Linux still vastly preferred for deep learning over Windows? by moekou
Yes I have a few Macs and do that all the time! …Really I was replying In the context of the OP post however…. setting up a new system for DL.
RichardBJ1 t1_ixtsdc5 wrote
Reply to comment by Appropriate_Ant_4629 in Is Linux still vastly preferred for deep learning over Windows? by moekou
Thanks, I hadn’t seen that implementation, I ran some deep learning models on TensorFlow adapted for iMac GPU, but it was less stable and significantly slower than the Nvidia Linux/Windows equivalents. Things have moved on I guess.
RichardBJ1 t1_ixs8s3c wrote
In my dept a mix of window/Linux. CUDA-GPU fitted machines, mostly TensorFlow /Python. No significant issues with either. But that is, in CS terms quite a limited remit.
RichardBJ1 t1_ixs8fsy wrote
Reply to comment by stillworkin in Is Linux still vastly preferred for deep learning over Windows? by moekou
Macs for deep-learning?
RichardBJ1 t1_ix9fcaw wrote
His book has some nice examples, works well. Really as the other answer has said though you need to follow your interests and apply those examples to something that interests you. Another idea is Kaggle; you can clone others code quite legitimately and understand what they were up to. So many examples on Kaggle you’ll surely find something that fits your interests!! Good luck
RichardBJ1 t1_ix7ficv wrote
Reply to GPU QUESTION by Nerveregenerator
I think if you get even similar performance with one card versus 4 cards the former is going to be far less complex to set up!? Just the logistics of that sounds a nightmare.
RichardBJ1 t1_iw733qt wrote
Reply to comment by jobeta in Update an already trained neural network on new data by Thijs-vW
Yes …obviously freezing the only two layers would be asinine! There is a keras blog on it, I do not know why particular layers (TL;DR). It doesn’t say top and bottom that’s for sure. …I agree it would be nice to have method in the choice of layers to freeze rather than arbitrary. I guess visualising layer output might help choose if a small model, but I’ve never tried that. So I do have experience of trying transfer learning, but (apart from tutorials) no experience of success with transfer learning!
RichardBJ1 t1_iw71rpv wrote
Reply to comment by jobeta in Update an already trained neural network on new data by Thijs-vW
Good question; I do not have a source for that, have just heard colleagues saying that. Obviously the reason for freezing layers is that we are trying to avoid loosing all the information we have already gained. Should speed up further training by reducing parameter numbers etc. As to actually WHICH layers are best persevered I don’t know. When I have read on it, people typically say “it depends”. But actually my point was I have never found transfer learning to be terribly effective (apart from years ago when I ran a specific transfer learning tutorial!). In my models it only takes a few days to start from scratch and so this it what I do! Transfer learning obviously makes enormous sense if you are working with someone else’s extravagantly trained model and you may be don’t even have the data. But in my case I always do have all the data…
RichardBJ1 t1_iw43b43 wrote
Well transfer learning would be the thing I would expect peep to say, freeze the top and bottom layers, re-load the old model weights and continue training….. but for me the best thing to do has always been to use throw the old weights away and mix up the old and new training data sets and start again…. Sorry!!
RichardBJ1 t1_jcy0nse wrote
Reply to comment by funderbolt in How noticeable is the difference training a model 4080 vs 4090 by Numerous_Talk7940
Ta!