Arigato!

Thank you kind sir

Thanks!

This! I would just add that you can never find the exact reason as to why your training from scratch is giving less accuracy. Do you have enough data for all the neurons to learn the features ? Can you cross validate the validation loss of your dataset and pre-trained one ? Did you try removing/adding a dense layer to check how the performance is changed ?

Oh, okay. Thanks a lot! The reason I don't feel like moving on to pytorch is because I am a bit comfortable in TF now. And I don't have the energy or time to learn a different framework right now.

I thought so too but then a few people also mentioned the same thing in my office. And currently, I am working on 3D computer vision and all the git repos I found for it has pytorch as its framework I barely found any with TF implementation

I couldn't find the blog but check this post in this sub itself

Lots and lots of training my frand

If you want to re-train the whole model with mixed dataset. The only option I can think of is transfer learning where you only initialise all the parameters in the same way which were used to train on the old dataset and re-train from 0th epoch

I believe that is an iterative process when doing transfer learning. First you will always freeze the top layers because low level feature extraction is done over there (extracting lines and contours). Unfreeze the last layers and try to train those layers only where high level features are extracted. At the same time it also depends on how different the new dataset is using which you are training the model. If it contains similar characteristics/features freezing top layers would be my choice

Agreed to this. It's a data pre-processing step making a model do this would be a very complicated task.