After all, I take it all depends on what kind of model we're working on, how much we weigh the importance and likelihood of possible risks associated with it, and how to act and measure accordingly.

Thank you very much for your thoughtful input. It's been really helpful!

Thank you very much for your detailed explanation, trnka. It's been really helpful! It seems inevitable to have lots of unexplained issues in the process and I guess we can't expect to be perfect all at once :)

How would you weigh the importance of validating/testing a model? (maybe it depends on sector/industry?) As a beginner, I hope I'm not putting too much time and effort into it than I should be.

Many thanks for your answer! I'll definitely do that. I'm also wondering if there are some kind of tools, services, or even methodologies that help pre-screen potential model defects or that catch unexpected reliability issues the model might have, so I can improve the model quality and accuracy with various methods.

I've finished training a model, but I'm not confident about how to test or prepare it against unexpected risks in terms of trustworthiness and reliability when deployed. Are there some kinds of rules of thumb or any recommended methods to thoroughly test a model against those unseen risks?