Awesome! I'll add it to my reading list :)

Nice. I wil have a look and possibly recommend it. Thanks for sharing, that must have been a huge amount of work

Nice, thanks for sharing your hard-work!

I love your book and refer to it often. I keep hitting F5 for Chapter 19. :-)

Btw, I absolutely loved your computer vision textbook, clear, comprehensible and so much fun! Best visulations in the biz. Also loved your UCL course on the subject, I was there 2010/2011 -- will definitely check out the next book

As a practitioner, I am surprised to see no chapter on finetuning

Obviously haven't had the time to read through it, and this is a clear nitpick but i really don't like when sites like this force you to download the files rather than display it in the browser by default

having a skim, it looks good mate. i like your writing style. please let us know once it's published and we can pick up a physical copy.

[removed]

Will you add a section for Forward-forward algorithm?

Looks good and fills a gap, imo there is no good DL book out there. What could be better: state results more clear as a Theorem with all needed assumptions.

I'll give people the choice in the end...

how recent is your chapter on generative models? I’m starting to pursue research in the area and need to get up to date

Added to my reading stack. Thanks for this. Will revert with comments.

Wow this looks like an amazing resource!

There's another book with the same title. Understanding deep learning

>u/SimonJDPrince

could you share a link to a CV textbook

Very nice work! Do you plan to release any solutions to the problems?

Will be please to look at it, especially the figure explaining algorithms, Thanks

Fitst impression base on transformers figure : You book could become a best seller…

I really like the book for it provides many beautiful pictures and gives us many intuitions behind deep learning algorithms, really appreciate the work you have done on this book.

That one’s full title is actually “ Understanding Deep Learning: Application in Rare Event Prediction”

This probably: http://www.computervisionmodels.com/

Many thanks for sharing

Didn‘t Hinton just try to start talking about it again at NeurIps? Isn‘t it like super irrelevant right now or am I missing something?

I will release solutions to about half of them. Have to keep the rest back for professors. You can always message me if you want to know the solution to a particular problem.

I'm planning to add extra material on line for things like this where it's still unclear how important they are. If they get widely adopted, I'll incorporate into next edition.

Yeah -- I feel a bit bad about that, but as someone else pointed out, the title is not actually the same. I should put a link to this book on my website though, so anyone looking for this book can find it.

There are five chapters and around 100 pages. I think it would be a good start.

Yup -- some of it is a bit out of date now, but the stuff on probabilistic/graphical models is all still good and so is the geometry.

Can you give me an example of a review article or chapter in another book that covers roughly what you expect to see?

Sounds great, thanks!

[deleted]

I can't give an example material outside of just huggingface documentation but it's the big thing right now to leverage pre trained models so if your book doesn't mention it then it's missing the hipest thing. And also adapterhub.

In page 41 just near problem 3.9 you write twice the. Do you need this type of comment too ?

Yes! Any tiny errors (even punctuation) are super useful! Couldn't find this though. Can you give me more info about which sentence?

I'd say that mine is more internally consistent -- all the notation is consistent across all equations and figures. I have made 275 new figures, whereas he has curated existing figures from papers. Mine is more in depth on the topics that it covers (only deep learning), but his has much greater breadth. His is more of a reference work, whereas mine is intended mainly for people learning this for the first time.
Full credit to Kevin Murphy -- writing book is much more work than people think, and so completing that monster is quite an achievement.

Thanks for tip about Hacker News -- that's a good idea.

just above 3.1.2 sum of slopes from the the regions

What is the most efficient way for someone to tell you about typos, or provide suggestions? I'll try to have a read over the weekend.

Sorry, you've written the instructions right there on the Web page! Just ignore me...

There's another on page 349 in section "Combination with other models":

>...will ensure that the the aggregated posterior...

Definitely would like something like this. Maybe SOTA benchmarks as well.

Fine tuning isn’t any different than just training…? You just don’t start with random network, but fine tuning doesn’t really have anything different besides that and the size of the dataset

That was kind of my impression. And I do discuss this in the chapters on transformers and regularization. Was wondering if there is more to it.

Thanks! If you send your real name to the e-mail on the front page of the book, then I'll add you to the acknowledgements.

When to reach for deep learning over older, simpler methods. Just an executive summary to keep folks from sandblasting soda crackers, or being forced to.

That's not a bad idea actually!

Looks like a great book so far. I think it is definitely valuable to focus on giving a clear understanding of some topics rather than covering everything while compromising depth of understanding

I have one minor gripe about deep learning textbooks. I think they are great references, but should not be used as a way for beginners to get into the field. I genuinely feel like time is better spent on the student going down a rabbit hole of actual papers of maybe one of the chapters of those books, say, a student reads the chapter on graph neural networks and the proceeds to read everything in graph neural networks, rather than read the whole book on different subsections.

Agreed -- in some cases. Depends on the level of the student, if they are studying in a class etc. My goal was to write the first thing you should read about each area.

Random ideas from the top of my head:

• intro why transfer learning works;
• old but good https://cs231n.github.io/transfer-learning/;
• a concept of catastrophic forgetting;
• some intuition on answering empirical questions like what layers should be frozen, how to adapt LR etc.

Thanks. This is useful.

As a novice, I’m not understanding why the test loss continues to increase— in general, but also in Fig. 8.2b, if anyone can explain… The model continues to update and (over)fit throughout testing? I thought it was static after training. And the testing batch is always the same size as the training batch? And they don’t occur simultaneously, right? So the test plot is only generated after the training plot.

You are correct -- they don't usually occur simultaneously. Usually, you would train and then test afterwards, but I've shown the test performance as a function of the number of training iterations, just so you can see what happens with generalization.

(Sometimes people do examine curves like this using validation data, so they can see when the best time to stop training is though)

The test loss goes back up because it classifies some of the test answers wrong. With more training iterations, it becomes more certain about it's answers (e.g., it pushes the likelihood of its chosen class from 0.9 to 0.99 to 0.999 etc.). For the training data, where the everything is classified correctly, that makes it more likely and decreases the loss. For the cases in the test data where its classified wrong, it makes it less likely, and so the loss starts to go back up.

Hope this helps. I will try to clarify in the book. It's always helpful to learn where people are getting confused.

I see, thanks! This seems like a great resource. Thank you for making it available. I’ll post any further questions here, unless GitHub is the preference.

Fig. 8.5 mentions “brown line” for b) but line appears to be black.

GitHub or e-mail are better. Only occasionally on Reddit.

Thanks! Definitely a mistake. If you send your real name to the e-mail address on the website, I'll add you to the acknowledgements in the book.

Let me know if you find any more.

Please let us know when it's published so I can tell my university to buy some copies for its' library!