I think you’re putting too much pressure on yourself. You shouldn’t be feeling any anxiety when your reading for pleasure. You can just keep going and if it turns out you missed an important detail, you can always go back later. And I wouldn’t worry about time investment. It may take you longer to get thru the book, but you shouldn’t worry about that if your reading for fun anyways. Go at your own pace.