Avoid Capital by Piketty is my advice then.

But seriously, I zone out too and wonder what I listened to and lose track of details now and then. And then I find it's hard to backtrack to where I lost it, whereas flipping pages and scanning to reorient myself in the story is easy--but I'm old and grew up with dead-tree books.

It might be the reader. Don't give up. Go to the library and check out a few more with different authors and readers to see it's the medium or the messenger.