Since you read mainly for self-improvement and learning, then perhaps you could:

• Research and create a list or lists of books that fit your criteria.

For example, many classics give us better understanding of history, society, culture, etc. Yet, they are not all equal when it comes to influence, education, being topical or offering interesting insights and food for thought. And it is also true for the contemporary writing. So it is worth to check various lists and go for those that are on school/university obligatory read lists, suggested by various experts or recommended by newspapers and magazines.

Many authors have also written shorter novels, stories or essays. So if you struggle with reading longer books by influential authors, those may be a great option for you. It's like a tasting menu.

In your list, you can also add keywords or notes on why did you choose this topic.

This way you will have your own list(s) that will not only make decision making easier but also will motivate you to actually finish the book because you will know that it is beneficial.

• Choose reading challenge that fits you and stick to it.

I found that reading challenge is a great motivator to actually choose and finish a book. There are so many to choose from - generic, only fiction or non-fiction, genre specific, etc.

Finally, if you only enjoy reading historical fiction and non-fiction, then there's nothing wrong with that. There are plenty of people who read mostly 2-3 genres and that's okay.

Edit: specifications