A phone with a jack, and the headphone cable could have physical control buttons. I have this combo and it's the best. The buttons on the headphone cable are easy to access.

I've got a sony Xperia 5 II (has a sd slot), and a pair of fiio fh5, and a pair of bullets from the one plus 3. They do their job.

There's mmcx or 2pin cables that have control buttons and a microphone too.

I've got an app for music, one for podcasts. The podcast one can resume any audio from the last played position.