Steel wool into holes is a good way of stopping mice, but let me stress this DO NOT PUT STEEL WOOL AROUND THE OVEN. I would definitely get an exterminator to come and assess the situation. I’ve also heard peppermint will deter them. I did hang bags of dried mint in and around my old apartment and it seemed to do the trick (although the plethora of steel wool likely helped)