No one recommends that people only listen to experts. Experts make mistakes and also can be guilty of epistemic trespassing, all of which should make you skeptical about fully entrusting experts always. Healthy skepticism is advisable.