It's what you can't see that you need to worry about. The vapour from transfering gasoline drops down below your car and will end up all around your engine bay, a spark could easily occur in this area (from a shorting ignition wire) igniting the fuel vapour and potentially the fuel going into the tank, and the fuel in the tank. Should this happen you will have an extremely bad day.

With the engine switched off there's no electricity going around in your engine bay and so no real chance for sparks. Accidents will always happen, but prevention can reduce the frequency by a huge amount, and that means lives saved or at least not destroyed.