Refueling situations are a bit dangerous. News outlets regularly report on them. Negligence on the part of the driver and the person refueling leads to dire consequences.
Generally speaking, the surest way to overcome fear is to face it without fear. As you said, it's impossible to grow without facing challenges.
Are they dangerous?
There are people smoking while refueling.
There are people refueling in different ways.
There are vehicles leaving before the fuel tank is finished.