Yup, I'd be like that too when transferring Bitcoin. And BitcoinCash at times. Especially when mine wasn't included in the following block.
Not like that with ETH (after the CryptoKitties congestion). And with speedy (and ultra cheap fees) coins like DigiByte, Navcoin etc.