Dude! You should go back and continue on. The David Tennant and Matt Smith years were the best of the whole modernized series. I have to admit, I have not been a fan of the last 2 doctors. I'm trying to like Jodie Whittaker but she gets scared too much. It's hard to explain. It's a bit 'un-doctor-like'. lol.
Also Torchwood, the spin-off show is pretty good.
I understand what your are saying. The doctor is generally so alouff about everything. Nothing seems to phase him. It would be weird to see one shaken all the time.