I opened an account trough them and haven't had any problems. My plan for was to buy about 100k$ in crypto then sell it for USD to buy property. I was able to increase the 10k liquitidy limit to 100k without any issue. If I thought I didn't have to give my home address for that than I would agree that they are a pain to deal with and are acting like banks. I haven't bought any crypto with the instant function so I have one balance.
Granted I have yet to cash out so I am unable to speak on that but so far so good.
To lower some of the fees I would learn some basic trading skills and learn how to post a maker order so you pay zero fees.
I would say stick to one exchange for doing your taxes and for things like Blockfolio, take the value they give as am estimate. Any prices needed for calculations, use the same exchange consistently.
I haven't found the need to use the app. Trading via GDAX's mobile compatible site works well enough. Maybe its the instant purchase part for Coinbase your referring to. I'd say stick to doing important transactions on a desktop.
As for the wallet, be fine with loosing it all and if your aren't, you have too many coins in there. Using 2FA is also a must and use Google Authenticator and not Authy which just sends codes to a phone number and not necessarily your device phone or tablet.
I don't think users have access to their private keys because only one person can have custody of those keys, either you or the exchange. While they hold them for you they give you access to their trading platform for a fee. One could go meet someone in person to trade but it would present many problems and risks. Coinbase feels that their service far more valuable than trading with someone in person.