Knowing you are being watched, followed, stalked, creates a terror inside of you. And this is made even worse by not know who it is; what their plans are; what their intentions are. This stress has been shown to shorten people's lives and make them more irrational then usual.
Of course, you can just learn to ignore it all. Sorta...
But, like you said, what really needs to be known is who, and what their plans are. Currently it is a group of thugs that want to take more of your money and make sure that you aren't planning to go after the thugs.
Being surveilled by a group of thugs that ARE out to get you is not tolerable and needs to stop. Either by the surveillance stopping, or the thugs being after everyone stopping.
As an example, surveillance in London has been massively increased. And has it increased safety? Not really. Has it increased people getting fined for stupid shit? YES.
I also know a person who takes this to the next level. He knows he is being surveilled, so he has his phone record his locations to a private server. He records important meetings and interactions (including all sexual encounters), again loaded to a private server.