Another thing to take into account is that many witnesses are Hive developers as well, the largest in both aspects being Blocktrades. Ask yourself what your goal is for Hive and whether or not they are working towards that goal.
My goal is decentralization and I don't feel like the core devs are working towards this goal. When you have the power, it's unlikely you'll make the changes necessary to balance that dynamic.
Even if you don't mind who's in the top 20, nor how long they've been there, I think it's a great idea to switch votes around. If we can remind some of these 'representatives' who's really in charge, complacency is less likely to set in.
I think complacency is an issue right now and a mega reshuffling of witnesses is in order.
*Speaking to everyone/no one in general..