Yes, and I misunderstood your comment about private keys. I thought you were suggesting the private key are sent to the server.
The idea of the server keeping the vote secret raises two issues:
Possibility of insider abuse of the secret information (as was alleged with daily fantasy sports and contributed to potentially almost killing a billion dollar industry). This is likely a higher burden to secure compared to static javascript code.
You can't prevent people from revealing their votes early without adding even more complexity to it, and it is often (always?) to their advantage to do so. So this incentivizes votebots that talk to a private server to share their votes and collude.
Another disadvantage: doubles the transaction rate for voting. Those are likely to be the largest share of transactions, right?