Actually I see reasons in your proposal too as many might make that kind of mistakes too unknowingly ,but at the other end I want to believe the confirmation window is their to be of help before you finally proceed to the last stage of confirming the process.
On a good note,if it can be moved to somewhere else just like you proposed also, is a very good idea in which am in support of too,but I also believed that if that happens by mistake too at times,which one way or the other I am not really sure such should happen cause you have one step to go through before the final confirmation and if it happens you can still un-downvote such content after noticing it.
Yes I know we can revoke our action but it might happen that one doesn’t even realise that they have downvoted a post.
Honestly it is very easy to make the mistake and that is the reason i want to suggest it should be far from each other.