It's that easy:
Just upvote a question or answer by accident and then try to redo your upvote by clicking on the downvote button.
You might think this will undo your upvote but that's not the case: It immediately works as a downvote (and you aren't asked whether you mean to do so).
Well, what have you expected? A downvote is a downvote. To undo an upvote, just click on the upvote button again. (And to undo a downvote, guess, just click on the downvote button again.)
And please note: Once you clicked on the downvote button, you get your "Critic" badge, and you will hold it no matter whether you left any downvotes permanently or not...
Call me "Critic by accident":)