You win some, you lose some.

That said, I generally use the down-arrow sparingly on this site, specifically against abuse, trollery, or willful stupidity.

It would be sad if the down-arrow became a general "I disagree" button like it is on reddit.