I think downvotes are important; they complement moderation and add a degree of QC. Furthermore, the possibility of occasional misuse is not a strong argument for taking away core functionality of the platform. If I didn't want downvotes, I'd go on mastodon.