It would be quite dumb to remove downvotes, suicide even.
It is equally dumb to fight about them and talk about them all day.
Each person uses their stake upvote what you like, ignore what you don't care about and downvote what you think is harmful.
We don't need controlling possies telling people what to vote for.
Of course they have the right, but that doesn't make it smart.