It can inhibit participation. A lot of people automatically downvote anyone with a negative score and upvote anyone with a positive score because of the peer pressure or something. It’s how the brigading and triple digit downvotes often get started.

To be fair, it could have a negative impact on whether or not new users or lurkers get involved.

I guess they are a signal of disapproval, but its better to be honest and say what you think then to try and get liked in every post.