But deleted posts should still count towards reputation obviously. Many useless trolls come on here and post such ridiculous crap and then they get -4 and the post gets deleted. That is the system working in my opinion.
If you have a specific problem with a specific downvote PM the mods, they then contact the downvoters and ask for an explanation - if the explanation isn't good enough then they usually remove the downvote.
It just seems like you don't like that you are regularly downvoted. It reminds me about the joke of the guy who meets 3 a-holes in a row in the morning.