You have to be assertive in order to fend off a bully. As someone who was bullied a lot when in school, I really think I'd have been respected more if I'd have just looked him in the eye and said, "Suck my fat one you cheap dime-store hood."
I would never recommend physical violence against a bully (self defense is of course acceptable) even if they're asking for it. As for how they're viewed, however, I think it tends to be a little of both. There are some bullies who are the most obnoxious and mean-spirited people in the world, and it turns out that they actually have the most traumatic background you can imagine (even someone as evil as the killer from Scream had a fucked up family life).
Some are also just truly sick minded people though. There was one kid in my high-school way back in the day who was horrifically mean to everyone and even said he got a "high" off of making people angry. In the early days of the internet he'd even go on suicide help forums and tell people to kill themselves and insult them (btw, he wasn't the victim of anything that would cause this twisted of behavior). So in his case I'd say it's definitely a situation where the bully deserves to be bullied. He was the closest thing to evil I've ever personally met in my life.
The cause and solution vary from situation to situation though.