If it's "distorted" as in the sound gets really muddy and it doesn't sound like the speaker is clipping (unable to keep up with the input signal) then damping the outer panel can help:
http://store.secondskinaudio.com/
Products like Second Skin's Damplifier will deaden the metal behind the speaker, reducing its reaction to the sound wave from the back of the speaker and the creation of a secondary distorting sound wave. If you like a lot of bass in particular it makes more of a difference. If you seal the door (damp inner and outer panels and close the inner panel off) you get closer to a wooden enclosure and away from a metal, vibration prone box.
I run a Pioneer head unit with stock speakers. I've fully damped and insulated all of the doors and most of the interior with the exception of the roof (someday I'll get around to it). It makes a big difference in audio clarity and I don't really feel the need to replace my speakers or add an amp as the apparent volume is boosted quite a bit. Really not sure I'd do aftermarket high power speakers without damping anyway...