A blinking LED performs the same with or without distortion.In my application, I am passing the output signal of this amplifier through a low pass filter, and through more circuitry to isolate bass beats, and in turn drive an LED. So I don't think distortion is particularly a big worry for my application?
I think you need AGC because when the bass level is low then the LED does not light and when the level is high then it might light all the time.
A N-channel Jfet can be used in an AGC circuit but I am sorry I don't have the time now to design it or find one in Google.