There are several issues that may be causing the pop. Pretty much all simplify down to a sudden DC level change. They depend on what you are doing to "turn on" the amplifier. Are you using an amplifier chip that has an enable pin? Are you fet or transistor connecting the speaker? Or are you enabling a signal into an amplifier that is always driving the speaker?
Some things that cause speaker pop are having a dc offset build up on the coupling capacitor that connects to the speaker. The other issue can be the phase of the signal that suddenly is driving the speaker (consider an output signal that is at the peak of its cycle, and you suddenly enable it through the amplifer, you go from 0 volts directly to the max voltage). Probably a few other causes, its been a while since I had to design around this issue.
JimW