The battery made it quiet because a battery is DC. Your power supply has some AC on its DC, or it has a terminal connected to earth.
1W is not loud. You need a more efficient speaker or more amplifier power.
What is scratchy sound? No bass frequencies? Distorted?
A speaker produces no bass if it is too small or does not have a properly designed enclosure.
An amplifier produces no bass if the value of one of its coupling capacitors is too low.
The output of an amplifier is very distorted if its input signal level is too high which causes the output to clip.