Last time I checked, a PCB speaker was not expected to provide power or fidelity. Hence, I wouldn't worry about power wasted in connecting a resistor in series with an 8-ohm speaker to increase the impedance presented to the driving source and certainly not worry about the damping factor being ruined by the extra resistance. PCB speakers are usually there simply to indicate that something is happening -- when was the last time anyone really needed to hear the subtle dynamics, transparent clarity and other audiophool characteristics through the speaker mounted on an old dial-up modem board? If you need fidelity and power, you'll be using external speakers.
The world will not explode if you try driving an 8-ohm speaker directly from the output of the chip. The chip may, but maybe not. What's to lose other than a few bucks. If not specifically designed for driving the low impedance, I wouldn't do it if the design was going to be replicated 50,000 times for retail sale. But if it's just for you, who cares if it doesn't pan out?
Dean