Agreed, you need a separate amplifier, you might be able to get away with four 64Ω speakers but it wouldn't be very loud.
What sort of speakers are these?
Power rating? Impedance?
How many of them do you have?
If you don't want to bother building more than one amplifier you could use an amplifier designed for a 4Ω load and use 64Ω speakers. This will enable you to drive up to 16 connected in parallel. This idea is only good for small speakers, if you want a PA then you'll need a 100V system.