I started a similar project a while ago, before getting into microcontrollers...
Basically, I wanted to play a song outside as the sunset every night. I came up with a $5 MP3 player that ran off a AAA battery, 16 meg (holds about three songs). AudioGuru helped me add a simple transistor switch, so it would start playing automatically when power was applied. Spent $4.95 on a small Gameboy amplifier with two small speakers, runs of 6 volts. kind of got stuck on how to drop the 6 volts down to 1.5 volts, without a huge waste or drain, and not have to go with bigger solar panels and batteries...