Why does your CD player need an output buffer? An output buffer won't drive a speaker unless it is an audio power amplifier, a completely different animal.
A "passive preamp" is a euphemism for a high-quality attenuator. See http://www.dact.com/html/passive_preamp.html.
You don't need a buffer unless you have a long cable between your "passive preamp" (snort!) and your power amp. A simple transistor buffer (BJT or FET) will undoubtedly make your audio quality worse than it was originally (without the "passive preamp" (snort!)). If you need a buffer, it will need to incorporate an op amp.
Makes you laught doesn't it! - the imaginary improvement of a 'passive preamp' (just a volume control), them adding an active buffer to it - so it's no longer passive!