Of course the base of the oscillator transistor has a capacitor to ground, it has a 1nF capacitor to the positive supply that is grounded by C1 and the output of the 5V regulator. The oscillator transistor will not oscillate without a capacitor causing it to be a "common base amplifier".
Your new circuit has the opposite of pre-emphasis since it cuts high audio frequencies (and cuts all stereo signals).
If you look at Harry's stereo encoder, it has a high output impedance so the oscillator capacitor will cut even more high frequencies.
EDIT: On Harry's website, does he say which of his transmitters will work with his stereo encoder?