A zener in parallel with C2 is a good way to limit the voltage. Either a 12 volt or 15 volts zener should be OK. It should be at least 2 watts power rating. You have the polarity correct. I think you should also raplace C1 as it could be breaking down as JimB suggested. I would reccomend a capacitor rated at above 250 volts AC (Rather than the 400 volt DC rating.) and X rated. (This indicates that thay are safe for this sort of application.) I think this capacitor is 1.5 uF from the 155j in the part number.
Les.