If you don't have an oscilloscope, you can't tell if the voltages have a large oscillation on top of the DC voltage. This could be mains voltage, or some other signal, being picked up and amplified or the amplifiers could be oscillating.
The idea of the capacitor between the output an the inverting input is to reduce the gain at high frequencies. When reading temperatures, you want to filter anything above about 0.1 Hz so large resistors and capacitors can be used to give a large time constant and a very low cut-off frequency.
A capacitor between the analogue input and ground can help, but you also need a resistor in series, as many op-amps will go unstable if you have a capacitor directly on the output.
Post your circuit if you can.