the OP is trying to detect the presence of metallic objects by having the object act like a shorted turn and changing the voltage and frequency of the oscillator.
one problem with this question, is the OP has not mentioned what the detection distance is that he's trying to achieve, or the dimensions of L1, so it's impossible to make any solid recommendations, even for what possible noise sources to eliminate.
L1 is 20cm in diameter. The detection distance with this flicker noise and the filtering algorithm I mentioned is 10 cm. for a 0.3 g. gold nugget, which is not at all bad. A 1 EUR coin is detectable at 20 cm. and a can of RedBull at 100 cm. However, due to this flicker noise it's not possible to plot the amplitude vs. frequency changes in a meaningful way in order to achieve discrimination. A solid baseline would be required to achieve that.