You should use COG components for any capacitors where the capacitance is critical to the frequency stability. They are far more stable than XR5 or XR7.
All the small value capacitors, 100 pF or less, will probably be most easily available in COG anyhow. Large capacitances, over 10 nF or so, have such low impedances at crystal frequencies that if their value changes, it won't affect the crystal frequency significantly.
It is the intermediate values, 100 pF to 10 nF, where you might get a choice of COG or X5R/X7R. That is where you should chose COG for oscillators.