As long as you only ever connect the ground of the oscilloscope probes to the circuit ground (PSU negative) it should not be a problem.
It should become a habit that you never connect a scope ground clip to anything that could have a voltage on it, to avoid damage to both the scope and equipment. (And yourself).
If you need to view a signal relative to a non-grounded point, set the scope in differential (subtractive) mode, as Ron says.
Temporarily put both probe tips on a signal or test point and adjust the channel gains so the signal cancels out, then you can display the difference between the two probed points - without the ground clips ever being at anything but ground / 0V.
ps. One other thing a lot of new users don't realise: Set the probes and input mode selects to "x10" and leave them there.
x1 causes the circuit to you are testing to be put under some load, and whatever voltage is fed directly to the scope. Plus the frequency response is usually rather poor.
x10 add a frequency-compensated attenuator in the probe which reduces probe load & helps protect the scope inputs - plus giving a much better high frequency response.
You need to use the calibration output of the scope to adjust the compensation trimmer in each probe, to get the cleanest square wave, as near to perfect square corners as you can, & then preferably keep the same probes on the same inputs. Once they are adjusted to that specific scope, they should be OK for years; just verify the square wave occasionally.