The physical interpretation of negative frequencies is as follows:
My first realization was that fourier is time-agnostic. That is, if you think about it, there is nothing in fourier analysis or the transform itself that can tell you what the 'direction' of time is. Now, imagine a physically oscillating system (ie a real sinusoid from say, a current over a wire) that is oscillating at some scalar temporal-frequency, f.
Imagine 'looking' down this wave, in the forwards direction of time as it progresses. Now imagine calculating its difference in phase at every point in time you progress further. This will give you your scalar temporal frequency, and your frquency is positive. So far so good.
But wait a minute - if fourier is blind to time, then why should it only consider your wave in the 'forward' time direction? There is nothing special about that direction in time. Thus by symmetry, the other direction of time must also be considered. Thus now imagine 'looking' up at the same wave, (ie, backwards in time), and also performing the same delta-phase calculation. Since time is going backwards now, and your frequency is change-of-phase/(negative time), your frequency will now be negative!
What Fourier is really saying, is that this signal has energy if played forward in time at frequency bin f, but ALSO has energy if played backwards in time albeit at frequency bin -f. In a sense it MUST say this because fourier has no way of 'knowing' what the 'true' direction of time is!
So how does fourier capture this? Well, in order to show the direction of time, a rotation of some sort must be employed such that a clockwise roation dealinates 'looking' at the signal in the forward arrow of time, and a counterclockwise roation dealinates 'looking' at the signal as if time was going backwards. The scalar temporal frequency we are all familiar with should now be equal to the (scaled) absolute value of our vector angular frequency. But how can a point signifying the displacement of a sinusoid wave arrive at its starting point after one cycle yet simultaneously rotate around a circle and maintain a manifestation of the temporal frequency it signifies? Only if the major axes of that circle are composed of measuring displacement of this point relative to the original sinusoid, and a sinusoid off by 90 degrees. (This is exactly how fourier gets his sine and cosine bases the you project against every time you perform a DFT!). And finally, how do we keep those axes seperate? The 'j' guarantees that the magnitude on each axis is always independant of the magnitude on the other, since real and imaginary numbers cannot be added to yield a new number in either domain. (But this is just a side note).
Thus in summary:
The fourier transform is time-agnostic. It cannot tell the direction of time. This is at the heart of negative frequencies. Since frequency = phase-change/time, anytime you take the DFT of a signal, fourier is saying that if time was going forwards, your energy is located on the +ve frequency axis, but if your time was going backwards, your energy is located on the -ve frequency axis.
As our universe has shown before, it is precisely because Fourier does not know the direction of time, that both sides of the DFT must be symmetric, and why the existence of negative frequencies are necessary and in fact very real indeed.