Hy kinarfi,
Don't get me wrong, SMPS are used with both audio circuits and RF circuits. In fact one of the best audio amplifiers uses off-line switch mode power supplies, but the design, filtering and layout are pretty complex.
**broken link removed**
By the way SMPS not only generate conducted interference but also radiated interference which couples to equipment by both inductive and capacitive coupling. There is a good book, 'EMC for Product Design' which covers the arena of ElectroMagnetic Compatibility (EMC) without getting too heavy.
https://www.amazon.co.uk/EMC-Product-Designers-Tim-Williams/dp/0750681705
It is true that USB power is universally generated by SMPs but the designers of mobile phones, PCs, etc have many resources to design out any problems. Also USB supplies are not up to HiFi standards. Compared to HiFi, the sound on my Android phone is full of distortion and hash and the sound on my laptop is even worse.
Audio, as you know, is generally taken to span the frequency range, 20Hz to 20Khz and RF covers the frequency range, 3KHz (VLF) to say 1TeraHz (microwave). Just in case you mistake 3Khz for an audio signal, remember that radio waves are electromagnetic and audio waves are compression and expansion of air.
You make a very good point that if the switching frequency of a SMPS is higher than the highest audio frequency any interference would not be audible, but the fact that the switching signal can get into the circuitry is not good and can cause problems.
Very often a high frequency signal will get rectified by the input semiconductors of an amplifier and produce an audio signal and a DC component, both of which can cause problems. For this reason audio amps often have a low pass filter on the input to prevent high frequencies from getting into the amplifier. These high frequency signals get picked up by the leads connecting to the amplifier for example. One of my home made audio amplifiers was an excellent broadband radio receiver.
With RF circuits the situation is much more complex and difficult to sort.
spec
PS: another link which may be of interest:
https://electronics.stackexchange.c...it-be-powered-by-a-switched-mode-power-supply