You can put several normal 1A diodes in line with the fans. Each one will drop the voltage the fans see by 0.6~0.8 volts. You'll need quite a lot to drop it far enough. Could also be line noise on the PSU though, caused by running them at such a low load condition. If you have something 24v that can draw ~2 amps, you may want to hook that up to the PSU to load it some, then hook up the fans and see if that fixes it. A good 12v lamp or two might work well.