Thanks for quick replies!
Appreciate both suggestions. If I get a wall wart with 5V output, I could simply tag a resistor to drop off 1.7V. I don't know the current requirement of router yet. If it is 600 mA at the most, the resistor will be dissipating 1 watt max. LM317 option sounds good too; I will have to do some searching for the circuit.
That's nice to know i could disconnect the fan of PSU. If the heat buildup is too much, I could attach bigger heat sink I suppose.