An LM317 needs a minimum load current of 10mA at high output voltages which is usually supplied by the 120 ohm resistor from the output to the ADJ pin and the other resistor from the ADJ pin to ground. If the minimum load current is not provided then the output voltage rises.
There are many low current low voltage regulators available.
I have never used a low current adjustable regulator.
Go to Google and enter LM317. There is a link to National Semiconductor where there are two low current regulators listed.
I found the LP38692 using Audiogurus suggestion. It has under 100ua
quiescent current drain with 1ua during shutdown. It's made by
National Semiconductor.
The LM317 can be used with a 220 ohm resistor also which drops current
down to about 5ma but that's still a bit high for battery operation.
Manufacturers are becoming aware of the importance of quiescent drain
in modern battery powered equipment so we might see more of these
lower drain power regulators coming out soon.
If the resistor is increased above 120R is might not regulate properly on lighter loads.
I've used the LM2936-5.0 before for this kind of thing - I think it's a brilliant regulator because it's low dropout (200mV), low quiescent current (15:mu:A) and very rugged.
I upgraded the 486/66MHz processor in my old pc to a 486/100MHz. The new chip was a surface mount IC on a daughter board with a 3.3V regulator. The daughter board plugged into the socket for the original full-size IC. No heatsink.
Maybe time to upgarde again or are you running DOS 6?
LOL
EDIT: I harvested a few MB's just to show the kids how to do it. About the only thing I used so far has been the coin cell battery holder. I saved a few of the other chips but do not even recall what they were.