I believe the very little solar IR gets through the atmosphere. If I remember right, the sky looks black in an infrared photograph photo.
You can help correct for bias drift by having the "zero" output voltage recalibrated everytime the robot comes to a stop (ie. the motors have been deactivated with brakes on for more than 1 second). This will fight the drift involved in the sensors' bias. You might actually want to program your robot to come to a stop for at least 1sec every time it stops so you can do this calibration more often. This should be effective in fighting against drift due to temperatures and other such things (it won't correct for the sensor's scale factor though)
If you used IR/sonar beacons then you could be more accurate for longer periods of time, with only drift-vulerable sensors filling in for the beacons when you lose contact. Basically with this method you are using localized "IR/sonar GPS" with the inertial sensors filling in for when you lose contact until you can restablish contact. Just to reclarify, the IR/sonar method sends an IR signal and a sonar pulse at the SAME time. Because light and sound travel at different speeds they will reach the robot at different times. You can use this time difference to calculate how far your robot is away from that beacon, enabling you to draw a circle around the beacon indicating possible positions of your robot. If you have 3 beacons (most effectively spread out to surround your robot from all directions) then you have 3 circles which will only intersect at one point. This is where your robot is. (Of course, reality will not make a perfect intersection, you will have to use an algorithm to narrow down where you might be in the "gray" area. Read up on GPS and see how they do it- to put it simply, they adjust the all the circles in EQUAL amounts until an intersection is made. You could also find which point in the gray area has is the same distance to each circle's edge.
If you want REAL check points (as opposed to beacon-positioning with inertial dead-reckoning during times of lost contact) then use RFID tags or something since their range is much smaller than IR and does not suffer from light of sight. If you sense one, you know where you are with much greater certainty. using just IR for a beacon, may not be a good idea since it's too easily visible from far away.
RFID tags are way simpler to use. YOu don't need to build a minimum of 3 IR/sonar beacons. You don't need position the beacons around your yard every time. You can just hang a few RFID in an RF transparent weatherproof plastic box in some trees or something (and make sure your robot knows where they are supposed to be). And you if you use the passive kind you need no power supplies. The price you pay is that if probability plays against you and the robot does not pass by a RFID tag for a long time, it's position errors wll continue to accumulate. But as soon as you hit a tag, bam, your error is reset.