Here is a simpler version using a TL431/LM431. It Cuts In at a higher voltage, and Cuts Out at a lower voltage, controlled by R3. The CutIn is settable over a range of 3-4V using the Pot. The quiescent current is very small.
In the simulation, I show what happens when the solar panel puts out less current than it takes to run the load, i.e. the load is off until the battery charges, then the load turns on but the solar current is not sufficient to supply it, which means the battery discharges until the load is turned off again. Obviously, if the solar current is more than the load requires, then the battery keeps charging and the load stays on.
Note that the solar panel is modelled as a current source, while the battery is modelled as a HUGE capacitor. The time axis in the simulation is arbitrary...