FIrstly, the negative connections of the load, the primary and the backup power should be connected together.
It's very common to have two DC power sources connected with diodes to a load. That way the largest voltage will power the load. That is a very simple system and for low power supplies it works very well. However, there will be a small voltage drop of around 0.8 V from the diodes, and you may not want to use the highest voltage supply.
If you have a relay, you need a change-over relay, which is one with three contacts, "common", "normally closed" and "normally open", usually labelled C, NC and NO. Relays like that connect C to NC when the coil is not powered, and they connect C to NO when the relay is powered.
For your situation, the C should be connected to the +ve of the load. NC should be connected to the backup power, so that is used when there is no power to the relay. NO should be connected to the primary power, so that is used when the relay is powered.
There are two possible problems. Firstly, there will be short break in the power when the relay changes over. Secondly, when the voltage drops, and the relay is not powered, there will is no load on the primary source, its voltage is likely to rise. That could cause the relay to turn on, putting the load back onto the primary source. That will cause the voltage to drop back, and the whole system could oscillate with the relay switching on and off constantly. With solar cells and a battery, that oscillation could happen anything from several time as second to every few hours, or even once a day as the sunlight changes.