A lot of the **broken link removed** that go with the plugs you find on AC plug-in power supplies (I call 'em wall-arts) have a normally closed switch contact which is opened by the act of plugging in. You could wire the battery though the switch contact so that the battery powers the circuit when nothing is plugged in.
Otherwise, two silicon diodes with their cathodes wired to the circuit (load), and each anode connected to battery and wall-wart, respectively would work, too.