That's essentially exactly what the board does - with USB connected it charges the battery - and the output will almost certainly come from the USB input, rather than the battery itself. However, if the USB input can't provide enough current the excess required will come from the battery, which will discharge rather than charge. However, that's unlikely to happen.
Without the USB input the battery simply charges the load.
It's really just like a car battery - the alternator provides all the electrical needs 'most' of the time, when the engine is running, and any excess is sent to charge the battery.