It did help, and thanks. I tried to leave out as much as possible to shorten the post but I realize that my lack of knowledge made me leave out the wrong/too much. Trying again:
The device I want to run is a small ARM-based computer using about 2W and accepts 5V (5.5 and over is said to burn it so I need the voltage to be pretty stable). What I want to do is to be able to run it both from a battery and socket power. When running from an external source I want to charge the battery at the same time.
I'd use an adapter when getting electricity from an external source so in would be somewhere from 5-18 V depending on what works best. I haven't decided yet if I'd want to have a regulator accepting a big span of incoming volts, but for now I'd just go with one and maybe in the future extend the capabilities.
For battery I'd go for a small/flat one, I think around 900-1700 mAh to get at least a theoretical 2-4 h uptime on battery. I haven't decided what type but I guess from what you wrote about lowering the power in 6-7 V out from the battery is what I'm looking for. These preferences makes me think that a Li-Ion or NiMH is my way to go.
About the charging would I want to be able to connect to something (I guess a microprocessor) to be able to obtain info about the battery. Of course the quicker charging time the better but that isn't really crucial. The info that I ideally want is: Are we running on battery or not, how charged is the battery and if discharging at what rate (to calculate how much longer until it's dead).
So there are still a couple of ifs and buts and maybe I'm trying to solve this problem backwards? My thought was that the circuit for charging/regulating the battery/voltage would be the tricky part and when finding out that part get a battery that suits what I find. Should I rather first find the battery I need and then try to hunt down info and most likely help about it?
Also, does these "smart" charger circuits have a name so I can try to read a bit about it? Is this something common enough to be able to buy ready or is this something very custom that'll need either a whole lot of money or time? I'm fumbling in darkness and would like to at least know a little bit about it (it's not the main goal of my little pet project but having a general idea would be nice).