Inquiry

How to know the actual voltage of a rechargeable lamp battery if there is no label attached to it

Measure it with a voltmeter.

JimB

Most modern ones use a single lithium-ion or lithium ion polymer (lipo) cell, at least voltage-wise; they may have a pack with several parallel cells, with some types.

If it charges from a USB connector, the battery is highly likely to be a single lithium cell, voltage-wise.

So, probably a "3.7V" nominal battery, 4.2V at full charge & down to somewhere around 3.3V fully discharged.

Charging more than one in series add some complications and safety issues that add cost and complexity, plus needing a higher voltage charger system, so it's usually avoided if possible for things that do not need higher voltages.

Older ones used NiCd or NiMH batteries, with some number of cells in series.