You can use a boost converter to give a constant voltage above the battery voltage.
You can use a buck converter or linear regulator to give a constant voltage below the battery voltage.
You can use a 2-switch buck (or 2-switch boost?), flyback or forward etc. converter to give a constant voltage that may be above or below the battery voltage.
I use a "low dropout" 5V voltage regulator IC to power all my circuits that use a little 9 battery. Then the circuit is powered by a regulated 5.0VDC until the battery voltage drops below 5.5V.
An ordinary 7805 voltage regulator drops out (stops regulating) when the battery drops to 7V which happens very qucikly.
... or ... just to be smart... you can use an inverter to generate a negative output voltage and put the battery in upside down. This avoids going through the nasty buck boost region