If it takes so long to get to a functioning voltage you have no usable output at all to start with, if everything goes via that. That's the problem with a capacitor, it's working range is from 0V.
A battery has a much smaller voltage "span" between discharged and full charge.
If I'm reading the supercap values right (2 x 50F in series) you need 282 joules to reach 4.75V, with limit of 344 @ 5.25V
That gives you just 62 joules usable stored power.
Rounding to 5V, you have a maximum power transfer rate of 2.5 joules per second at half an amp. Around 25 seconds charge or discharge to the limits of that range, @ 500mA.
OK;
treat the supercap as a battery and have an alternate path around it.
eg. Regulate slightly higher to allow for a couple of schottky diode drops, feed straight from regulator via two diodes to load and also via diode + current limiting resistor to the cap and diode from cap to load.
Limit the charge current to eg. 250mA and you have another 250mA available instantly for the load, while still "topping up" the cap within it's working range in around one minute
You could improve efficiency by using FETs to route power depending on the supercap state of charge & use a switched-mode current controller rather than just a resistor, so the main regulator still operates at the target voltage. You could also split off a second "charge" regulator from the high voltage section.
Re. a series inductor - it would maintain current flow; if your device suddenly takes less current, the voltage will spike.
Use capacitors to damp the spikes and/or add a large reservoir cap plus a ceramic for noise after the inductor, to even out the current through the inductor.