It's a wonder you're not destroying your display. Put a 120 ohm resistor in each of the seven segment lines (not in each of the six common lines). Of course, only turn on one transistor at a time. The 7805 will still dissipate 1 watt maximum, so if it still gets hot, you should get a bigger package style, and/or heatsink it.