A more common way to do this is to put a resistor in series with the input to the regulator. Divide the voltage you want to drop by the current drawn by the regutor to find the resistor value. The resistor power rating will be v * v / r.
(Resistor size) = (Voltage I want to drop)/(current from regulator)
=17.4V/.174A
=100Ω
(Resistor Power Rating) = [(Voltage I want to drop)(Voltage I want to drop)]/Resistor
=[(17.4v)(17.4v)]/100Ω
=3.0 W
Derate by at least 2 for a greater than 6w power rating, Digikey has 100Ω/10W resistors
I am attaching my circuit into an existing circuit that uses a 24 VAC transformer. Sometimes, perhaps for extended periods of time, my circuit will only pull 5 mA. This would put the voltage into the voltage regulator at 38.5 V (39 volt supply - 0.5V load (100Ω X 0.005A)). This is 3.5 volts above the maximum amount allowed on the regulator.
2) In what way should I alter the above design if the load from the regulator is less than 4 amps?
As mentioned before, instead of full wave bridge you could half wave it, that would lessen the size of your resistor somewhat, but if you want to keep the bridge then you could put a 33V TVS device like this one across the input of your regulator: SA28A
If you change the 7805 to an LM317 you can have a differential of 40 volts from input to output. It is 2 more resistors.
In either case you will need a pretty good heat sink ( 6 or 8 C/watt).