You can also make an analogy with an electric generator. You can spin a generator with no load and it does not overheat or explode.
The OP's question is a good one because the question of what happens to the heat energy is probably at the center of the confusion. If you exert a force (or more accurately a torque) on an electric generator, and there is no load, then there is no electrical power generated because current can not exist (boy that sounds dumb, -sounds better when you say "can not flow", but anyway) with the circuit open. What happened to the mechanical energy you put in? Well, there is no problem because you did not put much energy (or mechanical power) in. The generator creates a back force when current is existing (ouch, that almost hurts my ears) in the generator windings. No current means much less back force which means much less mechanical power in. Remember power is torque times rotational speed. Energy conservation is satisfied.
For a TEC (TEG), we need to wonder about what happens to the heat flow (damned ... isn't "heat" the flow of internal energy? ... oh this is getting too hard). It's been about 20 years since I studied the physics of TECs so I may get some of this wrong, but basically there is a parasitic heat term that is considered in the analysis of TECs and there is the issue of the thermal conductivity across the the TEC from hot side to cold side. This (effective) conductivity is current dependent. When all is said and done, and you do a proper analysis on the energy/power, you will find energy is still conserved. If you let the current exist in the TEC, by closing the circuit, more heat flows from hot side to cold side, and parasitic heat will try to heat both sides. If you don't let current exist by opening the circuit, less heat flows from hot side to cold side. Note that either way heat is still flowing from hot side to cold side because that is a basic fact of thermodynamics. We are not putting energy in, so we can't expect to oppose the law of entropy on our closed system.
By the way, operation of the TEC generation requires adequate heat capacity and mass (thermal reservoirs) on both the hot side and the cold side, or the thermal gradient will be quickly reduced and little power will be generated. Hence, one is either using stored thermal energy as a temperature difference between hot and cold sides, or those reservoirs are being reenergized (heat flow from cold to hot side) by another energy source.