Different logic families have different internal structures.
TLL gates are designed so the inputs default to high and need pulling low so the risk that the input will pick-up noise is very low unless the leads are very long.
CMOS gates are designed to have a very high input impedance and there is no default setting for the inputs, so if they're left unconnected, the input can pick up noise and be unpredictable.
You need to know what logic family the IC uses, there is no hard and fast rule.
Modern ICs tend to be CMOS, TTL is obsolete.