AC voltage passes through zero twice each cycle, so the relay is turning on and off continuously, even if it holds in.
The current will also be lower due to the inductance of the coil.
Relays or solenoids designed for AC are made slightly differently than for DC, different winding for the same voltage and with a "shading coil", which has current induced from the main coil and maintains the magnetic field through the AC voltage zeros. (Or some may have a rectifier and work on DC).
The motor is trying to reverse every half cycle of AC as the voltage reverses, so it will not move far and will be taking high current as it's effectively stalled each time it has to restart.
Brushed AC motors are built as "Universal motor" style, with wound field coils rather than permanent magnet field.
The field reverses with the AC reversals as well, so the motor turns the same way regardless of the instantaneous AC power polarity.