The speed of a dc motor varies if the torque varies even for a fixed supply voltage.
A dc motor will generate a back voltage as it runs, and that voltage will oppose the supply voltage. The difference between the two voltage, divided by the resistance of the windings, gives the current taken by the motror.
The torque is approximately proportional to the current.
If the load on the motor needs more torque, the motor needs to take more current, and that needs either a larger supply voltage or a lower voltage generated by the motor. If the supply voltage doesn't change, the only way to get larger current is for the motor to slow down and produce less back voltage.
So to keep a constant speed with varying loads, as speed controller is needed.
However, the speed may be consistent enough with a fixed voltage that setting the voltage is a good enough way of controlling the speed.
Also many devices called "speed controllers" for dc motors are just voltage controllers, and they don't actually have any feedback from the actual motor speed. The devices are often switch-mode controllers, that rely on the inductance of the motor.