That has got to be the result of the MOSFET and the wiring resistance. We need more details to know what losses you are getting, like motor current, MOSFET type and constructional details.
If you are using PWM, you should have a freewheel diode across the motor, and there shouldn't be a large capacitor across the motor. C5 will slow down the switching of the MOSFET a lot, so that will make the MOSFET stay in the linear region a lot of the time and get hot.
You've also got the wrong symbol for the MOSFET. The symbol shows a P-channel MOSFET, but the part number and the circuit layout are correct for an N-channel MOSFET