You have the following information from the datasheet and your operating environment:
-your maximum allowable temperature of the MOSFET junction (or the maximum temperature you are willing to tolerate)
-the maximum ambient temperature (usually 25C at least, but maybe 30C or even 40C especially if you're outside or it's a hot day)
-the thermal resistance of the heatsink to ambient
-thermal resistance of thermal grease
-thermal resistance from junction to heatsink (it might be called junction to case, or junction to board, or other names depending on the package and how the heatsink is supposed to mount to it).
The Thermal Resistance is the C/W, it will rise X degrees Celcius for every Y Watts of power dissipated. The MOSFET's thermal resistance for junction to ambient is what you use if there is no heatsink. Even if there is a heatsink, it plays a role. If you choose to ignore it, it just means your MOSFET will run cooler than predicted which is always good. It just means there are two paths to cool the MOSFET: through the heatsink to air, and directly from MOSFET package to air.
So
Code:
[Maximum Junction Temp] >= [Max Temp Rise]+[Maximum Ambient Temp] = [Power Dissipated] * ( [Junction to case/heatsink C/W] + [Thermal grease C/W] + [ Heatsink C/W] )
You just pick a heatsink with a C/W that gets you a low enough temperature rise so that it does not exceed your chosen junction temeprature at the ambient temperature. The thermal grease is usually very low impedance if applied properly so it does play a role but mainly you choose the heatsink only.
Heatsinks have different thermal resistance with different airflows and it is sometimes listed for different airflows. So if you know your airflow (like a fan) or if you know that you don't have forced airflow you use the thermal resistance that is your operating condition.
The hardest part of all this is actually calculating your MOSFET power dissipation from conduction losses (which is pretty easy) and switchign losses (which can get very complicated to get an accurate number).
EDIT: I looked at teh MOSFET datasheet and what you are asking is not possible. Because maximum allowable temperature is 175C and with an ambient of 25C, your maximum rise is 150C. So EVEN if your MOSFET and thermal grease had 0C/W you would still need a heatsink with a thermal resistance to ambient air of less than 2.14C/W. But the junction to case thermal resistance of the MOSFET is already 2.14C/W. It's already too high. You would need a heatsink with 0C/W to ambient. Without a heatsink, the package is the heatsink which adds on an additional 62.5C/W (junction to ambient thermal resistance). You need to parallel MOSFETs or find another MOSFET.