One problem with this circuit lies with your use of an AC coupling cap to feed RF from the 50 ohm resistor to the diode. This will allow the DC level on the anode of the diode to change when RF is fed to the diode which will then change the duration of RF cycle that the diode is conducting. The resulting reading will be wrong in that case. It might be easier to simply not use the coupling cap and attach the diode directly to the 50 ohm resistor.
For HF detection, it might be best to use a schottky diode, but then you have to be careful not to exceed the peak reverse voltage of the diode. That depends on how much RF power you are planning to measure.
The DC voltmeter probably has a very high input resistance. In that case, the time constant of the RC filter after the diode may be too long and it might take a long time for a voltage reading to die off. Consider grounding the far end of the potentiometer and use a pot of some value around 5K or 10K. This puts the pot value as the main load for the filter and so the time constant will be quite reasonable.
Most RF wattmeters put a resistor attenuator between the 50 ohm load and the detector diode to help protect the diode from overvoltage, and also to allow several ranges of power to be switched in and out, by switching various attenuators in and out.