I ran into a problem with this issue many years ago.
The male pin in the 50 ohm BNC coax connector is slightly larger in diameter than that of the 75 ohm BNC (this is how they get the different impedances). The female receptacles in the two connectors are correspondingly sized. The difference is subtle to the eye, but if you plug a 50 ohm coax into a 75 ohm receptacle, it will spring (distort) the receptacle. If you then switch back to 75 ohm coax, the connection may be unreliable.
I don't know if 75 ohm coax plugged into a 50 ohm connector is reliable or not.
EDIT: I just now read the Amphenol page that Eric referenced. They have obviously solved this problem, but the fact that they mentioned that the two connectors can be mated nondestructively suggests that it was a problem in the past. As I said, I encountered the problem many years ago - like 30. Surplus connectors could have this problem.