Below is the LTspice simulation of a circuit that, I think, does what you want:
It uses one quad rail-rail op amp package, along with a bunch of resistors to generate the desired function.
Note that the output (blue trace) goes from 5V at the 0% pot position (0.0 on the horizontal axis) to 0V at the 50% position, then back to 5V at the 100% pot position.
The yellow trace is the pot output voltage versus pot wiper position, the green trace is the output of op amp U2, and the red trace is the output of U1.
The output of U1 and U2 are summed together by U4 to give the desired output.
The output is ratiometric so will track any variation in the supply voltage with zero out always occurring at the 50% point of the pot.
All resistors are ±1% type.
View attachment 134613