Without adding anything, you can reverse the opamp connection.
Probably your photodiode is reverse-biased with its anode at ground and its cathode connected to a pull-up resistor to +5V.
Your opamp is probably an inverting circuit so that without light, the photodiode doesn't conduct and its output voltage is high. Therefore the opamp inverter gives a low to the LM3914. Light makes the photodiode leak current, so its output voltage falls, and the inverting opamp gives an increasing voltage to the LM3914.
With the opamp wired as a non-inverting circuit, its output voltage follows the photodiode's voltage.
Depending on the light intensity on the photodiode, it may have enough current to drive the LM3914 directly, without an opamp follower.