Hi,
Yes. The integrator is integrating the small input offset voltage of the op amp as well as the intended signal.
Sometimes fixing this is as easy as putting a large resistor in parallel with the integrating cap to reduce the DC gain. Try 1M.
I just noticed that you also use a difference amp in front of the integrator. That most likely has an output offset which also gets integrated. If the 1M resistor doesnt help enough you may have to provide an offset adjustment as well probably in the difference amp stage.
In theory if you had zero offset you would see no drift over time. For zero input you would have absolutely no output change. In real life you have to work a little harder to get this to happen. BTW you could probably test it with zero input to see if there is any unwanted ramping.