When I was doing things like this there was no "smart camera" and libraries of code. So writing from zero, it would be easy to see lights blink.
Your camera "keeps track of 1-4 objects". I think you are using the software that does that. When you blink a LED the software will loose track of that LED. So there is much of the problem. I think you have software written for a different project and you can't get around that.
I was doing something like this but I had Red, Blue, Green, etc light so that is easy.
I had control of the light source. I could turn on/off change color of any point. If you can not directly drive the light, then you must have the LEDs blink in some pattern and then synchronize the software to the pattern. This will slow down the process.