Your problem is due to contact bounce.
Push buttons and other types of mechamical switches do not open and close cleanly. When you press the button, the contcts close and open several times before finally closing. And the same occurs when you release the button, it opens and closes several times before finally going open.
If this is driving a counter, then the counter counts every pulse.
So you need a bounce suppressor.
This issue has been raised in this forum several times, so I suggest you do a search for "bounce suppressor".