That is a "zig-zag" piece of metal that latches one way or another, depending on the push. It is just like a push button ball point pen, push once and it toggles in one direction, push again, and it flips to the other direction.
It is hard to explain. However, see what looks like a small staircase (2 position notches) on the shaft, one leg has to go in there and the other leg goes elsewhere. Sorry, can't help guess where the other leg goes, but it might be onto the white slot in the main body. In fact, start with the white body in the main switch body first, put the zig-zag in there. It will likely go in some direction, maybe horizontal or vertical, hard to tell. Then try to put the button back on and check that the other zigzag leg goes into one of the slots/notches of the button shaft (as you have identified). I suspect the round spring will be compressed, leaving the button staircase clear for the zig-zag metal piece. With each press of the button, the leg of the metal piece will toggle between the two notches.
Good luck