Google does not know about a "mall rectangular", 9V, E style battery. I think the problems are caused by the battery or LM317 being overloaded by the high starting current of the pump.
You can add some "power up reset" parts to fix the timer triggering on power up problem and maybe a capacitor larger than the 1uF at the output of the LM317 to fix the timer triggering when the switch is used problem.
You can add some "power up reset" parts to fix the timer triggering on power up problem and maybe a capacitor larger than the 1uF at the output of the LM317 to fix the timer triggering when the switch is used problem.