Try measuring the DC resistance of your motor with a multi-meter, before it starts turning the resistance is likely to be very low - it may well take a 4A surge when you apply power!. If the motor reads 1 ohm, and the motor supply is 12V, ohms law tells you the current will be 12A - you might also try measuring the stall current?, connect a ammeter in series with the motor (set to a high amps range, 10A or 20A), then stall the motor with your fingers - read the current off, this will be similar to the start current pulse.
You really need to apply a fairly long start delay, before you start monitoring the current!.
How big is the motor?, normal little DC motors (as used in small models) usually have a higher stall/start current than dry batteries can provide! - with NICAD or lead acid, it's a different story!.