I vote that you go with the AVR. The AVRISP ($30 or so) is a good programmer and you can get a very good free, open source, optimizing C compiler called AVRGCC. Get it by downloading and installing WinAVR.
Alternatively, you can get an STK500 for about $100 and it'll come with a micro and it has an AVRISP built into it. You can use it to program chips either on the board or in a target system. The STK500 is a dev kit which allows you to connect the ports to either LEDs or buttons on the board or to anything else that you feel like adding to it.
And like 3v0 said, avrfreaks.net will be a better resource for information if you decide to go with the AVR.