Personally, I'm partial to the Arduino, which is based around Atmel's ATMega line of micro-controllers (mainly the ATMega8/168/328 - there are also variants which use the ATMega644, 1280, and 2560). Think of the Arduino as a "carrier board" for the micro-controller. The Arduino project itself is completely open source, and cross platform - the IDE and tools used (which are also used for other AVR devices) are completely open source, and run on Windows, Mac, and Linux platforms. The Arduino IDE is Java based (and derivative of the Processing platform). There are even some AVR cores in the Open Hardware arena, so in theory the entire thing could (in time) become completely open source.
A standard Arduino board (that is, one which conforms to the reference design; based around an ATMega8/168/328) varies in price depending on the vendor, but shouldn't cost more than $30.00; the software environment is free to download. Connection to the PC employs a USB cable; special software already on-board the ATMega (aka the "bootloader") enables a quick and easy method of getting your compiled code onto the micro-controller; while this bootloader does take up some memory on the micro-controller, it is a very small amount, and the convenience is worth it. If, however, you need the extra space, it is possible to forgo usage of the bootloader, and use a standard AVR programmer ICSP interface with the Arduino IDE to upload as well (that's how you get the bootloader on the micro-controller in the first place - in fact, it is possible with some hacking to turn the Arduino board itself into an ICSP programmer, and program the micro-controller in-situ).
Bootloaders also exist that have been optimized for space (the smallest possible is 512 bytes, which is a physical limit of the hardware; the standard bootloader is 2k - so on a 168 you have 14k free for program space, and on a 328, 30k is free).
The ATMega line of micro-controllers are more than capable of handling just about any project you throw at them; they aren't the most inexpensive micro-controllers available for embedded design, but that isn't something you need to worry about until you get into needing thousands of pieces. They also tend to use more power than other micro-controllers, and there have been issues in the recent past in securing quantities of certain models of the micro-controller line (by this I mean it was difficult to order them from places like Mouser and DigiKey).
Maybe someone else here will chime in on the pros and cons of PICs, as well as other micro-controllers; I am not very familiar with them...