When I grew up in the 1950s, I learned to ride a bike on an old 26-incher, no fenders, no chain guard, no accessories of any kind, already been through three or four owners -- just a frame, handlebars, two balloon tires, pedals, chain and a coaster brake. That bike would go anywhere, including down hard with me on it. It ran through rocks, dirt, mud -- whatever was on the ground on the farm. It sat out in the weather. You couldn't make that thing not work aside from flat tires.
Later in life, there was the expensive 10-speed bike. There wasn't anything on that bike that didn't easily fail or required constant adjustment and attention. If it sat out in the rain, the brake and shifter cables got cranky if they weren't already. You never knew what gear you were in. The brakes didn't work as well as the old coaster brakes. Hit a bump and the tires went flat. And those skinny little tires let you feel every bump in the road as compared to those big, fat balloon tires. But it was fast and the hills were a lot easier to pedal.
The analogy is this: aluminum electrolytic capacitors are like that old balloon-tire bike .... reliable, gets you where you need to go, cheap. The caps are great for regular power supply filters, decoupling, bypass, etc. The tantalum caps are specialty caps like the 10-speed is a specialty bike. Absolutely wonderful for very certain things, such as timing caps or super-low ESR bypass caps in decoupling circuits. But they just aren't as reliable, especially when the circuit get a little rough with them. Not only do they tend to short, they often pop or explode. They don't like the least little bit of over-voltage or reverse voltage while the aluminum caps tolerate all kinds of abuse.
Dean