I note that you did not agree with my response, Adam. I'm sorry it was unsatisfactory and will try to do better.
I suggested that you read a book on basic electronics because I do not know you and inferred from your post that your are very early in your education about electronic components. These texts are normally quite useful in such cases and will undoubtedly answer your questions. They would teach you the nature of a capacitor and how it is useful in electronic circuits. They would also teach in more detail how different types of capacitors, including ceramic, polystyrene, electrolytic, tantalum, mylar, mica and others are chosen for specific applications. But I agree that reading those is a lot of work and will take considerable time, and perhaps that is overkill for dealing with your more specific request for help. The problem I have is that your opening statement "about what capacitor to supply to a circuit. And why to supply it to a circuit." is impossibly broad to address in a forum such as this. So, let's focus on your specific example and follow-on question.
Capacitors are useful in situations where voltage varies quickly. In a completely static situation, such as when a resistor draws current from a battery, a capacitor is useless and not needed. Your example of an LED fed by a battery is a DC problem, or in other words, once the circuit is established and working, there is no change of voltage, so a capacitor is not necessary. I would note, however, that your circuit as described is incomplete. If 3.2 Volts is attached to a 2V LED, there will be an enormous and possibly destructive current flow. What you need in this case is a resistor, in series with the LED. The resistor's resistance value has to be chosen using ohm's law, in this case by calculating (3.2V - 2V)/0.02 amps = 60 ohms. This resistor will cause a voltage drop of precisely 1.2 volts when a current of 20mA flows through it. In this way, the current through the LED is limited to 20mA. Again, no capacitor is needed for this.
In a DC circuit, when a capacitor gets fully charged, and if nothing external is changed, then it remains fully charged and, as you say, floats. It is truly as if it was not even there, except if the voltage tries to change. If the voltage tries to change, the capacitor will act as a reservoir of charge, and will act like a shock absorber (or damper) on that voltage. So, much like in a car, a shock absorber (damper) does nothing until the car starts moving over unsteady ground and the capacitor does nothing until the voltage is unsteady.
The amount of time it takes to charge up depends on how much resistance is between it and its supply of current. The more resistance, the slower the charge can flow into the capacitor and so the longer it takes to charge. If you were to place a capacitor directly across the terminals of a battery, the resistance is only that of the wire and the internal resistance of the battery and these are so small that the capacitor would appear to charge instantaneously.
A capacitor is used in circuits where the voltage varies. Capacitors are commonly used to help smooth DC power supplies (as a shock absorber, or damper) and we call them "power supply filter" capacitors. Capacitors are also commonly used to block the flow of DC current, while passing the flow of AC current. We call this a "coupling capacitor". The application of capacitors is a vast field of study best left to very long explanations such as you find in books.