Think of how many times the card readers are used in a day, easily hundreds of times. If the person before you had jelly (or pocket lint, or mayo from his lunch, or what not) on his card and inserted it into the reader, the next few people after will have trouble with the card reader. The trouble will persist until the jelly is cleared from the reader. When you rub your card, you're cleaning off any contaminates your card may have picked up.
One thing to note, you should probably clean your card with a tissue or a soft clean cloth, not your fingers. The oils and dirt on your fingers can be abrasive and/or corrosive. Tissues, while still abrasive, are far less so.