As far as I know, capacitors have two metal plates and a dielectric between the plates inside them. So if they are just metal plates that are storing the electrical charge, why do they explode when I connect them backwards? If the plates are polarized how is this achieved?
Non-polar capacitors are not polarity sensitive and don't explode when connected either way.
Electrolytic capacitors are polarised in their construction. Wikipedia's Electrolytic capacitor article has much to say on the subject that doesn't need to be repeated here.