While charging the capacitor the electrons are forcefully jammed inside, working against the Coulomb force that is trying to push them out (opposite charges attract, same ones try to separate). As E=F*s you get the stored energy as from the force that is trying to push the electrons out, times the distance that they have been pushed against this force. This energy is saved in potential energy form since at any time, once the circuit is closed, the coulomb force can finally push out the excess electrons, thus increasing their kinetic energy again. It really does resemble a spring a lot, just that the "force trying to push things out" is now electromagnetic instead of the usual spring force.