Slowing a neutron down in a reactor in called moderation and the material that is responsible for moderation is the moderator.
Thermal Reactors
In a thermal reactor, moderation is essential as neutrons tend to be born with a lot of energy (fast) but thermal reactors relying on slow (also called thermal) neutrons to perform fission. To more efficiently slow down the neutron, as you note, we want to minimize the atomic mass of the moderator material--the mainstream thermal reactor concepts of the world are all based on moderators with $A \le 12$.
However, this is counterbalanced against other neuronic and materials factors.
Neutronic factors
In addition to scattering, neutron-nucleus interaction can also result in the nucleus capturing the neutron. To be an effective moderator, this has to be minimized; we need to look at the cross-sections of our would-be moderator to make sure they're not strong absorbers. Boron, for example, is fairly light but is a neutron vacuum cleaner.
$^{1}H$, with $A=1$, is of course the best choice from a scattering perspective at slowing down neutrons, but it also has a non-negligible absorption cross-section. $^{2}H$ (deuterium) with $A=2$ is twice as heavy but doesn't suffer from the absorption problem. Very pure carbon (e.g., graphite) with $A=12$ also has a fairly low absorption cross-section.
Materials properties
Even if we've picked some materials with good neutronic properties, we need practical materials to put in a reactor. The material must be stable at high temperatures, be able to be produced in high-purity forms in sufficient quantities, resist the effects of radiation, etc.
Hydrogen or deuterium gas are explosive in the presence of oxygen, aren't the best materials for heat transfer; a high heat capacity can be a nice property since we can then use the moderator as coolant. The answer is to use water or heavy water (water with deuterium instead of $^{1}H$) as the moderator, usually under pressure, and put up with the absorption cross-section of oxygen and the high-energy gamma ray that is produced in the process (thankfully with a short half-life).
Graphite is very stable to very high temperatures, but obviously can't serve as the coolant either.
Economics
Water is abundant and cheap, and can be purified relatively efficiently.
High-purity graphite is not cheap, but not as expensive as heavy water.
Heavy water is very expensive.
However, heavy water's neutronic properties mean that is can be used without having to enrich the uranium in the reactor, which can save money on fuel.
Light water can be used as coolant and moderator, but the uranium has to be enriched to make a practical power reactor.
Graphite is used for gas reactors, where compressed gas is used for heat transfer. Gas reactors under development have the potential to reach very high temperatures but still achieve "walk-away" safety, meaning that they are inherently very safe.
Light water reactors have been the dominant players, but heavy water reactors (e.g., the CANDU family) and gas reactors (e.g., the AGR) are in profitable service today.
Fast Reactors
Fast reactors do not require moderation, as they are based around using fast neutrons.
Summary
For thermal reactors, the moderation efficiency (basically, maximizing the "energy loss fraction") is one factor in choosing a moderator material that has to be considered alongside other factors: other neutronic factors, material properties, economics. The only three materials that have been chosen for production power reactors are light water, heavy water, and graphite.
For fast reactors, the ability of materials to efficiently slow-down neutrons is not a design concern.
