The USS Nimitz is nuclear powered, whereas the Allure of the Seas is not. Nuclear reactors are not cheap.
Also, the USS Nimitz, like all aircraft carriers has a catapult to help launch aircraft off the decks. The Allure of the Seas does not have a catapult. Catapults are expensive.
The USS Nimitz has kevlar armor over vital spaces, but civil cruise ships don't
Add to this the radar systems and armaments that aircraft carriers must have but are not required for civil cruise ships.
Aircraft carriers are build to handle severe weather where some cruise ships may not be. They must also have the propulsion systems and structural integrity to travel long distances at high speed. Civil cruise ship don't.
Most navy ships require very high redundancy of all critical system (Propulsion computer, navigation etc), the material specs are also usually very tight, meaning more costs, for example the paint used on some of the new carriers alone was a multi million dollar project to withstand the F35 Jets.