It depends a bit on the quality you expect from some of the peripherals and what their qualities are at given clock speeds.
Will you want a very high speed SPI interface and can this specific chip support that on the Fast Auxiliary Clock? Or are you going to do 1MHz SPI at most? So also for all the other devices.
It's of course a nice boasting thing to be able to say that everything runs on many-MHz, but if it's not needed it will create more power consumption, more heat and more programming effort for no result. It can be tempting to just do everything at maximum speed and highest accuracy, but in almost all cases a good systems design will start with indexing what are the minimum speed and accuracy for something to work and then taking the appropriate margins. (Caveat: If you may need a higher speed later, it may be advantageous to make the hardware such that scaling up is only a firmware issue).
As for activating the Crystal; I don't actually know this particular chip, but it seems compatible in build-up to the ones that I know that have a configurable clock source, that is selected at power-up, in which case setting the crystal as a source is a simple configuration and possibly the configuration of all the other clocks is too. If that isn't the case and you need to actively switch while already running the core, you need to make 100% sure the crystal is running before you switch over. The datasheet will make very clear mention of the procedures needed for that if this is the case. Usually it's something along the line of: Activate Clock --> Wait for a bit in a register to be set --> Switch over.
With respect to the ADC, that can become a very complicated case. If you need very low sampling jitter or highly accurate timing (for highly accurate control or very (very) high grade Audio) you will quickly get to the domain of using the Crystal as a clock source. Where you'll then also need to make some hardware considerations. Both in the Analogue domain and on the Crystal end, since crystal accuracy is influenced by the loading capacitance (which includes traces and pins).
But those applications are few and far between.
In most cases your timers and your UART will determine the clocks you need, since they are most sensitive to clock drift and/or offset. If you need a timer just for low-speed timing you are better off relying on a watch crystal, since both the crystal and the internal driving circuitry for such low frequency crystals are usually better suited to high accuracy and low drift over long time.
