The Wiki article on nuclear reactor physics states that
the average neutron lifetime in a typical core is on the order of a millisecond, the exponential factor is as small as 0.01, in one second the reactor power will vary by a factor of (1 + 0.01)^1000, or more than ten thousand.
These are called prompt neutrons. It also states that
so-called delayed neutrons (typically <1% of created neurons) increase the effective average lifetime of neutrons in the core, to nearly 0.1 seconds, so that a core with average neutron lifetime of 0.01 would increase in one second by only a factor of (1 + 0.01)^10, or about 1.1: a 10% increase. This is a controllable rate of change.
Most nuclear reactors are hence operated in a prompt subcritical, delayed critical condition: the prompt neutrons alone are not sufficient to sustain a chain reaction, but the delayed neutrons make up the small difference required to keep the reaction going. This has effects on how reactors are controlled: when a small amount of control rod is slid into or out of the reactor core, the power level changes at first very rapidly due to prompt subcritical multiplication and then more gradually, following the exponential growth or decay curve of the delayed critical reaction.
This still seems to imply that a very fast control loop is needed since not acting on a criticality increase within a few seconds still leads to an exponential increase, after about 100 seconds it would again be a factor of 10000. So how fast does the feedback have to be and how is it applied? Only through the control rods, moving a few millimeters?
How is it even possible to measure the state of the core so quickly?
How much does the reactor's power vary around the steady state, where one neutron causes exactly one other neutron to be absorbed? Does it look like a cycle? Does the output power vary by a magnitude or less?