I'll try to explain using examples.
Basic setup
Discrete time system in state space.
$$
\begin{align}
x_{n+1} &= A x_n + B u_n\\
x_1 &= A x_0 + B u_0\\
x_2 &= A^2 x_0 + AB u_0 + Bu_1\\
x_3 &= A^3 x_0 + A^2B u_0 + AB u_1 + Bu_2\\
x_n &= A^n x_0 + \sum_k^{n-1}{A^{n-k-1}B u_k}
\end{align}
$$[1]
Example 1
$$
x_{n+1} =
\begin{bmatrix}
0.9 & 0\\
0 & 0.8
\end{bmatrix}
x_n +
\begin{bmatrix}
1\\
0
\end{bmatrix}
u_n
$$
For simplicity, assume zero initial condition. Then
$$
\begin{align}
x_1 &=
\begin{bmatrix}
u_1\\
0
\end{bmatrix}\\
x_2 &=
\begin{bmatrix}
0.9 & 0\\
0 & 0.8
\end{bmatrix}
\begin{bmatrix}
u_1\\
0
\end{bmatrix}
+
\begin{bmatrix}
u_2\\
0
\end{bmatrix}
&&=
\begin{bmatrix}
0.9 u_1 + u_2\\
0
\end{bmatrix}\\
x_n
&=
\begin{bmatrix}
\sum{0.9^{n-k-1}u_k}\\
0
\end{bmatrix}
&&=
\left(\sum{0.9^{n-k-1}u_k}\right)
\begin{bmatrix}
1\\
0
\end{bmatrix}
+ 0
\begin{bmatrix}
0\\
1
\end{bmatrix}
\end{align}
$$
It can be seen that, whatever we chose as the input sequence $u_n$, we can control the state only in the $[1,\quad 0]^T$ direction; i.e. we can influence only the first state variable and not the second one. The direction $[0,\quad 1]^T$ is not controllable.
Also note that, to reach this conclusion, we utilized the values of the column vectors $B,\ AB,\ A^2B, \dots$. This is why the controllability matrix is made from those column matrices.
Example 2
$$
x_{n+1} =
\begin{bmatrix}
0.9 & 0\\
0 & 0.9
\end{bmatrix}
x_n +
\begin{bmatrix}
1\\
1
\end{bmatrix}
u_n
$$
Doing the same procedure as above,
$$
\begin{align}
x_1 &=
\begin{bmatrix}
u_1\\
u_1
\end{bmatrix}\\
x_2 &=
\begin{bmatrix}
0.9 & 0\\
0 & 0.9
\end{bmatrix}
\begin{bmatrix}
u_1\\
u_1
\end{bmatrix}
+
\begin{bmatrix}
u_2\\
u_2
\end{bmatrix}
&&=
\left(0.9 u_1 + u_2\right)
\begin{bmatrix}
1\\
1
\end{bmatrix}\\
x_n
&=
\begin{bmatrix}
\sum{0.9^{n-k-1}u_k}\\
\sum{0.9^{n-k-1}u_k}
\end{bmatrix}
&&=
\left(\sum{0.9^{n-k-1}u_k}\right)
\begin{bmatrix}
1\\
1
\end{bmatrix}
+ 0
\begin{bmatrix}
1\\
-1
\end{bmatrix}
\end{align}
$$
It can be seen that, whatever we chose as the input sequence $u_n$, we can control the state only in the $[1,\quad 1]^T$ direction; i.e. we can influence the first state variable and second state variable by the same amount. We cannot increase first state variable while decreasing the second state variable; i.e. The direction $[1,\quad -1]^T$ is not controllable.
Example 3
$$
x_{n+1} =
\begin{bmatrix}
0.9 & 0\\
1 & 0.8
\end{bmatrix}
x_n +
\begin{bmatrix}
1\\
0
\end{bmatrix}
u_n
$$
Doing the same procedure as above,
$$
\begin{align}
x_1 &=
\begin{bmatrix}
u_1\\
0
\end{bmatrix}\\
x_2 &=
\begin{bmatrix}
0.9 & 0\\
1 & 0.8
\end{bmatrix}
\begin{bmatrix}
u_1\\
0
\end{bmatrix}
+
\begin{bmatrix}
u_2\\
0
\end{bmatrix}
&&=
u_1
\begin{bmatrix}
0.9\\
1
\end{bmatrix}
+ u_2
\begin{bmatrix}
1\\
0
\end{bmatrix}\\
&=
u_1
\begin{bmatrix}
0\\
1
\end{bmatrix}
+ \left(u_2 + 0.9 u_1\right)
\begin{bmatrix}
1\\
0
\end{bmatrix}\\
\end{align}
$$
Here we can see that by suitably choosing $u_1$ and $u_2$, we can change the state of the system in the direction $[1,\quad 0]^T$ as well as $[0,\quad 1]^T$ and in any direction which is a linear combination of these. Since this is a two dimensional state space, and we have two independent directions, their linear combination can actually generate any and all directions in the state space. So this system is fully controllable.
One more thing to note here is that, by looking at the original system equation, we see that the input directly influences only the first state variable due to $B$ matrix being of the form $[1,\quad 0]^T$. But by looking at the second time step, we realise that input can influence the second state variable also (but indirectly). This is why this system is fully controllable; the input signal can directly and indirectly influence all the state variables.
How to identify the directions ?
Left as an exercise to the reader. Look at the eigen vectors of the $A$ matrices in the examples above.
What if the system being analysed is different from the examples?
The examples above were chosen to be easy to explain. But, we can transform all systems to the format of one of the above examples (diagonal or nearly diagonal) by using eigen decomposition of the $A$ matrix. This transformation is also called change of variable or change of coordinates.
Transfer functions, cancellation of modes and controllability
Continuing from the example 1, and assuming that its $C$ matrix is $[1,\quad 1]$,
$$
\begin{align}
C(zI-A)^{-1}B &=
\begin{pmatrix}
1 & 1
\end{pmatrix}
\frac{1}{(z-0.8)(z-0.9)}
\begin{pmatrix}
z-0.8 & 0\\
0 & z-0.9
\end{pmatrix}
\begin{pmatrix}
1\\
0
\end{pmatrix}\\
&=
\frac{(z-0.8)}{(z-0.8)(z-0.9)}\\
&=
\frac{1}{(z-0.9)}\\
\end{align}
$$
We notice that the modes correspond to the poles of the system and that one of the poles (the one at $z=0.8$) cancelled out. This is the mode that turns out to be uncontrollable. But, just by looking at the transfer function, we can't tell the direction in which the mode $z=0.9$ acts along, and the direction in which mode $z=0.8$ acts along.
