The torsion constant $J_T$ relates the angle of twist to applied torque via the equation:
$$ \phi = \frac{TL}{J_T G} $$
where $T$ is the applied torque, $L$ is the length of the member, $G$ is modulus of elasticity in shear, and $J_T$ is the torsional constant.
The polar moment of inertia on the other hand, is a measure of the resistance of a cross section to torsion with invariant cross section and no significant warping.
The case of a circular rod under torsion is special because of circular symmetry, which means that it does not warp and it's cross section does not change under torsion. Therefore $J_T = I_P$.
When a member does not have circular symmetry then we can expect that it will warp under torsion and therefore $J_T \neq I_P$.
Which leaves the problem of how to calculate $J_T$. Unfortunately this is not straightforward, which is why the values (usually approximate) for common shapes are tabulated.
One way of calculating the torsional constant is by using the Prandtl Stress Function (another is by using warping functions).
Without going into too much detail one must choose a Prandtl stress function $\Phi$ which represents the stress distribution within the member and satisfies the boundary conditions (not easy in general!). It also must satisfy Poisson's equation of compatability:
$$ \nabla^2 \Phi = -2 G \theta $$
Where $\theta$ is the angle of twist per unit length.
If we have chosen the stress function so that $\Phi = 0$ on the boundary (traction free boundary condition) we can find the torsional constant by:
$$J_T = 2\int_A \frac{\Phi}{G\theta} dA$$
Example: Rod of circular cross section
Because of the symmetry of a circular cross section we can take:
$$\Phi = \frac{G\theta}{2} (R^2-r^2) $$
where R is the outer radius. We then get:
$$J_T = 2\pi\int_0^R (R^2-r^2)rdr = \frac{\pi R^4}{2} = (I_P)_{circle}$$
Example: Rod of elliptical cross section
$$\Phi = G\theta\frac{a^2 b^2}{a^2+b^2}\left(\frac{x^2}{a^2}+\frac{y^2}{b^2}-1\right)$$
and
$$J_T = \int_A \frac{a^2 b^2}{a^2+b^2}\left(\frac{x^2}{a^2}+\frac{y^2}{b^2}-1\right)dA = \frac{\pi a^3 b^3}{a^2+b^2} $$
which is certainly not equal to the polar moment of inertia of an ellipse:
$$ (I_P)_{ellipse} = \frac{1}{4}\pi a b(a^2+b^2) \neq (J_T)_{ellipse}$$
Since in general $J_T < I_P$, if you used the polar moment of inertia instead of the torsional constant you would calculate smaller angles of twist.