If I build a quad copter but angle each rotor slightly inwards, should it be self stabilizing like an aeroplane with V wings, or does a variation of the rocket pendulum fallacy apply here? I feel like it should work but I keep second guessing myself.
First of all, lets look at what the basic state vector and control vector looks like for a coaxial rotor arrangement.
Now trace this down to equation 24, which gives the input vector as $u^T = (U1\space U2\space U3\space U4)$ , where the $U$s are total thrust, total pitch moment, total roll moment, and total yaw moment of the four rotors as a function of the speed of each rotor. You need a workable formula for each of those. If the rotors are coaxial, you can arrange things so that you can change just one of the $U$s and leave the others alone. If the rotors are not coaxial, you have a giant mess on your hands. It is not insurmountable, but you now have pitch, roll and yaw defined on the body, but they are mathematically different for each rotor and different for CW or CCW rotation. So you now need new expressions for the $U$s that reflect the geometry. And the four $U$s will not be totally independent.
Then, when you have the equations of motion sorted, you can look at stability cases. That part is done the same way as for a flat rotor, but evaluating the math is much harder, and many of the simplifying strategies, such as separation of variables, won't work.
The math on aircraft dihedral isn't any simpler, but there is a lot less of it. noncoaxial rotors have been done before. But they were always in sync.
Flettner Fl 282 "Kolibri"