Firstly True North is a well defined point which can be determined fairly easily by the pole star and indirectly (but still fairly easily) by a compass. Even if there was no pole star or compass this is still a fairly logical way to define direction on a rotating globe.
This also relates to the most sensible way to draw a grid on a globe. Having parallel lines of latitude and converging lines of longitude is as sensible as anything else, allowing for a simple global coordinate system which can easily be translated to a flat local map. If you define true north as the axis of rotation then from that simple definition you can work out the precise point as accurately as you want.
To put it another way if you draw an arrow on your map which points north it is then very easy for anyone else to determine exactly what direction that is supposed to be.
Also once you have established the direction of true north on a particular site you don't need to make every measurement from it directly. You can set up a physical marker or calibrate a magnetic compass.
Also you absolutely can use the sun as a reference point as long as you know the accurate local time relative to a reference time (noon at Greenwich being the traditional reference) in fact with the right instruments this can tell you absolute position as well as the direction of north.
As to why you use north rather than south, west or east, it doesn't makes any fundamental difference as they are all still defined relative to each other and historically north is the most convenient direction to find by the pole star, a magnetic compass or by observing the track of shadows. Plus if you have a universal convention that the top of a map is north it makes it easier for everyone to use and prevents confusion.
Finally I'm not sure what else you would use, you can pick any arbitrary direction you want but you still have to define it in terms of something and the same limits of accuracy of direction finding equipment still apply.