But the exercises and examples also treat Root locus as a method of designing compensation or gain to fulfill the system requirement like damping factor and setting time. Isn't the root locus method for stability only?
What determines stability? zeros or poles?
Poles on RHP always cause instability. That is what we would like to avoid. In many cases, as long as the system is stable, engineers do not care. They just like a system work either good or bad. But the system requirements is also important. But it is meaningless if the system is not stable. Therefore, stability is the first requirement. The system quality is a bonus.
Why the closed loop dominant poles of the root locus can show the response of the system?
Maybe what I am saying is not exactly what you mean by this question. However,
here, I emphasize on the term 'dominant poles'. When the absolute value of a pole $||p_i||$ is small, it means that the effect of the pole in this fraction is high compared with $s$
$$\frac{1}{(s-p_i)}$$
Therefore, $p_i$ is called dominant. But, if the absolute value of the pole is very big, it means that the fraction is not very sensitive to $s$ and hence $s$ can be neglected. Therefore, nondominated poles can be approximated by a gain.
Wont it neglect the effect of the closed loop zeros?
Zeros are important as well. Zeros on RHP (nonminimum phase) are not desirable but they do not cause instability directly. But, since they attract the poles by the increase of the controller gain, they attract the poles to RHP if a high gain is used. Nonminimum phase systems are bad but not necessarily instable.
can Nyquist plot also used to show the system response and to design the system?
I personally do not use them because root locus does the job for me. In practice, It has not have any application for me but I am not sure about the others.
But the exercises and examples also treat Root locus as a method of designing compensation or gain to fulfill the system requirement like damping factor and setting time. Isn't the root locus method for stability only?
See page 2 of this slide.
This paragraph is out of the scope of your question but worth mentioning. You mentioned RHP. For systems with fractional order such as
$$\frac{(s^{0.6}+0.4)(s^{0.6}+0.7)}{(s^{0.6}+1.3)(s^{0.6}+0.9)}$$
The stability criteria could be more relaxed. See Figure (6-a) at page 13 of this article.