Regarding the word dirt. It depends on what a person means by dirt. Generally, it is understood that dirt means loose material such as soil or sand on or near the surface of the earth.
In mining however, and this may apply to tunneling, the word dirt can sometimes be used used to describe the broken rock, or mined material, that is excavated from the mine or tunnel in the process of mining or tunneling. In this situation it's a form of jargon or colloquialism used by some in the English speaking mining community and possibly the tunneling community.
One reason why locally excavated "dirt" may not be suitable as a ground reinforcing or ground supporting material is that it's strength properties are inadequate for it to be used. Another reason is that it could be a material that expands and contracts when subject to changes in moisture content. This could cause the supporting medium to buckle and decouple from the ground mass. Another reason is the material being removed might be soft.
Materials that are unsuitable include: clay; chalk; talc; limestone; ultramafic rocks such as komatiites ; most sedimentary rocks such a sandstone and mud stone; platey material such as mica; and oxidized or weathered rocks.
Materials that are better suited to be used for ground support purposes are hard, competent, non-friable materials with high strength characteristics, such as basalt and dolerite. Which is why these types or rock are used as aggregate materials for concrete. Generally, these materials need to be brought in sites.
In countries like Bangladesh, situated in a massive river delta, where there are no hard rock sources, broken baked clay is sometimes used as aggregate for road construction purposes.