I am thinking of building a house with basement next to some existing structures. Are there rules of thumb on how closely/deeply you can excavate next to an existing structure? By moving farther away from the existing structure do I reduce the potential need for temporary retaining?
Usually your building department is the first source. They will recommend what needs to be done. If your neighbor is a conventional building with few stories of wood or light framing, and the soil is sound and firm without high level of subterranean water table, by staying outside of a 45 degree projection plane from the bottom of his foundation you should be ok.
Even then it is recommended to excavate near the neighbor in sections of 4 to 6 feet wide and build either a temporary shoring or part of permeant retaining wall and then dig and build next section.
Many Soils engineers for a small fee do a pre review of the site and give you a professional advice as to whether you need a soils report or no.
As to the basement walls you need to control and drain any moisture caused by rains or underground water from standing behind walls, through insulation and drainage system via a possible sump pump.
Finally your retaining walls should be designed to be strong enough for combination of worst conditions, both gravity and soils pressure.
There are various factors that need to be considered. Amongst them are soil strength, depth of the excavation, distance, weight (i.e. pressure exerted) of the neighbouring structure as well as the depth of its foundation.
Without any numbers it‘s not really possible to make an estimation about this.
In general, the further away the new excavation site, the lower the risk of settling of the neighbouring structure.
Without a competent geotechnical analysis by a qualified civil engineer, you are likely to face difficulties and lawsuits. Generally speaking, forces exerted by buildings act at 33° to the horizontal in soil. This does not account for hogging in soil or loss of compression against neighbouring structures of the soil. Fluid content aints for much of the rigidity or fluidity of soil strength. Opening it up to air will change its characteristics and behaviour messing something unexpected could happen.