What contaminant materials should be avoided for reaching ultra-high vacuum?

This question relates to ultra-high vacuum chambers (order of ~$$10^{−10}$$ mbar) in experimental setups. [This question was asked at physics.stackexchange.com but removed due to being more about engineering...]

For this particular vacuum chamber, we have a set of ion pumps and a turbomolecular pump (pumping speed: 50L/s). Once every 12 hours we also run a TSP (Titanium Sublimation Pump). This set of different types of pumps yields a base pressure in the order of ~$$10^{−10}$$mbar, each pump getting rid of different residual pressures. Ion pump for the noble gasses, TSP for chemically binding the light elements and molecules and the turbo for the heavier elements and molecules.

Still, apparently, some materials should be avoided because they cannot be pumped out properly. In particular, my advisor has warned me against evaporating antimony Sb and its oxides SbO3.

As far as I understand, one should select materials for ultra-high vacuum (UHV) based on vapour pressures and outgassing rates. Antimony, because of its relatively high vapour pressure (comparable to lead, calcium and bismuth [1]), would sublimate to the walls and outgas at process temperatures, but we rarely heat the contaminated surfaces to temperatures higher than 200 degrees Celsius, for which the vapour pressures are negligible <$$10^{−10}$$ mbar.

According to my advisor, antimony will not pump out properly because of its low sticking coefficient to the stainless steel walls. Is that something to consider? And if so, how does this work? He suggested using a LN2 cold trap to trap/solidify the antimony. Why would our other pumps fail to pump out the gaseous antimony?

I can perhaps ask my question in a more broad sense: What are the considerations for materials in UHV application and when do they become contaminants leaving you with high residual pressures? (And what could you do to solve this...?) Given that only a small amount is used of this material (no bulk pieces) for evaporation.

[1] R.E. Honig, Vapor Pressure Data for the Solid and Liquid Elements, RCA review 30 (1969).

First of all let me say that IMHO this is a very specialized topic (and I've worked at a high power (20 GW) attosecond laser facility, and the lowest pressure I've come across in an experiment was at about $$10^-7$$ mbar).