If you are interested in knowing why your tea is cooling in different times, you already answered the question yourself. It is due to the different thermal conductivities of the different items you use it to store.
$\dot{Q} = A U \Delta T \tag{1}$
is the heat flux you want to know. This gives you a relationship between the heat that is transfered to the enviroment and the dependance on the type of your container. So you need to know the thermal transmittance $U$.
If you are interested in calculating specific values for different items the following approach should do:
- Gather a list of all necessary heat transfer coefficients $\alpha_{ij}$ and thermal conductivity coefficients $\lambda_i$ In my opinion there is no need to calculate / measure these yourself, that's why books with tables of different coefficients exist.
- Create a simple but accurate enough model of your container. A cup can be regarded as a hollow cylinder for example. Make a sketch of it!
- Calculate the thermal transmittance $U$. Notice that different regions of your cup will have different heat transfer and thermal conductivity coefficients. For example the part of a coffee cup that is covered with a plastic lid and the lower part of the cup with no contact with the lid. Add them with the formulas for parallel and series (2). The attached image gives an example.
- Calculate the overall heat flux. A higher heat flux means a quicker cooling time.
$$U=\frac{1}{A R_{overall}}$$$$U=\frac{1}{R_{overall}}$$
$$\begin{array}{c} \\ R_{overall} = \sum_{i=1}^n R_i &\text{in series} \tag{2}\\ \dfrac{1}{R_{overall}} = \sum_{i=1}^n \dfrac{1}{R_i} &\text{in parallel} \end{array}$$
$R_i$ are the resistances for heat transfer and conductivity.
$R_{\lambda} = \frac{l}{\lambda A}$
$R_{\alpha} = \frac{1}{\alpha A}$
$l \equiv \text{length}$
$A \equiv \text{cross-sectional area}$