If you are interested in knowing why your tea is cooling in different times, you already answered the question yourself. It is due to the different thermal conductivities of the different items you use it to store.

If you are interested in calculating the values for different items the following approach should do:

 1. Gather a list of all necessary heat transfer coefficients $\alpha_{ij}$ and thermal conductivity coefficients $\lambda_i$ In my opinion there is no need to calculate / measure these yourself, that's why books with tables of different coefficients exist. 
 2. Create a simple but accurate enough model of your container. A cup can be regarded as a hollow cylinder for example. Make a sketch of it!
 3. Calculate the thermal transmittance $U$. Notice that different regions of your cup will have different heat transfer and thermal conductivity coefficients. For example the part of a coffee cup that is covered with a plastic lid and the lower part of the cup with no contact with the lid. Add them with the formulas for parallel and series.
 4. Calculate the overall heat flux. A higher heat flux means a quicker cooling time.

$$U=\frac{1}{A R_{overall}}$$

$$R_{overall} = \sum_{i=1}^n R_i \tag{in series}$$

$$\frac{1}{R_{overall}} = \sum_{i=1}^n \frac{1}{R_i} \tag{in parallel}$$

$R_i$ are the resistances for heat transfer and conductivity.

$R_{\lambda} = \frac{l}{\lambda A}$

$R_{\alpha} = \frac{1}{\alpha A}$

$l \equiv \text{length}$

$A \equiv \text{cross-sectional area}$

This might look confusing but maybe it helps you approaching it backwards.

$\dot{Q} = A U \Delta T$

is the heat flux you want to know. So you need to know $U$. You divide your container into compartments that have a constant $R_{overall}$. Find these compartments and calculate $R_{overall}$ for each of them.

[![enter image description here][a]][1]


  [1]: https://i.sstatic.net/RwrGI.png
  [a]: https://i.sstatic.net/RwrGIm.png