You shall first identify the location of the NA, then calculate the Q by either considering the shaded area above "line-a", or the unshaded area below "line-a". If you are doing correctly, Q for both areas should be identical, so the stress over the neck area (b = 50 mm).
Discussion:
In order to resolve your confusion on this matter, we need to explore the significance of indicating the direction of shear/shear stress.
Shear force/stress occurs on a plane within a rigid body, on that plane, the segment above moves in one direction and below moves in another direction. This phenomenon can be observed in the graph below, which shows a deformed/deflected beam in the longitudinal direction.
Conclusion:
So, does indicating the direction of shear stress have significance?
Yes, when there is the need to show the direction of the stress graphically as in the sketch above, and in 3D cases.
No, there is no significance for the general purpose of calculating shear stress for checking stress level or for design. For which $Q = \sum A|d|$, and $\tau = \dfrac {V|Q|}{Ib}$ are routinely used without the directional indicators (+/-signs) attached to the $V, Q, d$ terms.
Hope this helps.