- Relatively low lifting requirements (100 kg)
- Relatively fast lifting speed (80 fpm -> .4 m/s)
The first part of your power calculation was wrong. There's no need to go into pulley diameters at this early stage of the calculation. Let's start at the load hook.
100 kg ~ 1 kN. Speed > 0.4 m/s. So power > 400 Watts.
That's mechanical power, after a gearbox, after motor losses. So the motor will need to be at least 1 kW electrical input power. That's based on pure lifting speed. We can add a little more for load acceleration, or ignore it with little error.
Motors tend to have a 'nice' speed for decent power output. Too slow and you have to work at delivering a lot of torque, which increases the weight of the motor. Too fast and they scream and need careful balancing. The 2000 to 3000 rpm range is reasonable for this sort of motor power. 2500 rpm is about 250 rad/s, so a speed of 400 mm/s would need a shaft of radius 400/250 = about 1.6 mm radius to wind directly.
3 mm diameter is clearly impractical for a 1 kN load, so you need a reduction gearing to a drum of practical size. The range 50 mm to 100 mm radius feels about right to me, so you'd need a speed of 400 mm / 80 mm = 5 radians per second.
With a motor doing 250 rad/s and the drum requiring 5 rad/s, you need a reduction gearing of around 50:1.
Obviously different motors and different drum sizes would result in different gear ratios, speeds etc, but at least we have a good ballpark to start from, which is ...
1 kW (electrical) 2500 rpm motor
160 mm diameter winding drum
50:1 is quite a ratio to do in a single stage, even epicyclic. Worm drives are inefficient, so probably best avoided. An important side effect of their inefficiency is that they don't back drive, so could do away with the need for a brake, so maybe worth thinking about. Compactness might sit nicely with low weight, so suggest an inverted epicyclic stage within the winding drum, driven by a toothed rubber belt reduction stage from the motor. That's if you have the option to build it custom of course.