I would imagine that the idols would be placed on pedestals that could be rotated manually and then place them at the center of the room (or at least not along a wall). Seems a lot more practical. Even if you have to build the building such that there are multiple main entrances, all this is still easier than rotating the entire building or room.
This is still not practicable for large-ish Idols as rotating a large idol by hands only by a certain degree can be challenging.
I'm imaginging something like a round horizontal platform sitting on wheels or balls that run in a track. Like a lazy susan. Nothing fancy. The wider the hold, the more leverage you have to turn it. Rather than a platform of enormous diameter to provide such leverage, you could provide slots for a piece of lumber to be inserted to increase the leverage, with multiple points around the platform so multiple people can assist.
It'd be like pushing a truck with multiple people, except it is made to be pushed and is on a surface made to be pushed on, with good handholds so multiple people can assist. And unlike pushing a truck, you are rotating it so you have leverage. Might be more doable than you think.
Micheal Angelo's David weighs 5665kg.
Let's very conservatively assume a coefficient of friction of 0.3, which means the pushing force required is 30% of the object's weight (which equates to a very soft wheel on rough terrain that doesn't roll very well, but I would expect the bearing surfaces for your idol platform would be much harder, smoother, and flatter which should decrease this).
That means that the statue on the rolling platform should require a force of 1699kg to overcome the friction. All other force on top of that just serves to accelerate/move it.
The status's base has a "radius" of 1m so it takes 1699kg-m of torque to rotate.
There was a little bit of handwaviness here since I wasn't entirely sure how to translate the force of friction for pushing something with linear motion into the friction torque that must be overcome to rotate it. I think you have to integrate and I don't feel like doing that. But I'm pretty sure the true torque required is actually a lot less since using 1699kg-m basically assumes the entire weight of the statue sits on its perimeter where the most torque is required to move it. Mass located nearer the center of the rotation takes a lot less torque to move. The smaller the horizontal radius you make the rollers underneath the platform, the less more leverage you get and the less torque will be required but you trade off stability. Think about a spinning top: it's tiny point right at the center means very little friction torque but it will easily topple over.
A person can push with a force of 45kg quite easily (I am quite small and can easily lift that with my legs) and uses a lever that is 2m long, then the torque produced by one person is 90kg-m. It would take 20 people to overcome the friction, and any additional force from them would rotate the statue.
Not too unreasonable. And this was assuming what I feel to be a rather high coefficient of friction and light pushing load per person. In reality, it should be quite a bit less than this.