I am working with a normalized ball-screw drive where the nut has holes for screws to screw onto the moving part. The catalogue says these holes have diameter 13.5 mm. What is the appropriate screw size (metric) to use on such a part?
I would suggest you use the largest diameter bolt you can find that fits through the hole, and that you can be sure you'll be able to install properly. By "install properly" I mean that you leave enough clearance between the bolt and 13.5mm hole to account for misplacement of the holes in whatever you're mounting the nut to.
In my work, when I design a clearance hole for a bolt, I usually go 0.5mm above the nominal size of a bolt, and we feel comfortable going down to 0.2mm above nominal if we need a more snug feet. A 13.5mm hole for a 12mm bolt is a lot of slop, so I would suggest a small washer if you go this route.
If you have imperial sizing available, a 1/2" bolt will have a nominal diameter of 12.7mm, so that would probably actually be your best bet. 13mm bolts do exist, but they're not common.