It's accurate to suggest that a drill press is not designed to handle lateral forces on the bearings and spindle. Contrary to that, you'll see plenty of posts to various locations made by people who have added a drum sander to their drill press. This is certainly a lateral load on the spindle.
For a one-off task, it's likely that you would not cause excessive runout on the bearings and spindle, especially if you keep your load at the right level.
For aluminum, you'd want a medium to high speed on the spindle and to use a lubricant such as kerosene (paraffin?) although some use WD-40. This lubricant prevents the aluminum from "gumming up" the flutes, effectively damaging the work piece.
Should you decide to attempt this, create a strong jig to hold your workpiece. There is little-to-no chance of success if you attempt it by hand. You'd want a set of rails aside the tubing with a clamping mechanism to prevent rotation of the tube. Those rails should be mounted in a suitable sliding device to provide the necessary travel. Consider also to drill a start and end hole to facilitate the beginning of the operation, although the end hole isn't as important.
During the cutting operation, you should see slivers of aluminum, a sort-of cresent shape. If you don't get a uniform collection of slivers and instead get small particles, increase your feed rate and/or reduce the spindle speed.
You can also consider to cast about for a mini-mill group (groups.io) or a local mini-mill owner to assist you. If I learned of a maker in my area with this straight-forward task, I would certainly jump in to make the cut.
