First, to counter what some people are saying about printed metals, selective laser sintering can produce parts that are as thermally conductive as their base metals. They are limited only by porosity, and state of the art machines can produce fully-dense (zero porosity) metal parts.
Second, you probably don't need to take radiation into account because you are at low temperature and airflow is going to make a much bigger contribution than radiation.
Third, traditional heat sink design uses equations that calculate the temperature distribution along the fin from the root to the tip in order to come up with an "effective area" upon which an average convection coefficient can be imposed. This means that you aren't calculating the exact airflow, but rather using an average over the whole surface.
Any good heat transfer textbook will have formulas for many different fin shapes, including fins with a non-constant fin area. I personally like Incropera and DeWitt.
Nevertheless, if you design sufficiently complicated fins, the standard equations will break down. You then have three choices:
- Approximate your fins with similar fins. This can lead to results that are close enough.
- Simplify your design to make it fit an established pattern
- Go to a full finite element simulation.
I would advise using number 1 or number 2 because they might be more accurate. It is easy to make subtle mistakes in finite element models that completely invalidate your results1, however if you do choose to go that route, you should know that there are free open-source FEM solvers (Elmer comes to mind).
And now some general tips:
- As others have noted, stainless steel is a bad conductor of heat. Aluminum is probably a much better choice, but unless you decide not to 3D print it, you probably can't use aluminum. This is the reason that I probably would print something else if I had access to a 3D printer
- The Coefficient of Performance of Peltier coolers is often less than one, so in order to get e.g. 20W of cooling power, you will need to put in more than 20W of electrical power. Your heat sink needs to dissipate the sum of the cooling power and the electrical power.
tl;dr: You can model heat sink designs without using FEM and as long as your designs are not too crazy, you will probably get good results. Traditional "by-hand" calculations don't take into account radiation and assume an average airflow over the whole surface, which is usually good enough. I personally would avoid making heat sinks at all and use your time on the 3D printer for better things.
1 Trust me, I'm a graduate student in Mechanical Engineering, and I've screwed up so many models.