Git, Mercurial, others -- what's the best system for an engineering team new to version control?

Question

I recently started working with a new team doing detailed energy modeling for all flavors of building projects -- commercial, industrial, residential, new construction, renovations, additions. As the team has been growing, we've been discussing three particular problems that I believe version control could help with.

Our challenges

1. Coordinating multiple people working on a single energy model. Each building model involves different components -- geometry, envelope, HVAC, lighting, etc. For time-crunched projects different people may work on different components simultaneously. To bring each person's work together at the end can be complicated and time-consuming.
2. Keeping track of "known good versions" for a particular application. To streamline our work we create templates for various building components. Each template may only be ready to use for certain building types (say, HVAC template #n is working for commercial buildings but not tested for residential). When starting a new model we review release notes to make sure we apply the appropriate template, until our periodic review when we test/update templates for all use-cases. Both the tracking of historic versions, and the periodic integration of updates, is complicated and time-consuming.
3. Reproducing results from a report sent to a client. During model development we periodically prepare reports for clients. Each report is tied to a version of the model and templates which is archived on our server. This way we can re-open the old model to address any questions the client has, even as model development has continued. At times we also need to change aspects of the old model to answer specific client questions, before a model update is ready. At this point, the task of integrating two separate streams of model changes becomes... complicated and time consuming.

All of these processes can be improved with version control -- but nobody here has ever used version control!

I'm wondering if others here have been in a similar position, and implemented a version control system. What did you use, and how did it go? What best practices can you share?

Some details about our team and our work

• All engineers use Windows 10
• All of our modeling tools are Win32 applications (eQuest, Open Studio, TRNSYS), but modeling source files are stored as text-based files (not binary)
• We're considering Git, Mercurial, and Bazaar
• We do not currently have a server where we could run something like SVN, so we'd prefer a distributed system which could store files on a shared drive (such as a networked drive, SharePoint, DropBox, etc).

Answer 1

Source control isn't going to fix your problems. Let's review:

1. We don't have a way to allow multiple people to work on different components of a model simultaneously. Why do you have multiple people working on the same thing at the same time? Either (1) they're duplicating each others work, or (2) you have your stuff improperly written/stored as some monolithic thing instead of as sub-components, sub-assemblies, etc.
2. For some projects we want to use older versions with known issues, rather than newer versions with unknown issues. This often applies when a change has been made to fix a problem in one building type but hasn't been tested for others. Why is your team not testing the work that it's producing? Unit testing is a method where you have a list of inputs that you know could be provided to your software/tools, and then you also have a list of what the outputs that you expect your [stuff] to produce, given those inputs. As your team finds things/inputs that breaks your [stuff], you add that to the list of tests. There is no reason why your team should release work that would contain previously fixed bugs.
3. During model development we periodically prepare reports for clients. [...the toolchain configuration can be lost, which...] can result in hours spent figuring out how to make the model produce the previous results, which is about the biggest waste of time imaginable. Why are you generating reports without including information about how the report was generated?? Your report should have a section or appendix on "methods," which would generally detail how you're arriving at the results. Any time you use a reference, you cite it. If you are using a program/tool you or your company wrote, you cite it. Software citations should include the version number. If you're writing a report for a customer (non-profit or otherwise), and you're presenting numbers/facts with no explanation or reproducible source, they might as well come from your butt. Nobody has a means to verify anything you say.

So, to summarize, it sounds like your team is working on some monolithic project, where people are overwriting each others work, and then you're using this to generate reports with no citations to the software version you're using to generate that report.

Version control doesn't fix any of this.

Git doesn't structure your project. Git doesn't assign work flows. Git doesn't (automatically) know when you are at a stable release. Git archives your work. That's it. Any version control just archives your work.

Don't have time to setup a central repository? Then how do you expect each individual developer to share work? Don't have anyone responsible for maintaining the archive? Then whose job is it to merge branches?

Answer 2

Bearing in mind all of the points that Chuck's answer brings up, it still sounds like you have more people problems than technical problems. No piece of technology can resolve people issues, but based upon the comments it sounds like you need something to facilitate the process improvement.

The modeling files you describe don't sound like they'll merge gracefully, so I believe you'll need to use a version control system that allows locking instead of just merging. If Ashmi needs to wait until Jane is done working on a component before Ashmi can make her changes, then locking is the way to go.

It also doesn't sound like you need a distributed VCS. Everyone needs a client regardless of what you pick, but you don't need the ability (or complexity) of running your own local VCS. DVCS's are merge only for concurrency and are more useful when running detached from the main repository. Branching and merging are (potentially) more sophisticated with a DVCS, but again, I don't foresee you needing that capability.

I would recommend looking into SVN. Yes, you'll need to setup a "server" to provide a central repository. You need a central repository regardless of what VCS you choose, and SVN is comparatively lightweight to setup. And "server" is in scary quotes because in reality all you need is a spare desktop to act as the server. VisualSVN may be worth looking into if the Apache SVN route appears too daunting. TortoiseSVN is a pretty reasonable client to consider as well, but there are several others to consider.

Another advantage of SVN is that it's a lot easier to verify that "really-old-project-with-known-issue" has remained in that state by simply examining the file system hosting that branch.

But if for whatever reason that SVN is an absolute non-starter, then take a look into a private GitHub account for your main repository and git for your local clients. You'll end up with a lot more headaches this way, but it is an option.

Answer 3

It does not matter. Any version control is better than none. What you choose depends on your needs. Git has a more elborate system that allows editing of history afterwards. Personally i find that mercurial is more straightforward for people who haven't done much version control.

But this really depends on what system you are on and whether you expect to run the system from commandline or a GUI frontend of somekind.

Anyway even on a distributed system you still need a central server for synching. Even if it is just a shared drive.

Answer 4

To close the loop on this, and in case it's helpful for anyone else in the future... Here's the solution we ended up picking, and a few of the reasons and resources that moved us in that direction.

Let me preface by saying, as several have pointed out in the comments and answers, that the key to implementing version control is the mindset change and consistency of following a new process. We had a process that emulated what we do now, but it was complicated, time consuming, and prone to errors.

Here's what we ended up doing.

• git
  • While looking into SVN I found a helpful article, but even more helpful was this comment on the article: "5 reasons, coming from someone who has moved from subversion to git twice voluntarily". The ability to seamlessly work off-line or outside of our internal network were big selling points.
  • A comment I came across a few times (see this answer on softwareengineering.se, for example) is that merging in SVN is complicated, and for those less comfortable with SVN can lead to a reluctance to commit. This defeats the purpose.
  • In contrast, I found this article from Atlassian explaining that in git, "commits are cheap." Version control is most useful when frequent commits are made, so I wanted to lower the bar as much as possible for the team.
  • Because I knew I needed to get a certain level of buy-in before devoting significant resources to the project, setting up a server (such as would be needed for SVN) was off the table. I don't have the skills to do this, and convincing our small team to put time/money into something that they aren't convinced they need (when there are other things we all agree we need but that we can't afford yet) was a non-starter. Of course, this is a Catch-22: If we had a server with SVN set up, we likely could have gotten a satisfactory SVN process going.
  • I set up TortoiseSVN on my PC and used it for a couple of weeks. It was functional, but not exceptional.
  • Once I decided to move away from SVN to DCVC, the choice between Mercurial and git was decided by the fact that I had used git in the past.
• GitLab
  • Free and open source.
  • We did consider GitHub, but it is limited to three collaborators on private repos.
  • Bitbucket is limited to five users for free teams.
  • As we (and our resources) grow, we may revisit hosting. This part of the process would be relatively easy to change.
• Sourcetree
  • Based on comments on a forum for energy modeling (which is our use case), I decided to give Sourcetree a try. I found the interface helpful and intuitive.

Some other helpful resources:

• Oliver Steele's "My git workflow". The graphics were particularly useful for explaining git to our team.
• "Version control for energy models". Helped us think through developing/implementing a workflow for our particular application.