Road and Scouts: a versioning strategy for large research software

Road in Scotland. Photo by Sven Vee. The versioning strategy depicted here is permissive: “You can scout anywhere, providing you keep the main road in sight”, with a straight road.

Versioning of large research software applications for industry can be sloppy because audiences have different needs. End-users require a clear official version, while research asks for many exploratory versions.

Born in the 1990s, AVBP is combustion software used and developed today by dozens of industrial users and researchers simultaneously. This post will consider whether it is suitable for a semantic versioning strategy.

Research software audiences

Researchers

Researchers write software for their publications. Once the results are obtained, the focus moves onto the very demanding process of writing a paper (and finding funds, and supervising, and… ok). Versioning helps to make sure the results can be reproduced. 

Utopia is: One paper – one version

Industrial: production 

Industrial users employ software for design. A good simulation result is reproducible, independent of the software, and should not open new questions unrelated to the design (e.g. “I changed the tolerance, and it goes the other way!”). Trust is the main concern here, even before precision and performances. The trust in such complex tools takes years to build, and only weeks to crumble.
Utopia is: One version, period

Industrial: exploration 

Before the software is accepted for design, it undergoes a long validation process. Both industrial and academic tests are run until both Researchers and Industrial users give their approval.

Utopia is: One clean version per year, with a 3-year visibility on future versions

Industrial: demonstration 

Sometimes the case is really ambitious, and both sides agree to work on a demonstrator. This simulation will give insights on a specific application-driven issue, but will not be approved for production yet.
Utopia is: One demonstration, with as many versions as we need.

The main road: a bi-annual release

We start with the main official versions. The MAJOR version of AVBP today is 7.X., so all will start with a 7. 

Exploration vs stabilization

There are two MINOR versions per year.

  • The exploration version is released in March. This agenda allows the use of internships to test new features and make feedbacks. 7.2, 7.4, 7.8 were exploration versions.
  • The stabilization version is released in September. It allows a fresh start with a clean code base for new PhD students. 7.3, 7.5 were stabilization versions

Each version undergoes the following steps:

  1. Features are implemented through internal development
  2. Development is paused as tests are performed. These tests are:
    • a one-month test sprint for most versions
    • a “QPF campaign” once every 2 years (4 versions)
  3. A release candidate version is tagged and distributed to all AVBP customers, with all feedback requested within 3 months.
  4. The version is finally considered stable. A branch is created.

Note that “QPFs”, a.k.a Quality Program Forms are series of analytical test cases that the code is compared against. AVBP users can access these online.

You can see the development timeline, which is given below for the 3-year period 2018–2020. 

Note: due to COVID-19 disruption, 7.8 became a stabilization version.

Features & bug fixes

The version number is derived from the semantic versioning. In short, the features are frozen for each minor version. There are usually more new features on the explorations version. This means a new feature is never back-ported to a previous version.
However the bugfixes are added on minor versions, and can be identified by the third number. A minor version can have a personal evolution on its branch, with a rising quality (e.g. the 7.3 family : 7.3.0, 7.3.2, 7.3.3, 7.3.4 ).
As you have learned, these three digits (7.X,X ) mark out the main road of AVBP versions. You can read more about this in semantic versioning for non-regression.

Industrial validation

People in Industry do not switch automatically to each latest stabilization version. Indeed, even if the industrial non-regression is fully automated, each upgrade raises new questions about the production and can potentially erode trust.

The version upgrade is therefore decided by the Industrial user. Both sides prepare in advance for this period, which requires substantial effort on both sides if an unexpected problem arises.

From Rogue versions to Scout versions

When a demonstration is scheduled, it often requires a specific version of the software, (e.g. a new model under development by a Phd student. 
Forcibly merging this version in the official bi-annual schedule is not always a good idea. The AVBP versioning allows a fast-track.
The Industrial user can employ this “Scout version”, provided three requirements are met:

  1. The Version is on a development branch of the AVBP Git versioning system. The name is usually in a format like FEATURE/awesome_model. 
  2. Before any exchange of the branch, a new tag is added to help people in their discussion. Indeed, without a tag, a commit name is not human-readable: 0949c54a62f243488a63e695e4129315fada16f8. Make sure this tag is up to date and visible in your log file.
  3. The version is compared to the current official version with the closest Industrial validation case during the demonstration campaign.

The second item can be time consuming. This is why having an automated non-regression process for Industrial casesis critical.

Versioning the Scout versions

The tag format of an exploration version should follow semver with a unique word inside: 7.9.0-newmodel.1.
This is technically speaking a pre-release. The first part is the destination: in future this feature will be 7.9.0.
The alternative is to use the origin: this feature is built on the version 7.9.0. There are however good reasons to focus on the destination:

  • During a feature development, the developer must regularly merge the current best state of the code into its Feature Branch. So the origin changes, and the tag series does not make sense anymore: 7.9.0-newmodel.1, 7.9.0-newmodel.2, 7.10.0-newmodel.3. 
  • The main dev team is not omniscient. If the tag does not start with 7.9.0, they might forget you were targeting this version, and could add intrusive developments without warning you, so increasing your workload
  • It does not comply to semver: the – char says it is a pre-release. All 7.9.0-loremipsu* are assumed prior to 7.9.0. So you will create confusion for people who use semver on a daily basis, like the computer support teams of HPC centres.
  • As the minor releases are scheduled every six months, the tag name is a clue to the expected duration of the development.

Takeaway

The AVBP versioning uses:

  • a road, an official trunk of main versions with scheduled releases every 6 months. Names are 7.3.2.
  • scouts, rogue versions used for explorations and demonstrations, provided they match the current main version on the closest industrial reference case. These are semver pre-releases of a future version 7.3.2-awesomeidea.5.

Both are equally needed and complementary for an active research code.

Going off-road in Buena Vista (Photo Holly Mandarich)

Antoine Dauptain is a research scientist focused on computer science and engineering topics for HPC. 
Corentin Lapeyre is a research scientist focused on AI for physical modeling.