Designing a build system

When developing software it is somewhat obvious that the programmers need to be able to build the source code and run the tests on their own machines. It is slightly – but only slightly! – less obvious that the same code needs to build on a separate machine, known as a build machine. There are several purposes of that, mainly to

* Ensure everything compiles after each check-in (structural integrity)

* Execute unit tests to ensure functional integrity.

* Run various kinds of static analysis on the source or binaries.

* Prepare builds for internal (e.g. QA) and external (e.g. customers) consumption.

A build machine is typically controlled by a build orchestrator, like Cruise (now a part of Go), CruiseControl.Net, TFS or TeamCity. It may be a single machine, a system of agents or something else. Let’s call this the build machinery.

The build orchestrator is in some way connected to the version control system (VCS), be it Perforce, Git, Subversion or TFS.

When a file is updated in the VCS, the build orchestrator gets a signal and starts executing a build script, typically written in msbuild, ant/nant, make or some shell language. It is to prefer to use a language that is designed for build orchestration, thus allowing for specification of dependencies, error handling and maybe even logging.

The fundamental components of a build system are thus:

* A build script

* A build machinery

* A version control system

Luckily, the build machinery and version control system are standard products, so you don’t need to worry about rolling your own.

However, the build script is something where you want to invest some energy. The purpose of the build script is to ensure that you can reproduce a bit-identical build from bit-identical sources. It should also have the ability to execute a suite of other targets.

A target is the end result of a build script invocation, or rather, what you want the script to produce – the transformation of inputs to an output.

Some examples of targets you want your build script to produce:

* Debug - A debug build of all sources.

* Release - A release build of all sources.

* Testcases - Compilation of all unit tests.

* Documentation - Collect all documentation source documents and transform them into the desired output format.

* Installer - Collect all files that goes in to the installation file and produce it.

You can also have your build script do a number of other things, such as

* RunTests - Execute all unit tests in the test framework you use.

* RunSmokeTests - Execute all UI level tests using whatever test robot you use.

* Publish<something> – Upload some artifact to a different server for wider consumption.

* Clean<something>, Spotless – Remove output from the source tree, ensuring next build creates everything from source documents.

An important aspect of a build script is that it should be possible to execute both on the build machiner/build agents and the developer machines!  You should be able to “make debug” or “build documentation” or “want testcases” from a command line prompt on all relevant machines. Otherwise it is very hard to perform an action on a developer machine to ensure the check in will pass. It also helps a great deal when developing the build script.

There are some really critical consequences of the above:  The build system must be self-contained! This means everything required during compile time must be reachable by the build machinery – and if it isn’t checked in you need some other way to pin which version (of a tool, SDK or third party tool) you want. Speaking of which, you’ll also need to version control the build scripts themselves.

So, to summarize what you need to get started:

* A build machinery.

* A version control system.

* A build script that automates everything that happens between coding and preparing a testable artifact.

* Everything required during compile time and test time reachable by the build script (either checked in or reachable on another file system).

The build script and repository of prerequisites require by far the most design – or rather – those two items are by far the least described out there.

I’ve designed build systems using various build machineries (CruiseControl.Net, TeamCity, TFS) using both make and msbuild. Nowadays I write msbuild scripts building .Net and C++ projects for a teamcity environment. However, there are a number of concepts that do not change. I’ve found the most important to be:

* Do not rely on what is installed on the machine you’re building on. Everything must be controlled!

* Designate an output/staging directory, and make sure everything that is produced ends up in under that directory.

* Define a structure of all projects and stick to it.

* The build should behave the same way if building from inside an IDE as when building from command line.

Everything must be controlled

Developers typically install something that goes into a build, add some references to globally available (GACed in the .Net world) DLLs and everything works just fine. Until the build happens on a different machine, when things blow up, as the global state of that machine isn’t the same as on the dev machine.

It is imperative that everything you use during compile time is checked in! This means not only your third party components, but also the unit testing framework and the tools used in the build chain, such as help compilers, installer creators and so on.

Often it is quite simple to meet this requirement. At other times it is harder, like when COM-servers need to be registered, registry entries must be found and other evil global conditions that must be fulfilled. As the build master, your job is to understand how you can script things to make all machines look like that have enough of the global state to fool the tools and components you use.

If you adhere to this rule adding a new build agent, or replacing a developer machine is very much simpler, as fewer things need to be installed.

I draw the line at the .Net level – all involved machined are allowed to have .Net installed. It would be possible to write scripts that didn’t rely on the presence of a .Net installation, but I find it not worth it at this point. If we were targeting strictly different .Net versions I might reconsider.

Staging directory

Visual Studio adds a bin and obj directory under each project. Not nice. You’d rather have all output end up under stage\debug or stage\release or stage\installer. Keeping everything under a stage directory reduce the risk of someone checking in binaries they’ve just built, and makes cleaning the tree significantly easier. Instead of traversing the world, you just rmdir stage\debug. Or rmdir stage if you really want to make things spotless.

Achieving this is not incredibly difficult, as all compilers tend to have a flag indicating where they should deliver the output. The trick is finding the sweet spot that minimize the amount of configuration required for each new project.

Project Structure

Project structure makes it easier to find your way around projects, reduce the learning time when moving to a new team and makes it much easier to write a build script that works across the board, as everything looks quite the same. I don’t know what structure makes sense to you, but something like

-$+
  +-Build
  +-Common
  +-Documentation
  +-Source
  +-Stage
  +-Tests
  +-Xternals

is a good start. It is also fairly obvious what goes where. Try to keep the root directory rather clean!

Behave the same from command line and IDE

This is trickier than it sounds. Assume you set up a wrapper msbuild script that calls msbuild to build your solution file. You play nice and pass in whatever properties you need to the second level of msbuild and everything works as expected. From command line.

From inside Visual Studio however, you don’t get the parameters passed in from your script, as VS is now the root caller. Your fellow developers will get a different result than you and the build agents. There are likely several ways to accomplish this, and they all involve some manipulating the project files (csproj, vcxproj). If you are using some other IDE there are surely other ways.

That just about covers the high level aspects of a build system. Admittedly there are details left to cover, but they lack the generality I wanted for this post. The above should work for most circumstances and most platforms.

–Jesper Hogstrom