Here is my list of build machine/infrastructure centered focus areas – some technical and some attitude based.

* Use a source control system – be it svn, git, p4 or something else.
* Use a build machine (TeamCity, CC.Net, Cruise, Jenkins, …)
* Make that build machine trigger a compile of everything on check-in.
* Write unit tests.
* Run the unit tests on build machine after the checkin-build.
* Run static checkers and source analysis (fxcop, stylecop, cppcheck, cpplint) on the build machine on each check-in.
* Break the build on failure.
* Make build failures obvious (lava-lamps are great!).
* Foster a culture of checking in often.
* Foster a culture of keeping build-broken times to a minimum.
* Do not assign shame to a broken build.
* Make sure whatever your build machine does can be executed on every developer machine.

The above statements leave a lot to be worked out, I know. However, no matter how you solve it you are better off if you do!

If this seems like a tall order, you’ll probably need some help. Not to worry! The internet is full of advice.

Have fun setting up your team and the machinery to support it. I do look forward to your input!

–Jesper Hogstrom

Dear Vincent,

the other day I was in a discussion about what can get cancer, something that ultimately boiled down to what cancer really is. Our reasoning went along the lines of establishing that there are organisms containing any number of cells, ranging from one and up. If I remember correctly, C. Elegans has 957 cells. Presumably there is some organism with 956, 955 and so on.

It seems it doesn’t make any sense to talk about a one celled organism developing cancer – though I am interested to have that confirmed! The nematode just mentioned has cell specialization, so it could presumably develop some form of cancer. What is the lower limit of cells an organism must have to succumb to the decease or should the question really be posed in a completely different way?

Also, some organisms of very few cells occasionally gang up and form a super-organism. This includes some slime moulds and the pre-larvae state of jelly fish. Can such “temporary” organisms develop cancer?

The question is grander than just parasites, and I have a feeling that viruses, living or not, have no propensity to develop cancer. Hence my addressing the question to TWIM.

While I have your attention, allow me to once again thank you and everyone in each of the podcast teams for your effort in sharing your knowledge and doing it in such an enjoyable tone and fashion.

All the best,

Jesper
Software architect
Sweden

The question is answered by Vincent andElio near the end of the show.

The good reason and the fact that it was quite an interesting question got me thinking on how to do that with minimal effort. Minimal effort is always a good sign of a lazy programmer, and lazy programmers are better. At least that’s what I read somewhere

First of all, it wouldn’t be possible to simply call the target Build again, as msbuild doesn’t invoke the same target twice in one run.

I figured it’d be necessary to call msbuild again, and in the recursively invoked msbuild to all the tricks.

First off, I wrote a simple program to extract some information from a binary file.

Not overly complex, but good enough for demonstration purposes. I compiled this to genapi.exe and moved it to the root folder of the project I wanted to tamper with.

Msbuild is a quite powerful system, and it has numerous extension points. One of these is to override – or actually redeclare – a target named AfterBuild. This target is invoked after the chain of targets pertaining to build is complete. The default implementation of AfterBuild is to do nothing.

At this point I need to mention the importance of understanding the build system used. There are quite a number of problems that can and should be solved by extending and enhancing the build process. The first step on that journey to enlightenment is to start thinking about the build system as part of the product development infrastructure. It is not just some mumbo-jumbo mixed with two bottles of magic that happened to ship with .Net and Visual Studio. It is an orchestration environment of quite some power.

Anyway, start with right-clicking the project node in the solution explorer and select Unload project. Right-click the same node again and select ‘Edit…’. You’ll now get the csproj file as a textfile in your source editor in VS.

The end of the project file looks like

This is exactly what we want to do. First, to try the waters, I wrote a simple AfterBuild target just to make sure it got executed.

Reload the project file, and turn on more logging by selecting Tools | Options | Projects and Solutions | Build and Run and go for Normal in both the combo-boxes. View output from build by Ctrl-W, O.

Rebuild and you’ll get something like

Great. Build failed.

Now let’s do some serious work.

What we need is to generate the “api”. The target for this looks like

The generated file, ‘api.txt’ needs to be included. We could in theory add it to the project once we have it, but imho it makes much more sense to only add it after it is generated. Thus, we need to modify the Resource item. To avoid doing that too early, this must be done in a target as well. Manipulating items and properties in targets is msbuild’s way to handle deferred evaluation.

Now what with the afterbuild target? There are many ways to skin a cat, and I opted for two invocations of the csproj file, first to generate the resource, and then to rebuild the project with the freshly generated file.

However, the code above suffers from a serious mistake. When msbuild is invoked for the second time, it will surely call build, but it will also end up in the AfterBuild-target and call itself again. Infinite recursion. Not good. To prevent this from happening, we need to pass a token indicating we’re in level two, and also check that before executing anything. Improved version looks like

Much better! However, when are we calling the AddResource-target? To fix that we need to do some indirection, as msbuild doesn’t seem to commit item changes to global scope until a target is completed. Thus, instead of invoking the Build-target, we invoke a new target named SecondBuild. It in turn depends on AddResource and Build.

The complete code that works thus looks like:

Staying true to the agile programming cycle, this is where one should pause and refactor. Could the code be written any cleaner? Do we need two invocations of msbuild? As it turns out, no, not really. The same effect can be achieved with the following, more compact version:

There you go. A simple way to include information about a binary in the binary itself. All from one Build request from inside VS.

A very interesting question and a fairly simple and clean answer.

Happy building!

–Jesper

This simple little file will allow you to type things like build debug or build release without worrying exactly how to specify the target. If you want to make it extra nice, control log level with an environment variable, by adding the following line:

simply set loglevel=/v:n from command line to change the log level.

You may also want to pass in some variables, like current directory

That will give you the property $(StartDir) available inside your project file.

Speaking of which, here’s a minimal sample:

Admittedly very bare bone.

Let’s add some meat:

If you follow the good practice of adding new targets to the item list _availableTargets, you have a help page showing if you pass in no parameters.

Obviously we’ll have to write the actual targets too, but so far we’ve seen that
* We can add things to a list.
* Display that list with a different separator than semi-colon.
* Define properties with escaped characters (by using the hexadecimal ascii values prefixed by a percent character).

It’s far from complete, but it’s a very good start!

–Jesper Hogstrom

* Ensure everything compiles after each check-in (structural integrity)

* Execute unit tests to ensure functional integrity.

* Run various kinds of static analysis on the source or binaries.

* Prepare builds for internal (e.g. QA) and external (e.g. customers) consumption.

A build machine is typically controlled by a build orchestrator, like Cruise (now a part of Go), CruiseControl.Net, TFS or TeamCity. It may be a single machine, a system of agents or something else. Let’s call this the build machinery.

The build orchestrator is in some way connected to the version control system (VCS), be it Perforce, Git, Subversion or TFS.

When a file is updated in the VCS, the build orchestrator gets a signal and starts executing a build script, typically written in msbuild, ant/nant, make or some shell language. It is to prefer to use a language that is designed for build orchestration, thus allowing for specification of dependencies, error handling and maybe even logging.

The fundamental components of a build system are thus:

* A build script

* A build machinery

* A version control system

Luckily, the build machinery and version control system are standard products, so you don’t need to worry about rolling your own.

However, the build script is something where you want to invest some energy. The purpose of the build script is to ensure that you can reproduce a bit-identical build from bit-identical sources. It should also have the ability to execute a suite of other targets.

A target is the end result of a build script invocation, or rather, what you want the script to produce – the transformation of inputs to an output.

Some examples of targets you want your build script to produce:

* Debug - A debug build of all sources.

* Release - A release build of all sources.

* Testcases - Compilation of all unit tests.

* Documentation - Collect all documentation source documents and transform them into the desired output format.

* Installer - Collect all files that goes in to the installation file and produce it.

You can also have your build script do a number of other things, such as

* RunTests - Execute all unit tests in the test framework you use.

* RunSmokeTests - Execute all UI level tests using whatever test robot you use.

* Publish<something> – Upload some artifact to a different server for wider consumption.

* Clean<something>, Spotless – Remove output from the source tree, ensuring next build creates everything from source documents.

An important aspect of a build script is that it should be possible to execute both on the build machiner/build agents and the developer machines!  You should be able to “make debug” or “build documentation” or “want testcases” from a command line prompt on all relevant machines. Otherwise it is very hard to perform an action on a developer machine to ensure the check in will pass. It also helps a great deal when developing the build script.

There are some really critical consequences of the above:  The build system must be self-contained! This means everything required during compile time must be reachable by the build machinery – and if it isn’t checked in you need some other way to pin which version (of a tool, SDK or third party tool) you want. Speaking of which, you’ll also need to version control the build scripts themselves.

So, to summarize what you need to get started:

* A build machinery.

* A version control system.

* A build script that automates everything that happens between coding and preparing a testable artifact.

* Everything required during compile time and test time reachable by the build script (either checked in or reachable on another file system).

The build script and repository of prerequisites require by far the most design – or rather – those two items are by far the least described out there.

I’ve designed build systems using various build machineries (CruiseControl.Net, TeamCity, TFS) using both make and msbuild. Nowadays I write msbuild scripts building .Net and C++ projects for a teamcity environment. However, there are a number of concepts that do not change. I’ve found the most important to be:

* Do not rely on what is installed on the machine you’re building on. Everything must be controlled!

* Designate an output/staging directory, and make sure everything that is produced ends up in under that directory.

* Define a structure of all projects and stick to it.

* The build should behave the same way if building from inside an IDE as when building from command line.

Everything must be controlled

Developers typically install something that goes into a build, add some references to globally available (GACed in the .Net world) DLLs and everything works just fine. Until the build happens on a different machine, when things blow up, as the global state of that machine isn’t the same as on the dev machine.

It is imperative that everything you use during compile time is checked in! This means not only your third party components, but also the unit testing framework and the tools used in the build chain, such as help compilers, installer creators and so on.

Often it is quite simple to meet this requirement. At other times it is harder, like when COM-servers need to be registered, registry entries must be found and other evil global conditions that must be fulfilled. As the build master, your job is to understand how you can script things to make all machines look like that have enough of the global state to fool the tools and components you use.

If you adhere to this rule adding a new build agent, or replacing a developer machine is very much simpler, as fewer things need to be installed.

I draw the line at the .Net level – all involved machined are allowed to have .Net installed. It would be possible to write scripts that didn’t rely on the presence of a .Net installation, but I find it not worth it at this point. If we were targeting strictly different .Net versions I might reconsider.

Staging directory

Visual Studio adds a bin and obj directory under each project. Not nice. You’d rather have all output end up under stage\debug or stage\release or stage\installer. Keeping everything under a stage directory reduce the risk of someone checking in binaries they’ve just built, and makes cleaning the tree significantly easier. Instead of traversing the world, you just rmdir stage\debug. Or rmdir stage if you really want to make things spotless.

Achieving this is not incredibly difficult, as all compilers tend to have a flag indicating where they should deliver the output. The trick is finding the sweet spot that minimize the amount of configuration required for each new project.

Project Structure

Project structure makes it easier to find your way around projects, reduce the learning time when moving to a new team and makes it much easier to write a build script that works across the board, as everything looks quite the same. I don’t know what structure makes sense to you, but something like

-$+
  +-Build
  +-Common
  +-Documentation
  +-Source
  +-Stage
  +-Tests
  +-Xternals

is a good start. It is also fairly obvious what goes where. Try to keep the root directory rather clean!

Behave the same from command line and IDE

This is trickier than it sounds. Assume you set up a wrapper msbuild script that calls msbuild to build your solution file. You play nice and pass in whatever properties you need to the second level of msbuild and everything works as expected. From command line.

From inside Visual Studio however, you don’t get the parameters passed in from your script, as VS is now the root caller. Your fellow developers will get a different result than you and the build agents. There are likely several ways to accomplish this, and they all involve some manipulating the project files (csproj, vcxproj). If you are using some other IDE there are surely other ways.

That just about covers the high level aspects of a build system. Admittedly there are details left to cover, but they lack the generality I wanted for this post. The above should work for most circumstances and most platforms.

–Jesper Hogstrom

Let’s say we have a test that looks something like

In essence, the tests spell out the specs for MyCoolGizmo! It has the following capabilities
* Adding Only Permitted In Transaction
* Invoking Add AddsItemToList
* InvokingExecuteRunsExecute On All Items
* …

Since that’s exactly how we named the tests (what a coincidence ) all we need to do is extract that from the code in some way. My first choice is always reflection in situations like this.

Here goes:

First, create a command line application.

Add some supporting functions and definitions

We’ll also need to define the interface for the generators and a class to store the logged items in.

The scanner turns out to be quite straight forward

The only remaining this is to define the result generators, which can of course be changed to suit your needs to produce other formats should that be desired.

To compile you’ll need to add a reference to nUnit. Then run it with appropriate switches, something like

What I set out to do was to have a declarative way to define how parameters are constructed, all driven by an engine in a superclass. Declarative means attributes.

As an example, I want to wrap rsync in a task class. rsync has a great number of parameters, and I want to write something like

I also envision that I will have to write more wrappers of command line tools, and I don’t want to write a command line generator for all of them.

There are only a few types of parameters to pass to command line tools:
* Existence of a parameter, like “–recursive”.
* A switch with a value, like “–password-file=FILE”.

Now, if you’re not familiar with rsync, too bad for you! However, the reasoning holds for any executable you want to wrap in a task instead of using Exec.

Thus, in my subclass, I would like to specify this as something like

The way to go is to first define a couple of simple attributes.

Now we need an intermediate class to operate on these attributes. The most fundamental level is to construct the path to the tool.

This makes sure you can set the toolpath if you want to.

As you notice, I subclass ToolTask, as the msbuild team as done a great job of some parts of the infrastructure.

The key part is to make magic in the virtual GenerateCommandLineCommands.

The above, rather simple code, achieves the main objective. Boolean attributes cause a switch to appear or not appear, omission of an attribute prevents a nonsensical switch to appear.

For debugging purposes, let’s add a DryRun option. This should print out the command line, but not execute anything.

At this point you’ll notice that the output is not shown unless you increase the loglevel to msbuild. Why not make that declarative too?

This allows you to specify if the output of your tools when should show up.

In some cases you probably want a different default value than the .net defaults. Luckily there is already an attribute defined for this: DefaultValueAttribute.

In the rsync case, we might want to set recursive as default, specified as something like

Add to our base class the following code

As much as we all love writing documentation, we know that the others won’t read it. Why not make our tasks self documenting? What if we could add “ShowHelp=’true’” to our msbuild script and get the meaning of each parameter explained?

Again, we make use of an existing attribute, DescriptionAttribute.

Add it as follows in your subclass:

and add the following to the base class:

This will yield an output similar to

The full code for the base class:

A sparse implementation of rsync:

As you can see, there is no code – other than the declarative parts – required to wrap a command line excutable into an msbuild command, regardless of the number of parameters it needs.

Happy building!

–Jesper Hogstrom

Ta en påse Santa Maria Glöggkryddor och lägg i ett kärl.
Ta två skivor (3-4mm) ingefära och lägg i samma kärl.
Klipp ner ringar (2-3mm) av en halv thai chilifrukt i ovan nämnda kärl.
Häll över ca 1.5-2 dl vit sprit, vodka e dyl.

Vänta i ett dygn (det är den svåraste delen ).

Häll ner alltihop i en gryta, häll ner 1 flaska halvtorrt vitt vin (jag tar naturligtvis mitt eget äppelvin!).
Tillsätt ca 1.5dl muscavadosocker. Öka i förhållande till vinets torrhet!
Koka upp under varsam omrörning.

Stäng av plattan och häll ner 0.5-1 dl enkel whiskey (inte rökig singlemalt!) och en skvätt gin. Om du har lite sherry eller portvin så häll på en skvätt av det med. Det mjukar upp smaken.

Rör om och vänta ett par minuter. Häll upp i muggar – jag menar glöggkoppar och njut!

Som tillbehör använder jag kandisocker, mest för att i efterhand kunna balansera sötman i drycken. Lagom sött = inget kandi! Aningen för torrt – 1-2 kandi-bitar. Och så vidare.

Chilin ger glöggen en eldighet som värmer gott i vintermörkret.

När allt tagit slut kan du ta vara på kryddorna, lägga tillbaka dem i kärlet och hälla på ny sprit. Imorgon kvaäll kan du börja om. Då och då kan du lägga i mer av antingen kanelstång, ingefära eller nejlika.

–Jesper

Luckily, it’s not incredibly hard. I wrote myself a little msbuild task that does the job. Here goes:

Using it in a script might look something like:

Once I had that written and tested, I decided to check if either the tigris community tasks or the microsoft extension pack included something similar.

Yes they do. Both of them. However, it was a nice exercise in writing msbuild tasks and a reminder to first check what is available out there

–Jesper

The problem is that the tabs very soon spans the available space and scrolling becomes necessary.

To the rescue is the firefox settings.

Type in “about:config” in the address field. Acknowledge that you know what you’re doing.

Filter the rather large (and very interesting!) list of things possible to change by typing “width” in the filter-field.

Change 100 to 16 and restart firefox.

Now, your tabs will shrink down to 16 pixels, enough to keep the favicon visible, and you can see many more before scrolling kicks in.

Happy browsing!

–Jesper Hogstrom