Software! Math! Data! The blog of R. Sean Bowman
The blog of R. Sean Bowman
April 17 2015

I’ve been using Rake a little bit recently, and I’ve also been building a lot of C++ software, and the whole experience has made me think about build systems for the 27th time in my life. Make is okay (it’s ubiquitous, certainly), Rake is interesting, particularly in the Ruby ecosystem, and there are a ton of alternative build systems out there. Today I want to talk about my current favorite one, redo.

What’s the point of another build system? There are tons of general purpose build systems: Make, SCons, Rake, Ninja, Tup, Waf, in addition to language specific systems like Ant, Maven, Grunt, Gulp, Leiningen, sbt… The list goes on and on.

Redo is based on ideas of Daniel Bernstein, a famous mathematician and computer scientist who has written lots of nice software (qmail, for example). As far as I can tell, he never actually implemented the ideas in his build system, but other people thought it was cool enough to carry through. The good news is that nowadays there are several implementations of his system. The bad news is that… there are several implementations of the system. It’s hard to tell which are actively maintained, which have advantages or disadvantages over others, and so forth.

The one that I have used in the past is called Redo, built by Apenwarr on github. I have never had any problems with it, and although there are now many more implementations (some faster, some fix small problems with the original implementation, etc.) I still love this original Redo.

Redo

What’s so cool about it? The idea is this: there is a small set of programs; we’ll focus on redo and redo-ifchange for now. You write shellscripts that use these small programs to express dependencies among files/activities, and you otherwise describe how to build the files. It’s a very easy idea at its core, but it can take some getting used to. Let’s look at some examples.

Redo works by having lots of scripts with the extension .do. When these are called, they’re called with three arguments: $1 is the name of the target file (the file to be built), $2 is the target minus any extension, and $3 is a temporary file that will be renamed to the target file if the script is successful. With that in mind, let’s compile a small C program. In common.h we have

typedef struct X {
    int a, b;
} X;

int function(int);

in bar.c we have

int function(int x) {
    return 2*x + 17;
}

and in foo.c we have

#include <stdio.h>
#include "common.h"
int main(void) {
    X x;
    x.a = 4; x.b = 3;
    printf("sum is %d", x.a + x.b);
    return 0;
}

We’d like to make an executable foo from these files, and if any one changes, we’d like to rebuild foo. How do we do it using redo? Let’s start at the top level. Since we want to make the file foo, we’ll call our top level script foo.do and assume for a moment that we have object files.

DEPS="bar.o foo.o"
redo-ifchange $DEPS
gcc -o $3 DEPS

This simple shellscript uses the redo-ifchange program to determine if the files bar.o or foo.o have changed since we last build the software, and if so, calls another script we’ll describe shortly. Otherwise, it compiles the object files into the executable foo.

Now what if one of the object files changes? It may be that we have many object files, and so Redo allows us to specify a way to handle them all at once. We create a file default.o.do, which will run when an object file needs to be generated (unless some more specific script applies to it). Our default.o.do looks like this:

redo-ifchange $2.c common.h
gcc -c -o $3 $2.c

Again, this script says that if C file the object file depends on has changed since we last built it, we need to build it again.

When we run redo foo for the first time, we see output like this:

redo foo
redo foo.o
redo bar.o

At this point, our foo executable is ready. Now suppose that we change something slightly in in common. When we run the program again, it notices that common.h has changed, and so it needs to rebuild foo.o and bar.o, and then link them to get foo again.

There’s a high level overview of Redo. I’d like to keep talking about it because it has some really cool features. Next time, we’ll have a look at Redo’s unique take on dependency tracking.

Approx. 755 words, plus code