Wednesday, March 18, 2015

Every build is special

Since Semmle's business involves integrating with our customers' builds, I'm very sympathetic to Tim Boudreau's perspective on simple plain-jane builds that just work:
But with time and enough consulting customers, you grow a real appreciation for projects you can simply check out from version control and build.

It's always a relief when someone wants to, say, build a stock iPhone app using xbuild. It's often fine when they want to build a C program using msbuild. Third place is Maven, for a simple Java program; I say third place, because hardly anyone uses it without loading it with a bunch of plugins that fundamentally change how it works. But still, at least the overall project structure is somewhat standardized.

In my modest experience, I only really see the above for relatively small and simple projects. As "Scott" puts it in the first comment at the above link:

The idea that your build is "not as special as you think" more often false than it is true.

I just skimmed a code base I have handy, and here are some of the things I see in there that break the mold of any off-the-shelf build tool:

  • It uses parser generators, such as ANTLR.
  • It's mainly a Java code base, but there is a little bit of code in all of: C, Objective C, JavaScript, Scala, C#, and Python.
  • It uses a home-grown stack trace obfuscator and de-obfuscator that involves bytecode rewrites on the deployed jars.
  • It includes its own version of zipmerge, with options not available in the open-source version.
  • It uses the JIT support in mini-Lua to generate machine code that is then compiled into string literals in a C program.
  • It includes a proprietary query language and numerous queries in that language. Those queries get compiled during the build, using a query engine that is also compiled during the same build.