Avoiding big-bang updates

A big-bang update is a system update where the entire system must be shut down, multiple components are upgraded in parallel, and then the whole system is turned back on. If it goes well, then BANG, the whole system is upgraded in one big swoop. If it goes well, you lose a day or two of work from everyone working on that system, which is already pretty bad, but at least it's only a day or two. It often doesn't go well, however. Often, there is some dependency that snuck through early testing, and the one or two days extends into three, four, or five as engineers panic to patch up the lurking problems.

With a little foresight, big-bang updates can usually be avoided. The basic requirement is that system components can be updated one by one, without needing to update multiple components at the same time. That, in turn, implies that the bulk of components need to tolerate both the current versions of all other components as well as the upcoming version. Solve the multi-version dependency problem, and you can avoid big-bang updates.

Depending on multiple versions can be tricky. A common situation that arises, for any kind of system, is a desire to change a name that is shared between two different components. If the components are modules of source code, then the name might be a class name that is exported from one module and imported by others. If the components are hooked up using HTTP, then the names might be URLs. The temptation is to change the name, but that leads to a big-bang update. Every component that uses the name has to be updated at the same time. To avoid the big bang, try to find a way to support both the old name and the new name during a transition period. Then all components can be updated one by one to support both names. Once everything has been updated, it is possible to gradually drop support for the old name. It takes longer but it avoids a big bang.

An instance of the problem in programming languages is that module interfaces often tempt programmers into big-bang updates. For example, experienced Java system developers frequently exhort fellow developers not to change any interface once it's published. If you do change such an interface, then all users and all implementers of the interface have to update simultaneously. Bang. With Java as it stands, a better approach is to define a new interface with the desired changes and use dynamic typing to decide whether to use the old or new interface in any given circumstance. Alternatively, of course, an components-friendly programming language could directly support interface evolution directly in the language.