The fallacy of trust

I’ve recently been reading Absolute FreeBSD: The Complete Guide to FreeBSD by Michael W Lucas. The book is packed with info about using FreeBSD, mostly from a server perspective; it has a lot of information related to keeping server boxes secure (as it should). In a footnote in Chapter 9, Lucas provides the web address to an article which is no longer available at that address; Archive.org saves the day with a cached version of the article.

Entitled “Reflections on Trusting Trust,” it’s a transcript of a speech given by Ken Thompson, one of the graybeards behind Unix as well as a few other neat technologies including UTF-8. It describes a security breach involving an operating system’s compiler - the program which takes program source code and turns it into an executable program. The speech goes into detail, but to sum it up, the breach works like this.

The attacker modifies the source code of a compiler, adding two new instructions:

1. If the compiler is compiling the Unix login program, and the password provided is a certain password, allow the user full access rights to the computer. Otherwise, behave as normal.

2. If the compiler is compiling the compiler (another copy of itself), add these two new instructions to the compiler.

The attacker then compiles the code to produce a tainted compiler, then removes the two instructions above from the compiler’s source code to cover their tracks. But it doesn’t matter, because from now on, any copies of the login or compiler programs the compiler creates, or that a compiler created by the compiler creates, etc, will be “tainted.” If the compiler is then distributed in binary form to a wide number of systems - say, as an operating system release - then you’ve suddenly got a wide range of systems out in the wild which one hacker can gain root access to with a single password.

Once I wrapped my head around how the attack works, I was struck by both its simplicity and its practicality. Who’s to say such an attack isn’t already happening, really? Maybe not by a malicious hacker, but by a government interested on keeping tabs on its citizens and/or international neighbors…

Or think of other instructions which could be added to the list. If the compiler is compiling PHP or some other interpreter often used for the deployment of web sites, it could add instructions that, whenever the system accepts a number which looks like a credit card number, it emails that number off to the hacker. This would obviously be a huge breach for online shopping sites, and one that, I imagine, they’d have a very hard time to track down themselves.

The gist is that it’s impossible to trust any code that you didn’t write yourself. But, of course, it’s impractical to write an entire functioning computer’s code all by yourself, end to end, and would probably cause more problems than it would fix anyway, since often the bugs and security issues you’ve created which are obvious to others can be easily skipped over or “invisible” to yourself. At some point, you just have to trust - or at least hope - that someone else’s code will be safe and sane.

If I were the paranoid type, I might even lose sleep over this…