If you've done any real programming, sooner or later you're going to find yourself in need of a good debugger. There's only so much you can do by inserting printf() statements to spit out bits of debug text—eventually you have to roll up your sleeves and break out the debugger.
For the x86/Linux world, you could of course use good old gdb, which should be part of almost any Linux distribution. The thing is, gdb is most useful when you know there's a particular bug you're trying to track down. But it's the bugs you don't know about that come back to haunt you, because inevitably it's one of your customers who runs into the bug after you've shipped. That's where something like Julian Seward's Valgrind comes into play.
Read more about the winners of the 2004 Open Source Awards:
- Open Source Awards 2004
- Open Source Awards 2004: VideoLAN
- Open Source Awards 2004: Paul Davis for JACK
- Open Source Awards 2004: Pango
Valgrind is essentially an emulator: It creates an imaginary x86 environment where your program runs, and can report on any x86 activity, such as memory allocation and cache hits. In fact, you can use it in conjunction with gdb—once Valgrind identifies a problem, you can have it attach gdb at that point in your program to figure out exactly what is going on inside your application.
Jeremy Allison, one of the lead developers on Samba and a member of the Open Source Awards nominating committee, calls Valgrind "a pretty amazing piece of work." He speaks from experience because his Samba team used Valgrind to help optimize Samba 3.0, averaging about a 15-percent performance improvement.
In the interest of full disclosure, we should note that CNET uses Valgrind. Ron Rothman, a Senior Software Engineer at CNET, says that "Valgrind on Linux is roughly comparable to Purify on Solaris." High praise indeed given that Purify sells for thousands of dollars per seat license, which puts it out of reach even for many commercial developers.
We recently conducted an e-mail interview with Julian, who lives across the pond in England.
Builder.com: What prompted you to create Valgrind?
Seward: When I was a grad student around 1992, I came across Purify on SPARC-Solaris and thought it was an amazing tool. Time passed, 1996 came and so did cheap x86 boxes running Linux (I remember starting with kernel 1.0.4). gcc, gdb, X—Linux had most stuff for free, but there was no equivalent of Purify. The closest I saw was a system called Checker by Tristan Gingold.
I wound up with a job working on the Glasgow Haskell Compiler (GHC), which is a fabulous compiler for a fabulous programming language </shameless plug>. One of the major parts of GHC I worked on was the back-end x86 and SPARC code generators, and the register allocator. From this I learnt a lot about the x86 instruction set and code generation techniques, and the idea of making a memory-checking tool for Linux came back into view. There followed almost three years of spare-time messing around prior to the first Valgrind release (1.0.0) in June 2002.
Valgrind's design draws inspiration from various sources. In the early '90s, Sun released a dynamic translation-based user-space emulator called Shade, which is where I got the dynamic-translation idea. Some other technical ideas came from GHC. Last but not least, Chris Fraser and David Hanson did a simple, small, elegant, retargetable compiler, lcc and wrote a book about it. Their approach was to shoot for getting 80 percent of the performance for 20 percent of the complexity. I admire the way they have very carefully designed lcc to reduce the implementation burden to something two people could manage. This is a concept I tried to emulate in Valgrind.
Builder.com: What does the name stand for? We thought it might be "value grinder."
Seward: A lot of people think that, reasonably enough, but not so. The name comes from Nordic mythology. Originally (before release) the project was called Heimdall, who was the watchman of the (Nordic) gods. He could "see a hundred miles by day or night, hear the grass growing, see the wool growing on a sheep's back" (etc). This would be an excellent name for a tool like Valgrind. Unfortunately, the name was already taken by a security package "Heimdal" and so I didn't use it.
Eventually my partner Donna Robinson uncovered the Nordic name Valgrind, which is the main entrance to Valhalla. Over it stand three guardians: a wolf, a boar, and an eagle. These guardians will not allow anyone judged unworthy to pass through, which is the part that sold the name to us. Although not as good as Heimdall, the name stuck. Around that time we got a cat and named him Heimdall instead.
Builder.com: Where do you work?
Seward: I'm a compiler-writer at ARM in Cambridge, UK.
Builder.com: I see you've written a number of research papers—could you tell us about your research interests?
Seward: My long-term interests are tools and techniques for making more reliable software. Many programmers seem obsessed with performance, to the detriment of correctness, simplicity, modularity, and maintainability. Valgrind is only useful because C and C++ are such crappy programming languages.
Builder.com: How do those interests tie into your work on Valgrind? Specifically, I'm curious if you've used Haskell to create Valgrind, or is it in some more traditional language like C? Or the other way around—have you used Valgrind in doing any of your Haskell development?
Seward: Unfortunately, Valgrind is entirely in C. Valgrind and the program it is running have to share the same address space, so we cannot use any of the standard libraries—not even glibc—in Valgrind. That rules out more complex languages. We're investigating ways to better insulate Valgrind from the program being run. That work will allow us to write Valgrind in any language we please. I'd like to move to C++, for the usual reasons that people use C++.
It would be great to write at least some of V in Haskell, or another language which is strong at symbolic computation. Currently, though, Haskell implementations cannot compile Haskell code into a library that is sufficiently self-contained for use inside Valgrind. Also, they aren't portable enough: It would require a Haskell compiler on every platform for which we want V to work in the future.
Builder.com: Are you at the point now with Valgrind that you can bootstrap? That is, can you run Valgrind to help debug and analyze Valgrind itself?
Seward: No, not yet. We're moving in that direction, and it's definitely on our long-term roadmap. It should fall out naturally from other structural changes we hope to do. As the thing grows ever bigger, we more and more would like to valgrindify Valgrind, and it's a pain not being able to at present.
Builder.com: What tools do you use when working on Valgrind?
Seward: Well, not much, really. It's all very traditional: emacs + make + gcc and occasionally gdb if it crashes. Although gdb-ingValgrind can be a confusing experience, since you've got a mixture of Valgrind and some other program running at the same time.
Builder.com: Are there methodologies, coding styles, a favorite beverage, anything like that which you find most useful in your day-to-day coding?
Seward: Paranoia, careful design, lots of assertions, internal sanity checks, comment as much as possible. About a year ago Nick Nethercote added a regression test system, and that's been a big help.
Valgrind is loaded with assertion checks and internal sanity checkers which periodically inspect critical data structures. These are permanently enabled. I don't care if 5 percent or even 10 percent of the total run-time is spent in these checks—automated debugging is the way to go. As a result, Valgrind almost never segfaults—instead it emits some kind of a useful error message before dying. That's something I'm rather proud of.
Builder.com: What would you say to an IT manager who's trying to decide whether to use Valgrind versus a commercial product like Purify, or one of the other open-source tools like mpatrol or Electric Fence?
Seward: Well, if your app runs on x86-Linux, you haven't got much to lose trying Valgrind: You don't need to recompile your app, just try it. You can probably establish if V is going to be helpful, or not, inside an hour. And if it is helpful, you may save yourself days of debugging time.
What we see is that most non-trivial C/C++ applications have memory management bugs of some kind. If you haven't Valgrinded your code before, do so now. If it finds nothing, you haven't wasted much time. If it points out problems, you may be able to fix bugs which might otherwise be crashes experienced by your customers.
The OpenOffice/StarOffice developers recently ValgrindedOOo, which is a huge C++ application by anybody's standards. They wrote a short summary of their experiences, worth a read.
Builder.com: Any particular people you'd like to thank or recognize for their contributions to Valgrind?
Seward: Definitely. This is a team effort, and V would be a mere shadow of its current self without the work of others, but particularly that of Nick Nethercote and Jeremy Fitzhardinge, both of whom have put immense amounts of effort into the project.
The KDE development folks endured many pre-release snapshots, and continue to contribute numerous enhancements. KDE 3.0 was the first big project to be Valgrindified. Their feedback and support has been very valuable.
Last but not least, a big thank-you to Donna Robinson for creating an environment in which Valgrind could come into existence.