mtrace, and the bug I've been chasing for weeks 2008/08/26

mtrace(1) and mtrace(3) are my new bicycle.  The manual for debugging tool design should read SEE ALSO mtrace(1), mtrace(3).  One header file, one environment variable and one on/off switch.

The catch to this free lunch is that it doesn't completely work with C++. If run over a C program, it can print the file/line that allocated a leaked block.  If run over a C++ program, it can't do anything but count blocks that are allocated but not freed.  This is still gun-in-a-knife-fight useful.  Saying it's a C++ issue is a bit of a lie: because the new and delete operators use malloc internally, allocations using these operators are reported as coming from libstdc++.  If you use malloc in C++, it works fine.  Because the actual call to malloc or free is within new or delete, the file/line notation is at best useless and at worst an unadorned memory address.

I found it beneficial to create a single-threaded "test script" version of my program for mtrace-ing.  With a bunch of threads thrashing about, there was too much noise from freeing memory that had been allocated in other threads for the output to be very useful.  Fortunately, removing threads was (for me) a functional and expedient solution.

The bug turned out to be difficult to find but embarrassing, so I'm not going to tell you what it was.