TCondom - Preventing leaks, memory leaks that is...

By: Louis Louw

Abstract: Finding memory leaks can be very tricky. In this article we are going describe an easy method to find & prevent memory leaks.

TCondom - Preventing leaks, memory leaks that is...

Background:

If you are in the business of writing applications that need to stay up 24/7/365 you have probably stared at your resource meter before, scratching your head, trying to figure out why the heck it keeps using more and more resources...

Yip - memory leaks are ugly, hard to find buggers. Even worse - they are all YOUR fault! But don't despair: Once you get over the denial phase I have found a debugging solution that shaved hours off from the QA phase of our software development cycle.

In this article we are going to cover:

  1. Getting ready to find memory leaks
  2. Detect if you indeed have a memory leak
  3. Detect where exactly that leak is
  4. Detect very dangerous memory overruns & other errors
  5. Anti-leak framework to develop applications that can easily be debugged for leaks and other errors
  6. Using MemProof
  7. Conclusion

But before we start, lets first download the (free) needed tools.

KMM from Kestral Computing
http://www.mindblastsoftware.com/download/memdebug.zip (This is my custom, slightly modified distribution)
Or download from CodeCentral
The original files for KMM can be found here:
http://www.kestral.com.au/devtools/kmm/

MemProof from AutomatedQA
http://www.automatedqa.com/products/memproof.asp

1. Getting ready

First unzip memdebug.zip and copy the files to a directory (c:workdebug in my case).
Now in Delphi go to Tools -> Environment options -> Library and add the above directory to your library path.
Lastly copy the kstmem.dll file to your windows system directory (c:winntsystem32 in my case)

2. Detect if you indeed have a memory leak

We need to quickly setup the project to detect leaks. This is very easy to do.
In Delphi go to Project -> View source
This will bring up the project source file.
- Add the include file {$I DebugDef.inc}
- Then, as the FIRST units in your uses clause, add the kmm and other helper units as shown below.


program MemDebugDemo;

{$I DebugDef.inc}

uses
  {$IFDEF MEMDEBUG}kmm,i_kmm,kmmdefs,{$ENDIF}
  {$IFDEF DEBUG}Debug,{$ENDIF}

  Forms,
  unMain in 'unMain.pas' {Form1};

{$R *.res}

begin
  Application.Initialize;
  Application.CreateForm(TForm1, Form1);
  Application.Run;
end.

Note: Most of the work is done automatically inside the debug.pas unit. Debugging is turned on or off using the defines inside DebugDef.inc. More on this later.

Now detecting memory leaks is a simple 2 step process:

1. Compile your application and run it. Then exit without doing anything. (We do not want any code to execute that can cause leaks)
Open the memdump.txt file (that should be located in the same directory as your application) and write down the value of the "Current Bytes Allocated".

2.  Now delete the memdump.txt file and run the application again. This time putting it through its paces doing your best to run sections of code that you expect might contain some memory leaks. Then exit again.
Open the memdump.txt file and compare the "Current Bytes Allocated" with the previous value. If its the same - then congratulations - you have no memory leaks!
If its bigger than the previous value, well, then you got some leaks to find...

3. Detect where exactly that leak is

Assuming you have not followed the "Anti-leak framework", you can still find the leak by a simple process of divide-and-conquer.

The leak(s) can happen in 3 general places:
a) Resources allocated at startup, and not freed.
b) Resources allocated during the normal run of the program, and not freed.
c) Resources allocated inside threads, and not freed.

Now to find leaks in these sections we need to "name" the thread location where all resources are allocated during the execution of those sections.

This is done by adding the following lines between the begin...end of procedures you expect contains leaks.


procedure TForm1.InitStuff;
begin
{$IFDEF MEMDEBUG}KMSetThreadMarker('Init-B');{$ENDIF}

{Your code that creates objects or allocates resources goes here}

{$IFDEF MEMDEBUG}KMSetThreadMarker('Init-E');{$ENDIF}
end;

Note: You need to do this for all procedures you expect contains leaks. If you are the paranoid kind of person (like me) you will add these lines to every procedure you allocated resources in.
Also - each thread you create should have the same lines in the Execute method of the thread.

Oh - and don't forget to add the following to the units you put the above lines of code:


unit unMain;

{$I DebugDef.inc}

interface

uses
{$IFDEF MEMDEBUG}i_kmm,kmmdefs,{$ENDIF}
{$IFDEF DEBUG}Debug,{$ENDIF}

Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
Dialogs, ExtCtrls;

After doing this you are ready to run your application once more. Put it through its paces again and then exit.

Looking at the memdump.txt file now you will clearly see where the resources that were left behind were created in the first place.
Here is a snippet from the demonstration program.


Memory Dump - Allocated Memory
--------------------------------

Pointer    size series   thread marker  iterate  name            dump
00C2DFF8     56    332 000006AC Timer-B  0000004 ..unnamed..     .............
00C2DE64     56    329 000006AC Init-B   0000002 ..unnamed..     .............
00C2D9BC   1024    328 000006AC Init-B   0000002 ..unnamed..     .............
00C2D910      4    327 000006AC Init-B   0000002 ..unnamed..     ..@0
00C29C58     36    223 000006AC Main     0000001 ..unnamed..     D.B0.000`.@0.

In a simple program like our demo fixing this is now a simple process of looking at the code in these procedures and tracing all allocated resources to where they should have been freed - and after kicking yourself in the buttox for your obvious mistakes - adding that FreeMem or MyObject.Free line...

But alas - life is not that simple, especially on Tuesdays. So chances are that your project creates hundreds of objects and maybe even use 3rd party source that you just assumed had no memory leaks. Luckily for us KMM comes to the rescue again.

Once again, if you have not followed the "Anti-leak framework" from the start, the leak can still be found by "naming" each resource this time.

For each resource you create (You already know the sections where leaks are occurring - so only name the resources created in those sections) you name the resource with the following:


AObject := TObject.Create;
{$IFDEF MEMDEBUG}kmnamepointer(AObject, AObject.ClassName+':'+PointerCounter);{$ENDIF}

Getmem(APointer,1024);
{$IFDEF MEMDEBUG}kmnamepointer(APointer, 'APointer:'+PointerCounter);{$ENDIF}

Running your application once again and then exiting will produce a memdump.txt with the leaked resources clearly named.


Memory Dump - Allocated Memory
--------------------------------

Pointer    size series   thread marker  iterate  name            dump
00C2E460     56    340 00000778 Timer-B  0000006 ..unnamed..     ......
00C2E380     56    339 00000778 Timer-B  0000004 ..unnamed..     ......
00C2E1EC     56    334 00000778 Init-B   0000002 ARecord:3       ......
00C2DBDC   1024    331 00000778 Init-B   0000002 APointer:2      ......
00C2D910      4    327 00000778 Init-B   0000002 TObject:1       ..@0
00C29C58     36    223 00000778 Main     0000001 ..unnamed..     .B0.0
00C28CCC     20    201 00000778 Main     0000001 ..unnamed..     $~B000

For example its clear that the TObject created in InitStuff (named Init-B) was never freed.

Fixing those leaks should be easy from here on...
Note that the leaks inside Timer-B is still not named. Also note that I opted to include a counter when naming objects - this makes it easier to identify different objects of the same class.

Ok..so some of you might have noticed when we named the sections with KMSetThreadMarker we, for example, placed both a KMSetThreadMarker('Init-B') and KMSetThreadMarker('Init-E') marker.
It was done this way since if the leak happened in an unnamed procedure, say SomeProc, the memdump.txt will show a leak in the 'Init-E' thread section. So if you get a leak with a 'xxx-E' marker it means there is still some procedure you have not named,  that contains a leak and thus have to be named.

Also, naming each resource seems like a lot of work. This is where the "Anti-leak framework" steps in. It seeks to provide you with a framework to built reliable applications - FAST.

But before we get to that, lets solve another problem first.

4. Detect very dangerous memory overruns & other errors

I actually started using KMM after about a week of debugging a certain application, after which I still could not figure out why it was crashing at random times. By this time I was almost 100% certain the problem lied in the Delphi compiler code or somewhere else inside Borland code. But of course I was wrong. The trouble was a very small 2 byte memory overrun. What was I thinking anyway - blaming the error on the BEST RAD tool out there? In my defense - a week of debugging can make any programmer crazy...

Getting down to business: Detecting memory errors is easy. In fact, you do not have to do anything. Upon any memory error KMM will not only make an annoying beep sound, but with my modified debug.pas file it will now also log all memory errors to debug.txt automatically.

And finding the error is just as simple. After hearing the beep sound and reviewing the debug.txt file, simply set a breakpoint in the Debug.pas file in the ErrorCallback procedure and try to reproduce the error.
This time of course as soon as the error happens your breakpoint will be triggered, and you can then trace the execution from there to see exactly where the error is occurring.

Lets play a little game of "Who can spot the error?".


procedure TForm1.btOverrunClick(Sender: TObject);
const
  Count = 20;

type
  TMyArray = Array[0..Count-1] of DWord;
  PMyArray = ^TMyArray;

var
  P : PMyArray;
  T : Integer;
begin
  GetMem(P,SizeOf(TMyArray));
  try
    For T := 0 to Count do
      P^[T] := T*T;
  finally
    FreeMem(P,SizeOf(TMyArray));
  end;
end;

Yip - the correct statement should have been "For T := 0 to Count-1 do". Yet, it took me a week to figure out. So simple, yet so elusive. At least with KMM I can now hide my inferior brain-power from the masses.

The above method can of course also be used to solve any of the other errors KMM is able to trace, which includes memory underruns, uninitialized memory and invalid pointers.
You might also need to tweak the kmm.cfg include file to your specific needs.

5. Anti-leak framework

Prevention is better than cure. And in case of illness its always better to have some medicine handy.

So if you know you have to write an application meant to run 24/7/365, its better to code some decent debugging into it right from the start. The following section contains some tips & source code that might help you out. The really good tips are left for last...
Tips 1 to 5 covers memory leak prevention.
Tips 6 to 8 covers memory leak detection.
Tips 9 to 10 covers good programming.

Tip 1:  As soon as you allocate memory for a resource, include a try..finally block and free the resource in the finally section before even writing any other lines of code.

Tip 2: Interfaces are very useful since they are guaranteed to be freed. The following tip was taken from a Borland newsgroup, and full credit goes to the original author - although I can not recall right now who it was.
With this you can eliminate the try..finally block completely and create more readable code. (Although this point is debatable - I sometimes like the try..finally blocks since they provide a clear view of the expected life-time of an object)

This trick involves creating an interface that will automatically free the object assigned to it when the interface is destroyed.
I like to call these "Fire-and-forget" objects and pointers...

Thus:


var
  FList : TStringList;
begin
  {See Condoms.pas for the implementation of AutoFree}
  FList := AutoFree(TStringList.Create).Data; {Fire-and-Forget object}

  FList.Add('1');
  FList.Add('2');
  FList.Add('3');

  {Since the interface will be destroyed here, the FList object will also get freed}
end;

Tip 4: The same trick as above can be used for pointers. I've created a slightly modified version of GetMem called GetMemEx that will automatically free any resources allocated by it.

Example usage:


procedure Foo;
var
  Data : Pointer;
begin
  GetMemEx(Data,SizeOf(TMyRecord)); {Fire-and-Forget Pointer}

  with TMyRecord(Data^) do
  begin
    Name := 'John';
    Age := 32;
    Memo1.Lines.Add(Format('Name: %s, Age: %d',[Name,Age]));
  end;

 {Memory will automatically be freed here}
 {See Condoms.pas for the implementation of GetMemEx}
end;

Tip 5: Tips 1 to 4 is only useful for resources allocated and freed inside the same procedure. And leaks of these kinds are usually pretty easy to find as well.

The really hard to find leaks are those where resources should get freed in some other procedure than the ones where they were created in.

The first tip then in this category is the use of interfaces instead of just pointers. Not only does this provide a nice abstraction to pointers, but you know the memory will get freed when they are no longer in use.

In Condoms.pas we already created a pretty good example interface for you called IBuffer. You can even dynamically change the size of the pointer by simply setting the DataSize property.


uses
  Windows, Messages, ..., Condoms;

TMyObject = class(TObject)
 private
   ...
   FMyData : IBuffer;
   ...
 public
   ...
   procedure DoStuff;
end;

procedure TMyObject.DoStuff;
var
  T : Integer;
begin
  {Create a Fire-and-Forget buffer}
  FMyData := TBuffer.Create(1024);

  {PS: You could also have used this handy function}
  FMyData := GetMemEx(1024);

  {Do something with the pointer}
  For T := 0 to 1023 do
     TMyArray(FMyData.Data^)[T] := Random(100);

{This time the pointer's allocated memory will get freed as soon as this 
 object instance is destroyed, or FMyData is set to nil}
end;

Tip 6: In our divide-and-conquer strategy to solve memory leaks we had to "name" all objects that we suspected were not freed. If your application is complex with a lot of objects this might seem like a lot of work. A handy tip that should be applied from the start of the project is to derive all your custom objects from one of the TCondom classes in Condoms.pas.

The TCondom classes automatically name the objects that are instantiated from them while in debugging mode. This should provide you with an instant view of which classes of objects are actually leaking without you even having to name any sections or objects manually.

See the demo application for more details on the basic use of TCondom classes.

Tip 7: Always name the following sections of your code with KMM's KMSetThreadMarker
a) Procedures that initializes your application.
b) Procedures that is executed on a timely basis, i.e OnTimer events, since they pose the greatest threat for serious memory leaks.
c) The Execute methods of complex threads. (Or even better - use the TCondomThread class)

Tip 8: Interfaces might require a little bit of extra coding, but with SOAP, automatic "garbage collection", modularity and the "plugin" opportunity it offers, programmers should seriously consider basing applications completely on interfaces for maximum reusability and easy debugging.

Tip 9: Not only will applying tips 1 to 8 prevent most memory leaks, but just by using them you will be forced to think about leaks all the time and thus (hopefully) make less mistakes. Try to develop a standard framework that woks for you, and make sure you stick to it in all your applications.

Tip 10: Include all debug code between {$IFDEF} statements. This way you can quickly switch between debug mode and release mode applications. In our example framework a simple 2-line edit to the DebugDef.inc file allows you to switch between different modes.

6. Using MemProof

Before using KMM, MemProof was my knight in shining armor. But since a KMM-enabled application provides you with more than you need to debug your application, why do we still need MemProof?

MemProof is still used in the final QA phase of our testing because:
a) It gives you a good overview of the use of resources during the life of the application. This includes threads, critical sections, live pointers, exceptions, etc.
b) Another useful statistic gathered from MemProof is the peak size of memory used for live pointers as well as both the heap and virtual memory.
c) MemProof is also very useful if your application uses 3rd party DLL's that you suspect might be leaking memory.
d) MemProof can also detect various memory errors.

So in closing I'm going to quickly guide you through the steps of configuring your application so it can be debugged using MemProof. For more details please refer to the MemProof help files and documentation which provides a pretty good resource.

Setting up your project for MemProof:

Since it requires you to make some changes to your project options, I usually make two copies of my project DPR file so I can easily switch between a debug compile and a release compile. For illustration purposes lets assume your project is called MyApp.dpr.

Note: The described setup for MemProof is for Delphi v5 or later. For earlier versions please refer to the MemProof documentation.

  • Load MyApp.dpr and choose File -> Save Project as
  • Then save the project as MyApp_debug.dpr
  • Now go to Project -> Options
  • Under the "Compiler" tab
    Make sure that "Optimization" is unchecked.
    Make sure that "Stack frames" is checked.
    Make sure that "Debug information" is checked.
    Make sure that "Use Debug DCUs" is checked.
  • Then under the "Linker" tab (Still in Project->Options) make sure "Include TD32 debug info" is checked
  • Finally choose Compile -> Build to build the exe.

Launch MemProof and open the EXE with MemProof.

Now you can view all your resources live during the execution of your application - and on exit it will provide you with a complete summary of leaks and other important notices and even provide you with links to the sourcecode where these errors or leaks are occurring.

7. Conclusion

Preventing memory leaks can be done by using a good framework for application development and applying good self dicipline when allocating resources, making sure they will eventually get freed.

In a complex application we tend to make mistakes though, and thus having the above code built right into your application should make finding these bugs farely simple and quick.

A single small leak can bring down a very well written program in no time.

Alas, that is the life of a programmer - for we will never be known for all those millions lines of code we wrote correctly. Nope. We will always be known for that single line of code that annoys the end-user.

At least with KMM you can now listen to "Can you please move that edit box 2 pixels higher?" instead of "Can you make this #@#& application run for more than 4 hours straight?"

On a more serious note - I live in South Africa where AIDS is a serious problem. Luckily for me I have my personality to use as prevention. For the rest of you folks out there, practice good programming - always use a TCondom class before interfacing with any objects.

If you have any comments please feel free to email them to me. I always love to hear from fellow Delphi programmers.

Louis Louw
http://www.mindblastsoftware.com
Delphi & Kylix components


Server Response from: ETNASC04