The New Memory Manager In BDS 2006 - by Pierre le Riche

By: Pierre le Riche

Abstract: The memory manager was updated in Delphi 2006 with a faster and more feature rich version. This article explains the differences between the previous and current memory manager and how to leverage the new functionality.

Table of Contents

Introduction

New Features

Large Address Space Support

Improved Block Alignment

Intelligent Reallocations

Improved Locking Granularity

Improved Fragmentation Behaviour

Simplified Memory Manager Sharing Mechanism

Memory Leak Reporting

Extended State Reporting

Specifying Minimum Block Alignment

Enabling An Address Space > 2GB

A Supported Operating System

Appropriate Linker Directive

Third Party Component And Library Support

Reporting Memory Leaks

Reporting Memory Manager State

GetMemoryManagerState

GetMemoryMap

Sharing Memory Managers Between An Application And Libraries

AttemptToUseSharedMemoryManager

ShareMemoryManager

Using Replacement Memory Managers

AllocMem

RegisterExpectedMemoryLeak

UnregisterExpectedMemoryLeak

Implementation Details

Changed Behaviour

Double-free bugs

Using memory blocks after they have been freed

Memory overwrite bugs

    Introduction

The Win32 memory manager was replaced in Delphi 2006 with one that is faster and more feature rich. This article is a summary of the new features and how to use them. It explains how the new memory manager was implemented, and how this new design affects the behaviour of the memory manager and consequently applications compiled with Delphi 2006.

    New Features

    Large Address Space Support

The memory manager now supports an address space larger than 2GB (up to 4GB).

    Improved Block Alignment

Blocks returned by the memory manager are always aligned on at least 8-byte boundaries. 16-byte alignment can be forced by executing the following code: SetMinimumBlockAlignment(mba16Byte);

    Intelligent Reallocations

Reallocations are managed intelligently. The new memory manager tries to avoid moving memory blocks around unless the benefits outweigh the cost of the move operation.

    Improved Locking Granularity

The locking granularity is smaller compared to the previous memory manager, so memory manager intensive multi-threaded applications should see a significant performance improvement.

    Improved Fragmentation Behaviour

Due to improved internal management structures, address space fragmentation should be less of a problem than with the previous memory manager.

    Simplified Memory Manager Sharing Mechanism

A new memory manager sharing mechanism is introduced that avoids the need for the borlndmm.dll library, although the old method is still supported. (See the SimpleShareMem.pas unit for details.)

    Memory Leak Reporting

Allows applications to register (and unregister) expected memory leaks, and can optionally report unexpected memory leaks on program shutdown.

    Extended State Reporting

The new memory manager has extended state reporting functions that allow the tracking of address space usage per block size category, as well as the efficiency of the memory manager. (Efficiency is measured as the percentage of allocated address space that is wasted due to block management structures and fragmentation.)

    Specifying Minimum Block Alignment

The minimum alignment of blocks returned by the memory manager is 8 bytes. In the current implementation all requests for blocks > 148 bytes will always be at least 16 byte aligned. To enforce 16 byte alignment for all new allocations, execute SetMinimumBlockAlignment(mba16Byte). To restore the default of 8 byte alignment, execute SetMinimumBlockAlignment(mba8Byte).

    Enabling An Address Space > 2GB

There are three prerequisites for using an address space larger than 2GB:

    A Supported Operating System

A version of Windows that supports a user mode address space larger than 2GB is required. This is currently either Windows x64 edition, or 32-bit Windows XP/2003 with the /3GB option set in the boot.ini file.

    Appropriate Linker Directive

The IMAGE_FILE_LARGE_ADDRESS_AWARE flag must be set in the header of the EXE to inform Windows that the application supports an address space larger than 2GB. This flag can be set by specifying the {$SetPEFlags IMAGE_FILE_LARGE_ADDRESS_AWARE} directive in the project’s .dpr file.

    Third Party Component And Library Support

All libraries and third party components must support the larger address space. With a 2GB address space the high bit of pointers are always 0, so a larger address space may expose pointer arithmetic bugs that did not previously show any symptoms. Such bugs are typically caused by typecasting pointers to integers instead of cardinals when doing pointer arithmetic and/or comparisons.

    Reporting Memory Leaks

The new memory manager has the ability to report all memory blocks that have not been freed before application shutdown, i.e. memory leaks. There may be instances in which a memory block is leaked on purpose and the reporting of that particular leak should be suppressed. For that purpose there is a function called RegisterExpectedMemoryLeak that takes as parameter the pointer to the leaked block. Leaks registered with RegisterExpectedMemoryLeak will be excluded from the memory leak report.

The memory leak report is disabled by default, but can be enabled by setting the global variable ReportMemoryLeaksOnShutdown to “true”. If there are no unexpected memory leaks the messagebox will not be shown, even if ReportMemoryLeaksOnShutdown is “true”. A useful code snippit that can be used to enable memory leak reporting only when the application is run inside the debugger is “ReportMemoryLeaksOnShutdown := DebugHook <> 0”.

Note that the leak registration database is limited to 16K entries, so it is possible that RegisterExpectedMemoryLeak can fail and return false. Call UnregisterExpectedMemoryLeak to remove entries from the database.

Below is an example of a memory leak report (displayed on application shutdown):

Hide image

And here is the code that caused it:

Hide image

    Reporting Memory Manager State

The new memory manager is fully backward compatible and thus supports the GetHeapStatus call. There are two new state reporting functions that provide more detail:

    GetMemoryManagerState

Returns a list of all the internal block sizes and the number of blocks allocated for each. Also returns the amount of address space reserved for each internal block size, so the block overhead and degree of memory pool fragmentation can be measured.

    GetMemoryMap

Returns a list of all the 64K chunks in the address space and what they are currently used for, i.e. unused, allocated or reserved by the operating system, or used by the Delphi memory manager.

    Sharing Memory Managers Between An Application And Libraries

In the past the preferred method of sharing the memory manager between an application and its libraries was through the ShareMem.pas unit and the borlndmm.dll library. This method is still supported, and both older borlndmm.dll libraries and the new higher performance borlndmm.dll can be used.

The new memory manager adds an alternate sharing option that does not require the use of an external library. This method is managed through two function calls:

    AttemptToUseSharedMemoryManager

Searches the current process for a shared memory manager. If no memory has been allocated using the default memory manager the module will switch to using the shared memory manager instead. Returns true if another memory manager was found and the module successfully switched to it.

    ShareMemoryManager

Makes the module’s memory manager available for sharing to other modules in the current process. Only one memory manager may be shared per process, so this function may fail (and return false).

The new SimpleShareMem.pas unit implements this new sharing mechanism, and is a drop-in replacement for ShareMem.pas.

    Using Replacement Memory Managers

As in previous versions, the memory manager can be replaced with a third party memory manager. To enable third party memory managers to also implement the new functionality supported by this release, the SetMemoryManager procedure has been overloaded to also accept the new TMemoryManagerEx record. The TMemoryManagerEx record extends the old TMemoryManager record by adding the following three fields:

    AllocMem

This should point to a function that returns a zero-filled block of the requested size.

    RegisterExpectedMemoryLeak

This should point to a function used to register an expected memory leak. If memory leak checking and reporting is not implemented in the replacement memory manager, this can point to a function that simply returns “False”.

    UnregisterExpectedMemoryLeak

This should point to a function used to deregister a previously registered expected memory leak. If memory leak checking and reporting is not implemented in the replacement memory manager, this can point to a function that simply returns “False”.

For older memory managers that install themselves using the old TMemoryManager record, a default AllocMem, RegisterExpectedMemoryLeak and UnregisterExpectedMemoryLeak handler will be installed. The default AllocMem handler simply calls GetMem and then clears the memory block, while the default RegisterExpectedMemoryLeak and UnregisterExpectedMemoryLeak do nothing and return “false”.

    Implementation Details

The new memory manager is actually three memory managers in one: small (<2.5K), medium (< 260K) and large (> 260K) blocks are managed separately.

Requests for large blocks are passed through to the operating system (VirtualAlloc) to be allocated from the top of the address space. (Medium and small blocks are allocated from the bottom of the address space - keeping them separate improves fragmentation behaviour).

The medium block manager obtains memory from the OS in 1.25MB chunks. These chunks are called "medium block pools" and are subdivided into medium blocks as the application requests them. Unused medium blocks are kept in double-linked lists. There are 1024 such lists, and since the medium block granularity is 256 bytes that means there is a bin for every possible medium block size. The MM maintains a two-level "bitmap" of these lists, so there is never any need to step through them to find a suitable unused block - a few bitwise operations on the "bitmaps" is all that is required. Whenever a medium block is freed, the MM checks the neighbouring blocks to determine whether they are unused and can thus be combined with the block that is being freed. (There may never be two neighbouring medium blocks that are both unused.) The MM has no background "clean-up" thread, so everything must be done as part of the freemem/getmem/reallocmem call.

In an object oriented programming language like Delphi, the majority of memory allocations and frees are usually for small objects. In practical tests with various Delphi applications it was found that, on average, over 99% of all memory operations involve blocks <2K. It thus makes sense to optimize specifically for these small blocks. Small blocks are allocated from "small block pools". Small block pools are actually medium blocks that are subdivided into equal sized small blocks. Since a particular small block pool contains only equal sized blocks, and adjacent free small blocks are never combined, it allows the small block allocator to be greatly simplified and thus much faster. The MM maintains a double-linked list of pools with available blocks for every small block size, so finding an available block for the requested size when servicing a GetMem request is very speedy.

Moving data around in memory is typically a very expensive operation. Consequently, the MM has an intelligent reallocation algorithm to avoid moving memory as much as possible. When a block is upsized it adjusts the block size in anticipation of future upsizes, thus improving the odds that the next reallocation can be done in place. When a pointer is resized to a smaller size, the MM requires the new size to be significantly smaller than the old size otherwise the block will not be moved. Blocks are also never shrunk below a certain minimum level, currently approximately 64 bytes. This typically speeds up operations involving small blocks (common when manipulating long strings).

Speed is further improved by an improved locking mechanism: Every small block size, the medium blocks and large blocks are locked individually. If, when servicing a GetMem request, the optimal block type is locked by another thread, then the MM will try up to three larger block sizes. This design drastically reduces the number of thread contentions and improves performance for multi-threaded applications.

    Changed Behaviour

Note that the new memory manager may expose bugs that were previously not showing any symptoms (or delayed symptoms). Bugs that may have different symptoms with the new memory manager are:

    Double-free bugs

These are bugs where the same pointer is freed more than once. The new memory manager has a flag in the header of every block that is used to indicate whether the block is in use or not. The new memory manager will consequently throw an “Invalid Pointer Operation” exception if an attempt is made to free a block that has already been freed. The previous memory manager did not consistently raise this error.

    Using memory blocks after they have been freed

These kinds of bugs typically occur when an object is referenced after it was freed. The new memory manager reuses freed blocks more aggressively than the old memory manager, since this makes better use of the CPU cache. Freed blocks are thus often reused sooner than with the old memory manager, possibly causing bugs of this kind to come to the fore where they were previously hidden.

    Memory overwrite bugs

Both the new and old memory managers employ a 32-bit header in front of every block that is used to manage the heap. If this header is corrupted by the application, it usually results in a crash. Since blocks are more frequently reused with the new memory manager, a crash may occur sooner than with the old memory manager.

From the above notes it should be clear that the new memory manager is less tolerant of heap corruption bugs caused by the application. The upside to this is that this should make it easier to catch bugs during the development cycle. The downside being that it could make life more frustrating for customers having to use an application containing such bugs.

The old memory manager that was used by Delphi 2005 (and earlier Delphi versions) is available on CodeCentral in the form of a replacement memory manager module. This module may be useful in comparing the performance of the old and new memory managers, and can also be used in cases where the behaviour of the old memory manager is desired above the behavior of the new one.

Server Response from: ETNASC03