New information has been added to this article since publication.
Refer to the Editor's Update below.

Visual C++ 6.0

Don't Let Memory Allocation Failures Crash Your Legacy STL Application

James Hebben

Code download available at:LegacySTLFix.exe(111 KB)

This article assumes you're familiar with C++, STL, and MFC

Level of Difficulty123

SUMMARY

Most C++ developers make extensive use of the Standard Template Library (STL) in their code. If you are one of them and are using STL and Visual C++ 6.0 directly out of the box, your application is at high risk of crashing under low memory conditions. The problem arises because checking for failure of operator new is such an uncommon practice. To make things worse, when new does fail, the response is not standard. Some language compilers return NULL while others throw an exception.

In addition, if you are using STL in an MFC project, be aware that MFC has its own set of rules. This article discusses these problems, explains how the default behavior has changed in Visual C++ .NET 2003, and outlines the changes you must make if you're using Visual C++ 6.0 so that you can safely use the STL when operator new fails.

Contents

Background
Operator New Returns NULL
Standard Template Library
Visual C++ 6.0 and Operator New
Fixing Operator New
When Operator new(std::nothrow) Throws an Exception Anyway
Conclusion

How many developers check for the failure of operator new? Is there a need to always check for failure? I have seen large and complex C++ projects, written using Visual C++® 6.0, where not a single check in the entire code base is made to see if new returns NULL. Note that I said checking for new returning NULL. The default behavior of operator new in all versions of Visual C++ through version 6.0 is to return NULL rather than throw an exception when it fails. (See Knowledge Base article 167733 for more information, but do not implement the solution given there. I will explain why you shouldn't later in this article.)

The default behavior has changed for Visual C++ .NET, both for version 7.0 (Visual C++ .NET 2002) and 7.1 (Visual C++ .NET 2003), which throws an exception when operator new fails. While this new behavior under the Microsoft® .NET Framework adheres to the C++ standard and should be welcomed, be aware that it can break the runtime behavior of all migrated Visual C++ 6.0-style code that does not anticipate operator new throwing an exception. If you are developing with Visual C++ .NET, you will find that the issues raised here have been fixed. If you're not yet using one of the .NET Framework versions, this article looks at the serious implications and incompatibilities of operator new returning NULL, a problem that applies to all versions of the Visual C++ compiler up to and including version 6.0.

[Editor's Update - 2/28/2005: In Visual C++ .NET 2002 and Visual C++ .NET 2003, whether operator new throws an exception is decided dynamically by the linker based on which order libraries are passed to it. If a project includes only C++ headers and definitions of operator new, the linker will use an operator new that throws exceptions. Conversely, if a project only uses C headers and definitions of new, the linker will select to use a non-throwing new. However, if a project is mixed, the version of new that is used depends on which the linker comes across first. To override this behavior and to force the use of throwing new, explicitly link to thrownew.obj. Note that in Visual C++ 2005, new always throws unless you explicitly link to nothrownew.obj. Second, note that none of the behaviours described relate to managed code or the .NET Framework. These behaviours are those of native C++ code compiled with Visual C++ .NET 2002 and Visual C++ .NET 2003.]

Background

When Microsoft released the first version of its Visual C++ compiler, its primary role was to support the MFC framework. For all practical purposes, Visual C++ and MFC were seen as one product. Over the years, both MFC and the Visual C++ compiler have matured. At the same time, the Visual C++ compiler has become a product in its own right, not necessarily dependent on MFC and supporting other technologies such as the Active Template Library (ATL), the Standard Template Library (STL), and a host of various other technologies. Today, MFC is just one of the many libraries that the Visual C++ compiler supports. As a result of this, it is now quite common to come across development projects that use Visual C++ without MFC.

I started writing this article when I observed unusual behavior with my STL code after operator new failed. To my surprise, I realized that Visual C++ 6.0 (and all prior versions that support STL) was incompatible with STL when operator new fails. The project I was working on did not use MFC, so my observations were based solely on non-MFC code. When I started looking at MFC-based examples, I discovered that MFC defines a very different behavior for operator new. Before diving into this article, I want to summarize the behaviors of operator new when a memory allocation fails. Just for good measure, I will describe the behavior under Visual C++ .NET since it's different from previous versions.

The C++ standard states that operator new should throw an exception on failure. Specifically, the exception thrown should be std::bad_alloc. That may be the standard, but the behavior under Visual C++ 6.0 depends on how you use it and what version you are using. Figure 1 shows the Visual C++ behavior of operator new when a memory allocation fails.

Figure 1 Behavior of Operator New on Failure

As you can see, only non-MFC code under Visual C++ .NET adheres to the standard. If you are using MFC then new will throw an exception, but its type is incorrect. If the implementation of the STL that you are using contains catch (std::bad_alloc) statements to handle failed memory conditions then the only combination that will work out of the box is Visual C++ .NET without MFC. The STL implementation that ships with Visual C++ 6.0 uses catch(...) to handle operator new failures, so if you are using MFC, the STL implementation that ships with Visual C++ 6.0 will operate correctly when operator new fails.

Given that the operator new implementation provided with MFC throws an exception (CMemoryException) and also that under Visual C++ .NET the non-MFC operator new throws an exception (std::bad::alloc), I will assume that for all practical purposes, these cases do not pose a problem. What then of the common scenario of a Visual C++ 6.0-based project that does not use MFC but does make use of the STL? This is what I will focus on for the remainder of this article.

Operator New Returns NULL

Back to the question at the beginning of this article, there are generally one of two reasons given for not checking the pointer value returned from operator new for NULL: either operator new will never fail or operator new throws an exception.

Even if you think operator new will never fail, it is poor coding practice to not check its return value. While a desktop application is less likely to suffer from out-of-memory conditions, a user pressing F9 on their 100MB Excel spreadsheet may well cause your application to run out of memory. As for server-based applications that are expected to be up running and processing 24 hours a day, out-of-memory conditions are much more likely, especially on a shared application server. The chances of a memory failure are increased if you can't guarantee that your application does not leak even a single byte periodically. How many applications, especially those that are developed in-house, can deliver that kind of guarantee?

If your reason for not checking the returned pointer value for NULL is that operator new throws an exception, you can be forgiven. After all, the C++ standard stipulates that new should throw an exception on failure. This is not the default implementation for all versions of Visual C++ through 6.0 (when MFC is not used), which return NULL on failure. While this is fixed in Visual C++ .NET, the prior implementation can cause problems, notably with the STL. The implementation of the STL assumes that new will throw an exception on failure, regardless of the compiler that's used. In fact, if new does not exhibit this behavior and a memory allocation fails, returning NULL, the STL behavior will be undefined and will, in all likelihood, cause your application to crash. I will show a concrete example of this shortly.

Standard Template Library

More and more developers rely on the STL for their C++ development. The STL provides a wealth of classes and functions based on C++ templates. There are several benefits to using the STL in your application. First, the library presents a consistent interface for numerous general-purpose tasks. The second benefit is that the code has been extensively tested and can be assumed to be reasonably free from bugs. Finally, the algorithms are considered best-of-breed.

In order for the STL to work, the hosting compiler must implement the C++ standards that the STL was written to. The Visual C++ compiler comes prepackaged with an STL implementation, and a variety of other vendors' implementations can also be used.

If you are using the Visual C++ compiler (up to version 6.0), the STL will work as anticipated with one notable difference—low memory scenarios that cause operator new to fail.

Visual C++ 6.0 and Operator New

When operator new returns NULL on failure, it can be considered a bug because it is contrary to the standard which states that operator new should throw an exception on failure. All STL implementations that I am aware of, including the version that ships with Visual C++, anticipate that operator new will throw an exception on failure.

While it is possible to alter the behavior of operator new to throw an exception on failure, which I will demonstrate later, this itself results in a further anomaly. First I will show what happens when operator new fails in a default environment. All of my testing was carried out using Visual C++ 6.0, SP4 and SP5 running under Windows NT® 4.0 Service Pack 6a and Windows® 2000 SP2. To the best of my knowledge, this behavior affects code built with any version of the Visual C++ compiler, up to and including version 6.0 SP5, on all operating systems. The same code has also been tested using Visual C++ versions 7.0 and 7.1 and both of these versions show standard-compliant behavior when operator new fails. The STL libraries used were the version that comes with Visual C++ and the STLPort (https://www.stlport.org) implementation.

For the STL walkthrough, I will use what is probably the most common STL class, std::string, although the behavior described applies to any STL function or class that allocates heap memory through operator new. In this example, let's assume that I'm attempting to construct a new string object with some data and that the heap allocation will fail. The following code snippet will suffice:

#include <string> void Foo() { std::string str("A very big string"); }

The code in Figure 2 is taken from the STL string class code that ships with Visual C++ 6.0. The constructor eventually lands in std::basic_string<>::_Copy. An extract of this is shown in Figure 2, with some code removed. In the try block, the local variable _S is assigned the return from allocator.allocate that, in turn, makes a call to operator new. With the default Visual C++ 6.0 behavior, operator new will return a NULL value on failure, resulting in a NULL value being assigned to the local _S variable. Critically, no exception will be thrown.

Figure 2 STL basic_string Code

void _Copy(size_type _N) { // Code omitted _E *_S; _TRY_BEGIN _S = allocator.allocate(_Ns + 2, (void *)0); _CATCH_ALL _Ns = _N; _S = allocator.allocate(_Ns + 2, (void *)0); _CATCH_END // Code omitted _Ptr = _S + 1; // ACCESS VIOLATION _Refcnt(_Ptr) = 0; // Code omitted }

The next line of code to be executed is the assignment of _S + 1 to the member variable _Ptr. As the value of _S is NULL, the value 0x00000001 is assigned to the _Ptr variable. The following line, _Refcnt(_Ptr) = 0, effectively returns _Ptr - 1 (actually _Ptr[-1]), which evaluates against the original NULL pointer value returned from operator new. The _Refcnt member function returns a reference to the first element of a NULL pointer and the subsequent assignment of 0 to that reference (essentially *NULL= 0) will fail with an immediate access violation. Beyond the rather draconian measure of installing a structured exception handler to catch this error, there is no way of preventing this access violation from terminating your application. Although this behavior might seem like it's the result of a bug, the STL code is in fact correct, but in order to get this correct behavior it requires that operator new throw an exception on failure.

Now let's take a look at the flow of execution when operator new throws an exception on failure. As shown before, the initial call to allocator.allocate will be executed. When operator new fails, a std::bad_alloc exception is thrown and the code lands in the catch(...) (_CATCH_ALL) handler where the allocation is retried, possibly with a smaller request. If this second allocation fails, a further std::bad_alloc exception is thrown and this will propagate all the way back to your code, leaving the std::string object in a defined, empty state.

You should be aware that this overloaded string constructor could throw an exception. If you build classes that have std::string member variables whose overloaded constructor is called in the initialize section of your constructor, you should carefully read "Handling Exceptions, Part 10" of Robert Schmidt's Deep C++ column.

Fixing Operator New

Earlier, I stated that the implementation given in Knowledge Base article 167733 for fixing operator new to throw an exception should not be followed. The article presents two code samples, shown in Figure 3 and Figure 4. The first sample correctly installs a new_handler (for operator new failures), which I will refine to present an automatically installing handler. The second sample is why I gave the warning. This sample installs a new_handler and calls _set_new_mode(1) to indicate that malloc should throw an exception on failure.

Figure 4 KB Article 167733 Sample #2

#include <new> #include <new.h> #pragma init_seg(lib) int my_new_handler(size_t) { throw std::bad_alloc(); return 0; } struct my_new_handler_obj{ _PNH _old_new_handler; int _old_new_mode; _tag_g_new_handler_obj() { _old_new_mode = _set_new_mode(1); _old_new_handler = _set_new_handler(my_new_handler); } ~_tag_g_new_handler_obj() { _set_new_handler(_old_new_handler); _set_new_mode(_old_new_mode); } } _g_new_handler_obj;

Figure 3 KB Article 167733 Sample #1

#include <new> #include <new.h> int my_new_handler(size_t) { throw std::bad_alloc(); return 0; } int main () { _PNH _old_new_handler; _old_new_handler = _set_new_handler(my_new_handler); /* ... application processing ... */ _set_new_handler(_old_new_handler); return 0; }

If you set malloc to throw an exception on failure, all code that uses malloc may be somewhat surprised at this behavior. Operator new(std::nothrow) is also implemented in terms of malloc (for debug builds at least) and this change will cause operator new(std::nothrow) to throw an exception on failure. This is definitely not the behavior that you want.

Figure 5 shows the code for a new handler that is installed automatically, which is included in the NewHandler.cpp source file available with this article (see the link at the top of this article). It is essentially the same as the second sample code listing from the Knowledge Base article (shown in Figure 4), with the syntax fixed and the _set_new_mode call removed. You can use this code by adding the NewHandler.cpp file to your project.

Figure 5 Self-installing new_handler

#include <new> #include <new.h> // 4073: initializers put in library initialization area #pragma warning(disable: 4073) #pragma init_seg(lib) namespace { int new_handler(size_t) { throw std::bad_alloc(); return 0; } class NewHandler { public: NewHandler() { m_old_new_handler = _set_new_handler(new_handler); } ~NewHandler() { _set_new_handler(m_old_new_handler); } private: _PNH m_old_new_handler; } g_NewHandler; } // namespace

The test harness for this handler is trivial, as Figure 6 shows. The allocation in the test code should result in immediate failure on most computers. The sample code results in an invalid heap allocation size, leading to an immediate failure without any attempt to actually perform the allocation. Unfortunately, if you continuously allocate large blocks of memory until an actual heap allocation failure occurs, you will not be able to debug the code because the Visual C++ IDE invariably crashes under low memory conditions. To build the test, download the sample files. Open the Testnew_throw.cpp file in Visual C++ 6.0, and select Build from the Build menu. Accept the prompt to create a default workspace. If you step through the code you can verify that operator new now throws an exception of type std::bad_alloc on failure.

Figure 6 New Throws std::bad_alloc

// Testnew_throw.cpp Test new throws std::bad_alloc on failure #pragma warning(disable: 4307) #include "../NewHandler.cpp" #include <iostream> struct Big{ double m_d[99999]; }; void main() { try { // Force failure Big *p= new Big[99999]; } catch(std::bad_alloc& er) { std::cout << er.what() << std::endl; } }

When Operator new(std::nothrow) Throws an Exception Anyway

Toward the end of the Knowledge Base article, a note about operator new(std::nothrow) and Visual C++ 5.0 indicates that new(std::nothrow) will throw an exception if a new handler is installed. This problem still exists under Visual C++ 6.0, but the behavior is subtler. Using Visual C++ 6.0, operator new(std::nothrow) will behave as expected and return NULL on failure when linking against the Debug runtime libraries only. If you link against the Release version of the runtime library, then operator new(std::nothrow) will throw an exception anyway. This is certainly not the behavior you want in your applications. The test harness for operator new(std::nothrow) is trivial, but demonstrating the full behavior is not so straightforward due to a further quirk, this time with the optimizing compiler. The crucial part of this test is that the only code in the try block is the call to operator new(std::nothrow), shown in Figure 7.

Figure 7 new(std::nothrow) Throws bad_alloc

// Testnew_nothrow.cpp Test new(std::nothrow) returns NULL on failure #pragma warning(disable: 4307) #include "../NewHandler.cpp" #include <iostream> struct Big{ double m_d[99999]; }; void main() { Big *p; try { // Force failure p= new(std::nothrow) Big[99999]; } catch(std::bad_alloc& er) { std::cout << "Error " << er.what() << std::endl; return; } std::cout << "p= " << p << "\n"; }

Build the Win32® Debug configuration of this test, using the same technique as for Testnew_throw.cpp. Running the resulting executable yields the following expected result:

p= 00000000

Now build the Win32 Release configuration and run the resulting executable. The output may surprise you:

abnormal program termination

One thing is certain here: operator new(std::nothrow) is definitely not returning NULL. Exactly what is happening is not so clear. Try moving this line inside the try block:

std::cout << "p= " << p << "\n";

The results change:

Error bad allocation

Now the catch handler is called, confirming the release build behavior I alluded to earlier. The question still remains: why did I previously get an abnormal program termination? To see exactly what is going on here, first revert back to the original code shown in Figure 7 (with only the call to operator new(std::nothrow) in the try block). Next, change the project settings for the Win32 Release configuration to disable compiler optimizations (Project | Settings | C/C++ | General | Optimizations = Disable (Debug)). Build and run the executable. The program's output is expected, though still not quite what you're after:

Error bad allocation

This behavior is due to the optimizing compiler, which is in fact generating valid compiled code. The documentation for Visual C++ 6.0 states that while the compiler supports the syntax for exception specifications, it ignores them. As it turns out, this is not strictly true. To see exactly what is going on, you have to delve into the generated assembly code (very lightly). Retaining the disabled optimizations and ensuring that the only line of code in the try block is the call to operator new(std::nothrow), turn on mixed assembly code (Project | Settings | C/C++ | Listing Files | Listing file type = Assembly with Source Code). Build the executable and open the generated file Testnew_nothrow.asm that is in the Release folder. In this file, search for the string "try", ensuring you have "Find whole word only" checked to avoid partial matches with other instances. You should be able to see the mixed source/assembly for the try block, including a reference to a variable called __$EHRec$. This is part of the generated code for the try/catch exception mechanism.

Next, turn optimizations back on, rebuild, and locate the "try" source line in the Testnew_nothrow.asm file. The reference to the __$EHRec$ variable is gone. What has happened is that the optimizing compiler has detected that operator new(std::nothrow) is declared as throwing no exceptions and has correctly concluded that the entire try/catch block is redundant code. The result is that the entire try/catch block has been optimized away, leaving the code that was wrapped by the try block to run without the unnecessary exception handling support. While this is technically correct, you are left with the discrepancy of the compiler subsequently allowing an exception to be thrown from a non-throwing function, which may be impossible to catch.

Having discovered exactly what is going on with operator new(std::nothrow), I now present the fix to this problem in the form of a user-supplied version of operator new(std::nothrow), taken from Knowledge Base article 167733, which is included in the NewNoThrow.cpp file source file available with the code download for this article. This version correctly returns NULL on failure:

void *__cdecl operator new(size_t cb, const std::nothrow_t&) throw() { char *p; try { p = new char[cb]; } catch (std::bad_alloc) { p = 0; } return p; }

This requires a word of warning, however. If you are linking against a DLL version of the runtime library (Debug Multithreaded DLL or Multithreaded DLL) then the new_handler is effectively installed into the runtime DLL. This means that any DLL file that loads into your process address space that also links against a matching version of the runtime library DLL will be affected by this handler (new will throw an exception on failure). The implication of this depends wholly on whether the client DLL anticipated new returning NULL or assumed that it would throw an exception (ATL supports both modes).

The fix for operator new(std:nothrow) is only required if new is changed to throw an exception, a change that is mandatory for using STL correctly. However, the fix is local to the project into which the source file is inserted. In this case, any third-party DLL that uses operator new(std::nothrow) and builds against a compatible release runtime DLL as I've shown will now be at the peril of new(std::nothrow), throwing an exception on failure. This happens because the global scope of the new handler is mismatched with the local scope of the replacement operator new(std::nothrow). The only practical solution is to either link against one of the static runtime libraries or verify that third-party code does not call operator new(std::nothrow) or does not link to the DLL version of the runtime library. If there is a possibly redeeming feature to this unfortunate state of affairs, it is that operator new(std::nothrow) is a rarely used overload.

The final gotcha is that the STL provided with Visual C++ 6.0 does in fact use this overload in a single place, namely the get_temporary_buffer template function. It is quite unlikely that your code calls this function directly. However, a search through the STL source code shows that the internal _Temp_iterator template class calls the get_temporary_buffer template function. The _Temp_iterator template class itself is called, indirectly, by the following public functions defined in algorithm: stable_partition, stable_sort, and inplace_merge. The behavior of these functions is undefined if operator new(std::nothrow) does throw an exception. If you are using these functions and you have the new handler installed, consider using a different implementation of the STL, many of which are available. I successfully use STLPort (https://www.stlport.org) in my work. To the best of my knowledge, this implementation does not call operator new(std::nothrow) anywhere in its implementation.

Conclusion

I have demonstrated that if you're using the STL with Visual C++ up to version 6.0 in a non-MFC project, its out-of-the-box behavior can cause the STL to crash your application under low memory conditions. The version of operator new provided with Visual C++ 6.0 is not compatible with the STL. Even with the fixes provided here, it is still possible that trouble may ensue with third-party code or with a few functions in the version of the STL that's supplied with Visual C++ 6.0. The current mismatch in Visual C++ 6.0 between operator new, operator new(std::nothrow), and the STL cannot be completely fixed. However, if you are using STL in your code and you do not include the fixes I've recommended in this article, you run a real risk of your application crashing in STL code under low memory conditions.

For MFC-based projects, whether or not STL will survive a failure within operator new depends wholly on how the STL implementation that you use handles exceptions through this operator. Most implementations seem to use catch(...) rather than catch(std::bad_alloc) when handling failed allocations, but this is not a requirement.

Finally, as I stated at the beginning of this article, both currently available versions of Visual C++ .NET have fixed all of the problems I have described, with the exception of the MFC behavior.

For related articles see:
Handling Exceptions in C and C++, Part 1
Visual C++ Developer Center
https://www.stlport.org

James Hebben is a UK-based consultant specializing in Visual C++ financial and trading systems. He can be reached at James.hebben@extraview.uk.com.