Basic truths

The most basic truth we need to learn is that new, that is, the dynamic allocation, is bad because it is expensive! And that is because of two reasons—first, the allocation operation in itself is quite expensive, and second, it results in bad data locality. As we learned in Chapter 1, Understanding Performant Programs, bad data locality results in pipeline stalls and cache misses, ruining the smooth instruction flow in the processor.

Obtaining a fresh memory form the memory allocator is a costly operation because the system allocator must fulfill many different requirements at once and, to achieve that, it has to observe several trade-offs. First of all, it has to do the following:

• It has to support multithreading, so it has to have some form of synchronization.
  In the older days, that was a major problem, as the allocator had simply a global lock and hence a single point of contention, but modern allocators use some form of per-thread memory pools. Nonetheless, there is a cost to pay for the remaining synchronization when these pools become exhausted.
• It has to support efficient allocations of many same-size objects, many variable-size objects, some very big objects, and some very small objects, and do all of that fast!
• It shouldn't deteriorate over time, waste memory, or fragment the memory.
• Sometimes, it has to call the kernel to give us additional memory. (This one can take quite a long time!)

We see that memory managers have their work cut out for them, so it's not a surprise that the overhead of a memory allocation is rather heavy!

Technically, in C++, we can replace the default memory manager (which is the malloc() function) both globally (that is, for the entire program) or only for a single class. We are doing that either by overriding the global new() operator or the new operator for a specific class as shown in the following:

// override global memory handling
void* operator new(size_t size) { ... }
void operator delete(void* p) { ... }
void* operator new[](size_t size) { ... }
void operator delete[](void* p) { ... }

// or only for a class
class Person
{
public:
  void* operator new(size_t size) { ... }
  void operator delete(void* p) { .. }
  // etc...

Technical note: The preceding example shows C++11 code. C++17 adds overloads to specify alignment. C++98 used the now deprecated throw() specifier.

So, we can do that, but do we want to do it? Let's have a look at that.