Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I know that in C it is relatively easy to define your own memory management functions (I have written my own, complete with very primitive garbage collector, in an attempt to track down memory leaks before I had heard about Valgrind).

I believe Python essentially does this under the hood by allocating buffers and managing its own memory requests on them (allocating more when it needs more, obviously).

I wonder if it is possible to do this kind of thing in C++? It occurs to me that it might be hard to write platform-independent code which mimics the behaviour of the `new` keyword on a preallocated buffer. A quick Google reveals some syntax[1] that I'd not seen before which allows one to do this, although it looks like your call would look something like:

    MyType* instance = new (my_malloc(sizeof(MyType))) MyType();
which is hardly concise. Perhaps a macro called my_new would allow you to text-transform your way there without as many parens or repeats of MyType.

In any case, asking for all your RAM at the beginning, and then re-blanking and re-using it, is certainly much faster for certain problem types (3D games with physics emulation, some scientific problems.)

[1] http://stackoverflow.com/questions/8301043/create-objects-in...

disclaimer: This post is about reinventing too many wheels, and should be taken with a pinch of salt! Moving malloc calls out of loops (and replacing them with hand-written zeroing functions) is probably the most useful tip in practise.



In C++ you're supposed to do this with std::allocator and alternatives. This allows you to change the allocator used by vector<>, map<>, etc. Or you use the placement new syntax you found with operator overloading (yes, you can overload "new MyType()")

You're really not supposed to do it with macros.


To be clear, there are four options you mentioned:

1. Changing the allocator used by the standard library by passing in a special allocator to instantiations of std::vector, std::map, etc.

2. Allocating raw memory elsewhere, and instantiating an object with that memory using the placement syntax for new.

3. Overriding operator new on just a particular class. This does not change the global new; it just means that objects of this class will be allocated with this version of new.

4. Overriding the global definition of new. All objects whose classes do not define their own operator new will be allocated with your own, global new.

More details here: http://en.cppreference.com/w/cpp/memory/new/operator_new


std:allocator and overloading new sound like much better alternatives, thank you.


In C++, apart from the placement new that others mentioned, it is possible to globally replace new and delete by writing the 4 functions

    void* operator new  ( std::size_t count );
    void* operator new[]( std::size_t count );
    void operator delete  ( void* ptr );
    void operator delete[]( void* ptr );
(from http://en.cppreference.com/w/cpp/memory/new/operator_new and http://en.cppreference.com/w/cpp/memory/new/operator_delete)

If you do it that way, you do not need to change any of the code that calls new and delete.

And I disagree about the 'relatively' in "I know that in C it is relatively easy to define your own memory management functions"

It is easy to piggy-back on an existing memory allocator, and it is fairly easy to make an implementation that works from a single thread, or an implementation that performs horribly from multiple threads, but it is not easy to make one that performs well from multiple threads and is useful.

[That "and is useful" is from the hacker in me. It is trivial to make what I think is a conforming implementation that multi-threads exceptionally well along the lines of (I am guessing at the prototypes):

    void * malloc(size_t t)            { return NULL;}
    void * calloc(int n, size_t t)     { return NULL;}
    void   free(void *m)               {}
    void * realloc( void *m, size_t t) { return NULL;}
but that would not be a very useful implementation]


I should clarify, I wasn't actually making my own malloc, I was only macro-switching it so that I could keep a list of all the malloc call returns. I was also interrupting free, so I could see which objects had been malloc'd but not free'd. I think I mentioned it was to track down memory leaks.

The example I quoted in Python's implementation, though, seems to hold truer - rather than calling malloc whenever it needs more space it allocates buffers and hands out memory from them, reducing the frequency of calls to malloc, and meaning that its garbage collector sometimes can't free as much RAM as you hoped (since there are some nearly-empty buffers that can't be released.) This makes running python instructions which need a little more RAM much faster, at the expense of complexity when RAM is short, of course. And overall the program should be faster, since it has fewer system calls (assuming that the code which hands out RAM is well optimised, which should be possible since it can be much simpler than the system malloc.)


... as long as you understand what's going on with dynamically loaded code, and you try to free something that was allocated by a module that uses a totally different allocator.

C++ standard meets modern runtimes. Wheee! :-)


The allocators that appeared in the first C++ Standard were designed by the Committee and turned out to not be as useful as expected. Both Electronic Arts ( http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n227... , https://github.com/paulhodge/EASTL/ ) and Bloomberg ( http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n185... , https://github.com/bloomberg/bde ) proposed a different approach, which allow what you want. C++11 includes stateful allocators and scoped allocators.


Sure it is possible. Just pass the constructor your type:

my_type* t = New( MyType ) ;

Where MyType passes information to New() of what to create.

...

I don't know what I'm downvoted.

This clearly works. The naive solution without macros would have MyType be a simple integer and New() has a switch statement. But it is easy to avoid that.


You are being downvoted because the language comes with well-defined and specifically designed ways to customize allocation/deallocation. Your approach choses to ignore those facilities and even adds a runtime decision to code that absolutely has no need for it. You are reinventing the wheel and it is not even round.


Because what you've written doesn't make much sense? You can't pass a type as a function parameter to a user defined function, and if you mean the builtin "new" (no capital) then that uses the normal allocator.


I think he meant that MyType is an enum and New is a user-defined function with a huge switch that does allocate (with placement new?) the correct type based on the value of MyType. Yea works, but introduces runtime overhead and is the gate to maintenance hell.


It is basic OOP in C. Look it up.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: