Memory pool interface design

[Elazar Leibovich]

The question of whether to use a global malloc function, or to use a
function pointer is orthogonal to my question.

My question is, should I support the case of malloc failure. On one hand,
it complicates the API significantly, but on the other hand it might be
useful for some use cases.

It's pretty obvious to me that in a modern Linux userspace program,
supporting malloc failure does not worth the trouble. But are there other
use cases where it's vital?

Another clarification, my code would never have abort. What I was saying,
that the malloc could simply abort current task, if it does not have memory.

As a side note, In my experience, it is sometimes useful to use
preallocated "memory pools"[0]. Letting the user choose memory allocator is
also useful when using it in the kernel, since otherwise the library simply
won't compile. See for example protobuf-c which receives an allocator in
its functions,
https://github.com/protobuf-c/protobuf-c/blob/master/protobuf-c/protobuf-c.c#L2019


[0]
http://eli.thegreenplace.net/2008/10/17/memmgr-a-fixed-pool-memory-allocator

On Fri, May 15, 2015 at 9:00 PM, Baruch Even <baruch at ev-en.org> wrote:

> I would question the need to abstract away the memory allocations of your
> library compared to everything else. If someone cares enough about it he
> can replace malloc and free completely to use a different allocation scheme.
>
> In most cases I've cared about memory allocations I just wanted none of
> them at all and only wanted intrusive data structures and just running the
> system with a fixed memory allocation from the start to the end. It's not
> always possible in a generic library though..
>
> If you are writing a library you should never abort inside it, that would
> be very annoying to the user. Give him a null and let him crash or handle
> it as he sees fit.
>
> Baruch
>
> On Fri, May 15, 2015 at 5:47 PM, Elazar Leibovich <elazarl at gmail.com>
> wrote:
>
>> I'm writing a small C library, that I want to open source.
>>
>> I want them to be usable for embedded environment, where memory
>> allocation must be controlled.
>>
>> Hence, I abstracted away calls to malloc/realloc, and replaced them with
>>
>> struct mem_pool {
>>     void *(*allloc)(void *mem_pool, void *prev_ptr, int size);
>> };
>>
>> User would implement
>>
>> struct my_mem_pool {
>>     struct mem_pool pool;
>>     ...
>> };
>>
>> struct my_mem_pool pool = { { my_alloc_func }, ...);
>>
>> I've had to design question I'm interested with:
>>
>> 1) Should I support both malloc and realloc?
>>
>> I think the performance benefits of supporting malloc instead of
>> realloc(NULL) are negligible, and not worth complicating the interface.
>>
>> 2) Should the memory pool be allowed to fail?
>>
>> In typical Linux system, where memory overcommit is allowed, checking
>> malloc return value provides little benefit. But is it the same for
>> embedded system?
>>
>> My feeling is, embedded system should predict the memory usage for each
>> input size, and avoid processing input which is too large.
>>
>> For example, stack overflow error can never be handled, and one is
>> expected to calculate the longest stack length for any input and make sure
>> he wouldn't overflow.
>>
>> So I think it's still reasonable never to report allocation failure, and
>> to expect the memory allocator to raise the relevant abort/panic/exception
>> in such a case.
>>
>> But I'll be happy to hear other considerations I missed.
>>
>> Thanks,
>>
>> _______________________________________________
>> Linux-il mailing list
>> Linux-il at cs.huji.ac.il
>> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.huji.ac.il/pipermail/linux-il/attachments/20150516/cbba3e53/attachment.html>