Modern Linux memory management

[guy keren]

On 01/26/2012 05:54 PM, Ori Berger wrote:
> On 01/26/2012 10:16 AM, Baruch Siach wrote:
>
>>> Only by using valgrind, that I could find the exact location and figure
>>> out, that it was another function that had the problem.
>>>
>>> How does the modern memory management system is working then, that it
>>> takes
>>> so much time for the problem to surface ?
>>
>> Now, if you corrupt the internal glibc data structure, glibc won't notice
>> until you try to call one of malloc(), free(), etc.
>
> And in addition to what Baruch said:
>
> Valgrind will always catch these errors, but will result in significant
> slowdown (x10-x20).

valgrind doesn't find ALL these errors.

specifically, it will not detect:

1. stack over-runs.

2. global (static) variables over-run.

3. cases where one function or class writes into the a memory area that 
is supposed to be used by another function/class.

> There are tools like DUMA (and its earlier
> incarnation, Electric Fence) incur almost no CPU overhead and can detect
> many kinds of corruptions as soon as they happen, by using the memory
> management units.

electric fence (and DUMA) has a much harder problem - it increases all 
memory allocations to use full pages (4KB by default on x86 and x86_64 
systems) - so even if you attempted to allocate 16 bytes - you'll end up 
getting a 4KB page.

any non-trivial program that allocates a lot of small objects using 
malloc - will consume eons of memory when executed under electric fence 
and DUMA.

>
> (Because of the MMU granularity, you need to run your program twice -
> one in which allocations are aligned to the lower address, and one when
> they are aligned to the top address)
>
> There is also a middle ground; gcc's mudflap
> <http://www.stlinux.com/devel/debug/mudflap>

this is interesting. it does catch stack overruns, which valgrind 
doesn't. i'll try it at work with a larger application.

and -- if your program is
> pure C and can be compiled by tcc,
> <http://bellard.org/tcc/tcc-doc.html#SEC21>; These are comparable to
> valgrind in functionality (for code you compile with them; standard
> library code runs at full speed/unchecked), but usually only introduce a
> small slowdown (10% or so).
>
>
> _______________________________________________
> Linux-il mailing list
> Linux-il at cs.huji.ac.il
> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il