Python question - first call is slower?

[Nadav Har'El]

On Thu, Jun 21, 2012, Nadav Har'El wrote about "Re: Python question - first call is slower?":
> My example program actually *forked* right before that first call.
> 
> It turns out that (at least, this is my understanding now) memory allocation -
> in Python but also in C - is significantly slower during this first call, but
>...
> The short explanation is copy-on-write page faults.

I have since discovered that there's another reason for slow memory
allocation after a fork, even if we avoid copy-on-write by filling the
parent's malloc arena before the fork:

When the child allocates *new* memory with sbrk(), and later uses it, "minor
page faults" happen causing the kernel to zero the new pages before the
program can using it. This zeroing takes time, but is important (for
security reasons). What worries me more, though, are these minor page
faults. These happen lazily - only when I access a page, a page fault
happens, so I am getting one minor page fault per page, and these appear
to have significant overhead (page table locks, TLB flushes, or I don't
know what).

What I'd rather happen in the child is that when sbrk() allocates new
memory, the kernel immediately maps and zeros all these pages - and
doesn't wait until they are actually used (which I know they will).
Do anyone have any idea if this can be done, hopefully without modifying
the kernel? Is there some madvise() or other trick which can be used to
ensure that a large number of pages get mapped *now* instead of mapping
them one after another?

Nadav.

-- 
Nadav Har'El                        |      Tuesday, Jul 3 2012, 13 Tammuz 5772
nyh at math.technion.ac.il             |-----------------------------------------
Phone +972-523-790466, ICQ 13349191 |The human mind is like a parachute - it
http://nadav.harel.org.il           |functions better when it is open.