Kernel memory management problem

[Alexander Indenbaum]

Valery,

See answer inline.

On Mon, Mar 2, 2009 at 1:11 PM, Valery Reznic <valery_reznic at yahoo.com> wrote:
>
> --- On Mon, 3/2/09, Alexander Indenbaum <alexander.indenbaum at gmail.com> wrote:
>
>>
>> We noticed that if we copy large media file ( 1.4G ) to
>> flash storage
>> filesystem, using wget/sftp/whatever kernel grabs almost
>> all the
>> available physical memory and buffers are  not released
>> even after
>> transfer is completed and sync; sync; sync is issued :). It
>> constitutes a problem - applications have no enough memory
>> to run -
>> crash and burn in hell. See bellow memory consumption
>> numbers (top
>> output):
> Are you really experience some problem or just worried
> about what free is shows ?
>
> To me it's looks perfectly normal:
> almost all of your memory is cached.
>
> I.e kernel know, that this memory was read/written to/from file(s)
> and if any of this content is needed kernel can provide it without access to the filesystem (disk, flash, whatever).
> But on the other size when kernel ready to give up that memory, when real need arise.
>

Thank you for your insight.

Unfortunately this is real problem: if you do something like
while(true) { transfer } then application crashes and burns in hell :)
 Nondeterministically - guys researching this issue called it
Heisenbug ( http://en.wikipedia.org/wiki/Unusual_software_bug ) .
There is clear expectation that application should have at least 25 MB
to run in. Hunger of kernel for memory must have hard limits, even if
it means making transfer more slow - which is not a problem at all.

~baum

> Valery.
>
>
>>
>> Before test start: Mem: 97296K used, 807096K free, 0K shrd,
>> 1732K
>> buff, 54712K cached
>> During/After test: Mem: 897908K used, 6484K free, 0K shrd,
>> 2872K buff,
>> 835088K cached
>>
>> It worth mentioning that probably we see this issue since
>> network IO
>> rate is higher than storage IO rate. I’m not sure which
>> kernel
>> subsystem is memory grabber: VFS buffer cache, storage
>> driver maybe
>> something else? Bottom line is we don’t want to let the
>> kernel to be
>> so memory hungry and it’s desirable to set some
>> reasonable limit on
>> memory usage and thus work around this problem.
>>
>> What are your recommendations?
>>
>> ~baum
>>
>> _______________________________________________
>> Linux-il mailing list
>> Linux-il at cs.huji.ac.il
>> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il