Kernel memory management problem

[Alexander Indenbaum]

Guys/Girls/Dudes,

I'm working with embedded Linux box powered by 2.4 kernel ( I know :)
we are in process of switching to 2.6 ) with 1G physical memory, no
swap defined and some slow flash storage mounted.

We noticed that if we copy large media file ( 1.4G ) to flash storage
filesystem, using wget/sftp/whatever kernel grabs almost all the
available physical memory and buffers are  not released even after
transfer is completed and sync; sync; sync is issued :). It
constitutes a problem - applications have no enough memory to run -
crash and burn in hell. See bellow memory consumption numbers (top
output):

Before test start: Mem: 97296K used, 807096K free, 0K shrd, 1732K
buff, 54712K cached
During/After test: Mem: 897908K used, 6484K free, 0K shrd, 2872K buff,
835088K cached

It worth mentioning that probably we see this issue since network IO
rate is higher than storage IO rate. I’m not sure which kernel
subsystem is memory grabber: VFS buffer cache, storage driver maybe
something else? Bottom line is we don’t want to let the kernel to be
so memory hungry and it’s desirable to set some reasonable limit on
memory usage and thus work around this problem.

What are your recommendations?

~baum