Kernel memory management problem

[guy keren]

Alexander Indenbaum wrote:
> On Mon, Mar 2, 2009 at 4:44 PM, Valery Reznic <valery_reznic at yahoo.com> wrote:
>> --- On Mon, 3/2/09, Alexander Indenbaum <alexander.indenbaum at gmail.com> wrote:
>>
>>> From: Alexander Indenbaum <alexander.indenbaum at gmail.com>
>>> Subject: Re: Kernel memory management problem
>>> To: "guy keren" <choo at actcom.co.il>
>>> Cc: "linux-il." <linux-il at cs.huji.ac.il>
>>> Date: Monday, March 2, 2009, 4:24 PM
>>> On Mon, Mar 2, 2009 at 1:16 PM, guy keren
>>> <choo at actcom.co.il> wrote:
>>>> if you don't care about the speed of copying the
>>> data and of slowing down
>>>> the disk media consderably during this copying, try to
>>> change the copy
>>>> program, so it will use the O_DIRECT flag when opening
>>> the file (and then
>>>> you'll need to make sure the buffers you pass to
>>> write() are aligned to disk
>>>> sector size, as well as being in disk sector size
>>> multiples). this way,
>>>> you'll bypass the buffer cache altogether.
>>>>
>>>> maybe someone else will come up with a better
>>> response.
>>> choo, nice to chat with you again. Still have an appetite
>>> to red heads :) ?
>>>
>>> Thank you for your answer.
>>>
>>> While O_DIRECT is a valid way and probably will solve the
>>> problem of
>>> crashing, I would not want to patch wget/sftp/whatever. The
>>> list is
>>> long, I have better things to do mean while, instead of
>>> burning in
>> Patching every single program is not nessesary.
>> You can use LD_PRELOAD mechanism to add O_DIRECT to each open
>> function call in each program (ok - to each dynamically linked program)
>>
> 
> Nice and valid point. Thank you for idea.
> 
> 

of-course, it will not work, because when working with O_DIRECT, all I/O 
buffers must be aligned to disk block size, and be in multiples of disk 
block size. applications not written to work with O_DIRECT will crash if 
you use such a trick, not to mention all the library functions that call 
open underneath.

if you write such an LD_PRELOAD module, you'll need to catch all the 
calls to read and write for the FDs, and then make sure you do NOT do 
this for network connections ;)

--guy