Kernel memory management problem

[Alexander Indenbaum]

On Tue, Mar 3, 2009 at 1:28 AM, guy keren <choo at actcom.co.il> wrote:
> - Show quoted text -
> Alexander Indenbaum wrote:
>>
>> On Mon, Mar 2, 2009 at 4:44 PM, Valery Reznic <valery_reznic at yahoo.com>
>> wrote:
>>>
>>> --- On Mon, 3/2/09, Alexander Indenbaum <alexander.indenbaum at gmail.com>
>>> wrote:
>>>
>>>> From: Alexander Indenbaum <alexander.indenbaum at gmail.com>
>>>> Subject: Re: Kernel memory management problem
>>>> To: "guy keren" <choo at actcom.co.il>
>>>> Cc: "linux-il." <linux-il at cs.huji.ac.il>
>>>> Date: Monday, March 2, 2009, 4:24 PM
>>>> On Mon, Mar 2, 2009 at 1:16 PM, guy keren
>>>> <choo at actcom.co.il> wrote:
>>>>>
>>>>> if you don't care about the speed of copying the
>>>>
>>>> data and of slowing down
>>>>>
>>>>> the disk media consderably during this copying, try to
>>>>
>>>> change the copy
>>>>>
>>>>> program, so it will use the O_DIRECT flag when opening
>>>>
>>>> the file (and then
>>>>>
>>>>> you'll need to make sure the buffers you pass to
>>>>
>>>> write() are aligned to disk
>>>>>
>>>>> sector size, as well as being in disk sector size
>>>>
>>>> multiples). this way,
>>>>>
>>>>> you'll bypass the buffer cache altogether.
>>>>>
>>>>> maybe someone else will come up with a better
>>>>
>>>> response.
>>>> choo, nice to chat with you again. Still have an appetite
>>>> to red heads :) ?
>>>>
>>>> Thank you for your answer.
>>>>
>>>> While O_DIRECT is a valid way and probably will solve the
>>>> problem of
>>>> crashing, I would not want to patch wget/sftp/whatever. The
>>>> list is
>>>> long, I have better things to do mean while, instead of
>>>> burning in
>>>
>>> Patching every single program is not nessesary.
>>> You can use LD_PRELOAD mechanism to add O_DIRECT to each open
>>> function call in each program (ok - to each dynamically linked program)
>>>
>>
>> Nice and valid point. Thank you for idea.
>>
>>
>
> of-course, it will not work, because when working with O_DIRECT, all I/O
> buffers must be aligned to disk block size, and be in multiples of disk
> block size. applications not written to work with O_DIRECT will crash if you
> use such a trick, not to mention all the library functions that call open
> underneath.
>
> if you write such an LD_PRELOAD module, you'll need to catch all the calls
> to read and write for the FDs, and then make sure you do NOT do this for
> network connections ;)
>
> --guy
>

Even better :) Thank you for clarification.

~baum