<div dir="ltr"><br><br><div class="gmail_quote">On Thu, Aug 13, 2009 at 9:59 AM, Micha Silver <span dir="ltr"><<a href="mailto:micha@arava.co.il">micha@arava.co.il</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
We suffered some annoying file and directory corruption on a CentOS 5.3 64 bit server two days ago after a long power failure - long enough to drain the UPS battery, with several short "spurts" of power until it finally stabilized. Files appeared as directories, directories turned into files, duplicated inodes, in short, a mess. And most of the corruption was under /usr, i.e. *not* files that were being written to when the power went off. The file system is ext3 but the journaling didn't help - well maybe it did, but not enough...<br>
<br>
<br>
The machine is a Dell PE 840 with their PERC 5i controller and 4 SATA disks in a RAID 5 array. It has its own battery backup to preserve the writeback cache in case of power failure (but again the files that got "kevorked" were not being written...). And it's relatively new ( < 2 yrs). Two identical machine attached to the same UPS, but w/o RAID came thru the event with no problems. And other, older servers with SCSI disks also showed no ill effects.<br>
<br>
Does anyone have any ideas how to prevent this kind of thing in the future? (short of adding lots of additional batteries)<br>
</blockquote><div><br>Sample the UPS unit for "how much battery time left do you have?" and initiate automatic shutdown when the number falls below the 5 minutes threshold ?<br><br>-- Shimi<br></div></div></div>