<html>
<head>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
</head>
<body>
<div style="font-family:sans-serif">
</div>
<div>
<br>
<div>
<p>20 Jul 2022 13:15:13 Rabin Yasharzadehe <rabin@rabin.io>:</p>
</div>
<blockquote style="margin:0;border-left:3px solid #ccc; padding-left:10px">
<div dir="ltr">
<div class="gmail_default" style="font-family:tahoma,sans-serif">
Using ZFS with <a href="https://github.com/jimsalterjrs/sanoid">sanoid</a>
<br clear="all">
</div>
<div>
<div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr"><span style="font-family:tahoma,sans-serif"><br></span>
</div>Z<span class="gmail_default" style="font-family:tahoma,sans-serif">FS will give you all the benefits of COW filesystem, compression, and snapshots (and much more),</span>
</div>
<div class="gmail_signature" data-smartmail="gmail_signature"><span class="gmail_default" style="font-family:tahoma,sans-serif">combined with sanoid utility will allow you to automate the snapshots and send them to a remote system,</span>
</div>
<div class="gmail_signature" data-smartmail="gmail_signature"><span class="gmail_default" style="font-family:tahoma,sans-serif">and because ZFS is block-level aware of the changes between snapshots, send&recive is much more efficient, <br></span>
</div>
<div class="gmail_signature" data-smartmail="gmail_signature"><span class="gmail_default" style="font-family:tahoma,sans-serif">because unlike Rsync which needs to stats and compare each file to determine what to sync, <br></span>
</div>
<div class="gmail_signature" data-smartmail="gmail_signature"><span class="gmail_default" style="font-family:tahoma,sans-serif">ZFS only need to compile a list of block which have changed between 2 snapshots and send only them.<br></span>
</div>
<div class="gmail_signature" data-smartmail="gmail_signature"><span class="gmail_default" style="font-family:tahoma,sans-serif">which also works if the volume is encrypted, which allows you to have a remote system, which is encrypted on rest, <br></span>
</div>
<div class="gmail_signature" data-smartmail="gmail_signature"><span class="gmail_default" style="font-family:tahoma,sans-serif">and keep pushing/sending snapshots to it without having to unlock it.<br></span>
</div>
<div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr"><span style="font-family:tahoma,sans-serif"><br></span>
</div>
<div dir="ltr"><span style="font-family:tahoma,sans-serif"><br></span>
</div>
<div dir="ltr"><span style="font-family:tahoma,sans-serif">--<br>Rabin</span>
</div>
</div>
</div>
<br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">
On Sun, 17 Jul 2022 at 16:50, Shlomo Solomon <<a href="mailto:shlomo.solomon@gmail.com">shlomo.solomon@gmail.com</a>> wrote:
<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
I recently lost some files because of a bad disk - hardware problem.
<br>
<br> I do regular backups so I was not really worried, but I now see that I
<br> have a problem with my backup strategy so I'd like to know how others
<br> handle/prevent what happened to me.
<br>
<br> I backup files using rsync and I basically have 2 types of backups.
<br>
<br> My most important files are backed up every night. I do incremental
<br> backups using: rsync -aqrlvtogS --ignore-errors --backup
<br> I keep about 4 months of backups. So if a file is damaged,
<br> missing or accidentally deleted, I can find a good file - even if, for
<br> example I screwed up the file and only discovered the problem a few
<br> days later.
<br>
<br> BUT, all the rest of my files - music, videos, pictures, etc are backed
<br> up daily and weekly on 2 different physical drives using:
<br> rsync -qrlvtogS --delete --ignore-errors
<br> I use --delete to prevent accumulating garbage on my backup disks.
<br>
<br> So here's the problem: Because of a hardware problem, several files on
<br> one of my disks were lost. As a result, the daily backup script
<br> "thought" that those files should be deleted from the daily backup.
<br> Unfortunately, I did not notice the problem. A few days later, those
<br> same files were also deleted from the weekly backup. So they are lost.
<br>
<br> So on one hand, I need --delete to avoid keeping backups of old
<br> garbage, but on the other hand, the --delete option does not know if I
<br> deleted the file or if it's gone because of a hardware problem.
<br>
<br>
<br>
<br> --
<br> Shlomo Solomon
<br> <a href="http://the-solomons.net" rel="noreferrer" target="_blank">http://the-solomons.net</a>
<br> Claws Mail 3.17.5 - KDE Plasma 5.18.5 - Kubuntu 20.04
<br>
<br> _______________________________________________
<br> Linux-il mailing list
<br> <a href="mailto:Linux-il@cs.huji.ac.il" target="_blank">Linux-il@cs.huji.ac.il</a>
<br> <a href="http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il" rel="noreferrer" target="_blank">http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il</a>
<br>
</blockquote>
</div>
</blockquote>
</div><span dir="ltr" style="margin-top:0; margin-bottom:0;">Coming to this late, I use "backintime" which is a Python wrapper and GUI for rsync.</span>
<br><span dir="ltr" style="margin-top:0; margin-bottom:0;">One of the settings, "smart remove" is set to remove old snapshots, but keep some for exactly the kind of problem described here, e.g.</span>
<br><span dir="ltr" style="margin-top:0; margin-bottom:0;">2/day for the last week, then 1/week for a month, then 1/month for the previous year, etc. (you set all these yourself)</span>
<br>
<br><span dir="ltr" style="margin-top:0; margin-bottom:0;">I made a similar deletion error once which propagated through my Dropbox that I only noticed after I was past the 30 day backup there, but I easily pulled the files off my backup HDD from an old image</span>
<br>
</body>
</html>