recommended remote backup service?

[Shachar Shemesh]

Dvir Volk wrote:
> Hi,
> I need to find a new, secure and very reliable remote backup service
> for my employer's office server.
> This will be used to backup mainly stuff like SVN dumps, TRAC database, etc.
> 10-20 gigs should be more than enough, and ssh/rsync/sftp etc.
> scriptable access is a must.
> any recommendations?
>
> Thanks,
> Dvir
>
>   
Full disclosure - I have a personal AND business interest in this 
matter. Details later on. Feel free to question my objectivity.

Depending on the size of the individual files you are going to encrypt 
and your upstream bandwidth, I suggest you be wary of using duplicity. 
It offers quite good security (unless you intend to lose your encryption 
key, in which case don't bother backing up), but they do the delta 
calculation on the client side, and push an encrypted version of the 
delta. A restore operation involves restoring the last full backup you 
made, and then restoring all incremental backups done since then. They 
must ALL be available, or data will be lost.

With 10-20GB of data to back up, a full image upload may well be an 
impossible task to perform on a regular basis. Over a 1.5/150 ADSL link 
pushing 1GB of data will take you about 18 hours, which means that a 
20GB backup will take you a whole week or more (depending on how well 
the data compresses) to upload if you dedicate your line to it. It 
follows that having a weekly snapshot is pretty much out of the question.

I have not looked into duplicity much, yet, but it stands to reason it 
needs a copy of your entire data locally in order for it to have 
something to compare against. Deleting this data means that you cannot 
perform the incremental data.

<plug>

Or you can use rsyncrypto (http://rsyncrypto.lingnu.com). It's open 
source, it's fairly mature technology, and it is entirely scriptable. It 
encrypts the entire file locally, but in a rsync friendly way, so that 
you can use the standard rsync in order to push the file remotely. This 
means that any snapshot is a "full snapshot", in that they can be 
deleted in arbitrary order without jeopardizing your data. A local copy, 
in this case, is also saved between runs, but it can be recreated from 
the session keys, so it should be seen as more of a cache than a 
functional aspect of the program. The session keys are 68 bytes per 
file, and take no space at all (and they can be recovered from the 
actual encrypted data, of course)

This tool was created specifically for your use scenario, so if you find 
it lacking, I would really like to know in what way, so I can make it 
better.

<double plug>

My company, Lingnu open source consulting, is running a backup service 
that uses the technologies you mention as their technology. Our end user 
"nice gui" option is not as mature as we'd like, but for your intended 
use (scriptable working) the service is available today, and you would 
not be the first customer to use it.

Being as it is that the technology is open source, you are, of course, 
free to use it with any other service that supports rsync. rsync.net and 
Amazon's S3 were mentioned, and I have heard good things about both. 
They are, both, also likely to be cheaper than our service. What we 
offer is a service that specifically aims at your precise use scenario 
(small businesses, fully encrypted data with a key that is not shared 
with us, using rsyncrypto as the base technology). This may not amount 
to much today, but we have plans for the future :-).

If you decide that these are not compelling enough reasons to go with 
us, but would like to know more about the "future plans" when they are 
no longer "future", drop me an email and I'll keep you posted (I promise 
- no spam).

</double plug>

</plug>

Whatever you do, I whole heartily join the recommendation to keep your 
encryption key safe. I have some clients that chose to allow me to keep 
their key for them, and others that chose to keep the key themselves. 
The later group almost invariably had trouble getting to their key (in 
some cases - twice!). With rsyncrypto, if you did not lose your session 
keys, you can still recover without pushing the entire backup over the 
net from scratch, but this is still not a nice situation to be in.

Shachar

-- 
Shachar Shemesh
Lingnu Open Source Consulting Ltd.
http://www.lingnu.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.huji.ac.il/pipermail/linux-il/attachments/20090513/f8e89e6c/attachment.html>