Cheap VM cluster (software, storage) ideas

Thu Mar 5 06:03:39 IST 2009

Hi,
Why would s3 bother you ? Its there it does not cost money if you do not use
it it cost fairly little when you do
and it works ok.
The only thing you will need s3 for is to store images of systems you want
to launch, you could use public images
but the cost of keeping private onse in s3 is negligible and this gives much
more flexiblity in creating custom servers.
EBS has been in production for quite a while (I also used it before it was
in production.) So far after a few month no glitches.

You can do anything you want with it software wise.
You do have to learn how to handle it aka instances can die and then you
loose their disk this has been rare lately, was
more common a few month ago. The EBS drives however do not die with the
machine so data on them is much safer, they
are kept in 3 copies at Amazon in different locations. You can split your
servers to 3 different zones for more reliability.
You can use s3 for backups and it is probably much safer then anything else
you have.

You can put mysql, or any other software in images or load them when the
instance is up.
You can run RedHat, Gentoo, Ubuntu or even Windows...
You can scale up and down the number of your servers and EBS disks as
needed.
Make an account and play with it!!
A small machine/instance (32 bits) is 10 cents an hour. If you cannot afford
that you are not really commercial :)
A few minor calculations should give you the correct cost numbers.

As sysadmin using it in a real web company it is very good, has a learning
curve like everything else.

On Wed, Mar 4, 2009 at 4:44 PM, Ira Abramov
<Lists-Linux-IL at ira.abramov.org>wrote:

> Quoting Ghiora Drori, from the post of Wed, 04 Mar:
> > Hi,
> > Amazon EC2 instances plus their EBS disks.
> > Scalable, available, reliable (from my experience) and you can experiment
> > for a few $'s a day.
>
> I tried to calculate the hosting costs, but was lost when I could not
> find the full explanation on how to calculate some of the parameters
> Amazon asked. Also it seemed the move to the EC2 would force me to use
> S3 and that was a deal breaker. this EBS feature is something I missed
> from the stroll in their site, so I suppose it's not something they are
> pushing hard yet. is it production-ready? can I stick a mysql server and
> all and just keep it all running as-is?
>
> As for hosting cost calculations - is there a tool you can recommend I
> can run on existing servers, or just a checklist to go over, that will
> help me judge if my system would make the move smoothly and at what
> price? I see so many conflicting suggestions on Google and I don't know
> which to trust, and have 0 time for useless trial and error.
>
> Also, knowing this client, I think he's not too happy about shooting his
> precious core business machines over the net to be hosted out of reach.
> It may be a psychological thing, but I can sympathize :-)
>
> > On Wed, Mar 4, 2009 at 11:06 AM, Ira Abramov <
> Lists-Linux-IL at ira.abramov.org
> > > wrote:
> >
> > > related to the Citrix-vs-VMware question, in the spirit of the times...
> > >
> > > I want to create a way to host a cheap HA solution for a web hosting
> > > outfit. they are running a few pretty busy asymetric servers and want
> to
> > > start improving that infrastructure. each machine holds several dozens
> > > of virtual hosts.
> > >
> > > At the moment each server has its own local storage and mysql. every
> > > part is a SPOF other than the minimal RAID and such things.
> > >
> > > I'm thinking:
> > > * Move to a central non-virtual MySQL for the backend.
> > > * have two servers go P2V and have those VMs hosted back on their
> > > original hardwares (sadly this means some painful downtime), and find a
> > > way to let them crash-migrate for HA (still trying to figure this out)
> > > * Second stage, add a second MySQL in a master-master setup.
> > >
> > > I'd love to have two servers with symetrical setup, but as you can
> > > guess, the virtual hots are dozens of different apps that are too
> > > expensive to go and rewrite for clusters at this point, with the issues
> > > of user-uploaded files having to be available to both Apaches, etc.
> > >
> > > Assuming we want the cheapest reliable solution, i.e. not a $6K-20K SAN
> > > and FC, I am looking for an easier solution (easier on the pocket at
> > > least). However NFS proved to be a disaster in such cases (high-load
> web
> > > services), OCFS has not been nice to me with any setup other than maybe
> > > Oracle clusters, and GFS also never ran smoothly in my tests.
> > >
> > > Also OCFS and GFS require a common disk, which at this budget would be
> a
> > > Linux machine running an iSCSI target at best (or OpenNAS).
> > >
> > > Am I missing something? Can this kind of reliability be achieved
> without
> > > shelling out big bucks?
> > >
> > > Of course, The other option is just separate the MySQL, have a third
> > > machine rsync the files of the two (non virtual) servers every few
> > > minutes and have the hosting farm's layer4 switch redirect to the
> > > fallback if something happens. Not very "smart" nor scalable, but does
> > > 70% of what we need till a bigger investment is required.
> > >
> > > your thoughts, as before, are welcome...
> > >
> > > Thanks,
> > > Ira.
> > >
> > > --
> > > Can't catch me yet
> > > Ira Abramov
> > > http://ira.abramov.org/email/
> > >
> > > _______________________________________________
> > > Linux-il mailing list
> > > Linux-il at cs.huji.ac.il
> > > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
> > >
> >
> >
> >
> > --
> > Constant change is here to stay!
>
> > _______________________________________________
> > Linux-il mailing list
> > Linux-il at cs.huji.ac.il
> > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
>
>
> --
> An out of body experiance
> Ira Abramov
> http://ira.abramov.org/email/
>
> _______________________________________________
> Linux-il mailing list
> Linux-il at cs.huji.ac.il
> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
>

-- 
Constant change is here to stay!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.huji.ac.il/pipermail/linux-il/attachments/20090304/0e33caa7/attachment-0001.html>