Cheap VM cluster (software, storage) ideas

[Ira Abramov]

Quoting Ghiora Drori, from the post of Wed, 04 Mar:
> Hi,
> Amazon EC2 instances plus their EBS disks.
> Scalable, available, reliable (from my experience) and you can experiment
> for a few $'s a day.

I tried to calculate the hosting costs, but was lost when I could not
find the full explanation on how to calculate some of the parameters
Amazon asked. Also it seemed the move to the EC2 would force me to use
S3 and that was a deal breaker. this EBS feature is something I missed
from the stroll in their site, so I suppose it's not something they are
pushing hard yet. is it production-ready? can I stick a mysql server and
all and just keep it all running as-is?

As for hosting cost calculations - is there a tool you can recommend I
can run on existing servers, or just a checklist to go over, that will
help me judge if my system would make the move smoothly and at what
price? I see so many conflicting suggestions on Google and I don't know
which to trust, and have 0 time for useless trial and error.

Also, knowing this client, I think he's not too happy about shooting his
precious core business machines over the net to be hosted out of reach.
It may be a psychological thing, but I can sympathize :-)

> On Wed, Mar 4, 2009 at 11:06 AM, Ira Abramov <Lists-Linux-IL at ira.abramov.org
> > wrote:
> 
> > related to the Citrix-vs-VMware question, in the spirit of the times...
> >
> > I want to create a way to host a cheap HA solution for a web hosting
> > outfit. they are running a few pretty busy asymetric servers and want to
> > start improving that infrastructure. each machine holds several dozens
> > of virtual hosts.
> >
> > At the moment each server has its own local storage and mysql. every
> > part is a SPOF other than the minimal RAID and such things.
> >
> > I'm thinking:
> > * Move to a central non-virtual MySQL for the backend.
> > * have two servers go P2V and have those VMs hosted back on their
> > original hardwares (sadly this means some painful downtime), and find a
> > way to let them crash-migrate for HA (still trying to figure this out)
> > * Second stage, add a second MySQL in a master-master setup.
> >
> > I'd love to have two servers with symetrical setup, but as you can
> > guess, the virtual hots are dozens of different apps that are too
> > expensive to go and rewrite for clusters at this point, with the issues
> > of user-uploaded files having to be available to both Apaches, etc.
> >
> > Assuming we want the cheapest reliable solution, i.e. not a $6K-20K SAN
> > and FC, I am looking for an easier solution (easier on the pocket at
> > least). However NFS proved to be a disaster in such cases (high-load web
> > services), OCFS has not been nice to me with any setup other than maybe
> > Oracle clusters, and GFS also never ran smoothly in my tests.
> >
> > Also OCFS and GFS require a common disk, which at this budget would be a
> > Linux machine running an iSCSI target at best (or OpenNAS).
> >
> > Am I missing something? Can this kind of reliability be achieved without
> > shelling out big bucks?
> >
> > Of course, The other option is just separate the MySQL, have a third
> > machine rsync the files of the two (non virtual) servers every few
> > minutes and have the hosting farm's layer4 switch redirect to the
> > fallback if something happens. Not very "smart" nor scalable, but does
> > 70% of what we need till a bigger investment is required.
> >
> > your thoughts, as before, are welcome...
> >
> > Thanks,
> > Ira.
> >
> > --
> > Can't catch me yet
> > Ira Abramov
> > http://ira.abramov.org/email/
> >
> > _______________________________________________
> > Linux-il mailing list
> > Linux-il at cs.huji.ac.il
> > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
> >
> 
> 
> 
> -- 
> Constant change is here to stay!

> _______________________________________________
> Linux-il mailing list
> Linux-il at cs.huji.ac.il
> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il


-- 
An out of body experiance
Ira Abramov
http://ira.abramov.org/email/