Cheap VM cluster (software, storage) ideas

[Ira Abramov]

related to the Citrix-vs-VMware question, in the spirit of the times...

I want to create a way to host a cheap HA solution for a web hosting
outfit. they are running a few pretty busy asymetric servers and want to
start improving that infrastructure. each machine holds several dozens
of virtual hosts.

At the moment each server has its own local storage and mysql. every
part is a SPOF other than the minimal RAID and such things.

I'm thinking:
* Move to a central non-virtual MySQL for the backend.
* have two servers go P2V and have those VMs hosted back on their
original hardwares (sadly this means some painful downtime), and find a
way to let them crash-migrate for HA (still trying to figure this out)
* Second stage, add a second MySQL in a master-master setup.

I'd love to have two servers with symetrical setup, but as you can
guess, the virtual hots are dozens of different apps that are too
expensive to go and rewrite for clusters at this point, with the issues
of user-uploaded files having to be available to both Apaches, etc.

Assuming we want the cheapest reliable solution, i.e. not a $6K-20K SAN
and FC, I am looking for an easier solution (easier on the pocket at
least). However NFS proved to be a disaster in such cases (high-load web
services), OCFS has not been nice to me with any setup other than maybe
Oracle clusters, and GFS also never ran smoothly in my tests.

Also OCFS and GFS require a common disk, which at this budget would be a
Linux machine running an iSCSI target at best (or OpenNAS).

Am I missing something? Can this kind of reliability be achieved without
shelling out big bucks?

Of course, The other option is just separate the MySQL, have a third
machine rsync the files of the two (non virtual) servers every few
minutes and have the hosting farm's layer4 switch redirect to the
fallback if something happens. Not very "smart" nor scalable, but does
70% of what we need till a bigger investment is required.

your thoughts, as before, are welcome...

Thanks,
Ira.

-- 
Can't catch me yet
Ira Abramov
http://ira.abramov.org/email/