<div dir="ltr"><br><div class="gmail_quote">2011/8/23 Hetz Ben Hamo <span dir="ltr"><<a href="mailto:hetzbh@gmail.com">hetzbh@gmail.com</a>></span><br><blockquote class="gmail_quote" style="margin:0 .8ex;border-left:1px #ccc solid;border-right:1px #ccc solid;padding-left:1ex;padding-right:1ex">
<div dir="ltr">Hi,<div><br></div><div>I'm thinking about some project, and I was wondering about something: lets say I'm setting up a really-big-web site. Something in the scale of Walla or Ynet.</div><div>My question is about the DB: Where are exactly my limitations that I need to go master/slave, replications etc.</div>
<div>Is it related to memory? processor? I know that many uses replication etc due to shortage or RAM and/or CPU power.</div><div><br></div><div>Lets say I have a machine with 512GB RAM, 20 cores, 20TB disks. Such a machine should handle a huge load without any issue. With such a machine, theoretically, can a single MySQL / PostgreSQL handle such a load? (few hundred thousands connected users at once).</div>
<div><br><br></div></div></blockquote><div> </div></div>It really depends on *what* you do, like has been said, ACID consideration, and of course, the ratio between read/write operations. <br><br>There is a certain point where "a bigger box" simply won't cut it, for various reasons in the way hardware and software are designed. <br>
<br>It would probably be better to scale vertically than horizontally... For example, if you talk about a site like Ynet - most of Ynet are read operations (I assume...) - you would be better off with running one master, to which you'll write updates, and many nodes running slave replication from the master (and not receiving any updates themselves), and each backend processing web server, would connect to a matching SQL backend, so the load is spread between them all... <br>
<br>It's also smart to just avoid accessing SQL altogether where possible (with caches like memcached, and caches on the web frontends where possible) - and use low resource-consuming frontends / load balancers (I know you didn't ask about it, but you would probably return here and ask about it if you would choose Apache ;)) - I like nginx, which is a performance beast.<br>
<br>NoSQL has been mentioned here - and I re-iterate about ACID considerations again, see the funny <a href="http://www.youtube.com/watch?v=URJeuxI7kHo">http://www.youtube.com/watch?v=URJeuxI7kHo</a> :)<br><br>To sum it all - you need to design what type (and volume, per user, multiplied by the number of users) of queries you'll be doing... and... hire a professional DB consultant ;-) Such questions cannot be answered "on one foot"... Proper queries and indices can change the load on a DB by orders of magnitude, too.<br>
<br>So I say, "no, this machine would probably not work with the load you mentioned", simply because I don't think it scales in a linear manner as one might imagine...<br><br>HTH,<br><br>-- Shimi<br></div>