Disk I/O as a bottleneck?

[is123 at zahav.net.il]

On Sun, 08 May 2011 18:11:24 +0300
Gilboa Davara <gilboad at gmail.com> wrote:

> On Sun, 2011-05-08 at 14:56 +0000, is123 at zahav.net.il wrote:
> > On Sun, 08 May 2011 17:28:07 +0300
> > Gilboa Davara <gilboad at gmail.com> wrote:
> > I don't track Linux very much but I can see from conky on my boxes Linux
> > just doesn't do that well. And race conditions are unfortunately an
> > ongoing problem in many apps.
> 
> I don't think race condition means what you think it means...
> You're most likely mixing race condition and resource / lock contention.

I'm not talking about contention which I understand they're trying to solve with the
removal of the BKL, I'm talking about bugs in application code. But both
contribute to software quality/usability issues for the end-user,
especially with multicore processors.

> More-ever, you're mixing SMP kernel issues (Such as the
> soon-to-be-officially-dead BKL [big kernel lock] and SMP scheduler
> issues) and application design issues. (Read: Application that are not
> design with big/huge-SMP in mind)

I'm not mixing anything, I'm saying *all* those things contribute to
performance problems.

> > I work on a different platform where multithreading and multiprocessing
> > were a very early part of the design and I have seen a big difference in
> > performance and lack of race conditions in that environment because it
> > was based on a native multithreading model whereas UNIX was based on
> > process forking and threading came much later and you could argue was
> > not exactly implemented seamlessly. It's not an apples and apples
> > comparison but the difference in software issues on those systems is
> > night and day. As far as I can see those problems still haven't been
> > resolved at the design or implementation levels.
> 
> A specific example will go a long way to explain your POV.

As I said my development experience is on a different platform with a
fundamentally different design. In that system, process forking is very
expensive and threading is very cheap- the opposite of the *NIX model. And
there are three or so decades of symmetric SMP exploitation so that stuff
is done and works and is not part of ongoing development and doesn't
break or cause problems and most of the software is designed to protect
against application level misuse and resource contention and deadlocks by
killing offending work to keep the system running well. Those kinds of
protections are not available in Linux or BSD AFAIK. For example you cannot
spin the processor on System Z from an application for very long without
being killed by the supervisor, but you can easily write userland code to
hog the machine in *NIX.

As *NIX was and is being changed to exploit SMP (and look at all the
code that has been added let's say in the last 5 years to do this) it's
very apparent to exploit the hardware threading is more useful than process
forking. But that way of thinking is newish in *NIX and not all the tools
and facilities (resource serialization etc) that are available in other
systems (System Z for example) are available to *NIX so there are growing
pains. A lot of progress has been made, no doubt. But there is still a lot
of room for improvement.