New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark

[Elazar Leibovich]

Reviving an old thread, here's a research, claiming link order and memory
layout can have a big effect on performance. The tool randomizes them, to
give a meaningful comparison between two programs.

A nugget from the slides, the difference between -O2 and -O3 on some LLVM
benchmarks was found to be a random statistical noise.

(no need to reread the old thread, it's loosely related)

http://plasma.cs.umass.edu/emery/stabilizer


On Wed, Feb 2, 2011 at 10:10 AM, Nadav Har'El <nyh at math.technion.ac.il>wrote:

> On Tue, Feb 01, 2011, Tzafrir Cohen wrote about "Re: New Freecell Solver
> gcc-4.5.0 vs. LLVM+clang Benchmark":
> > On Tue, Feb 01, 2011 at 12:49:00PM +0200, Elazar Leibovich wrote:
> >
> > > The program they tested[1] is strictly CPU bound. BTW, standard
> deviation
> > > wouldn't work as well, but it shows (I think) that there's  no such
> think as
> > > "ideal minimal runtime".
> > >
> > > [1]
> > > static int i = 0, j = 0, k = 0;
> > > int main() {
> > > int g = 0, inc = 1;
> > > for (; g<65536; g++) {
> > >   i += inc;
> > >   j += inc;
> > >   k += inc;
> > > }
> > > return 0;
> > > }
> > >
> > > [2] http://www-plan.cs.colorado.edu/diwan/asplos09.pdf
> >
> > That program is CPU-bound, but the time it runs is short enough for the
> > size of the environment to actually mean something.
>
> Right, I think this example demonstrates Oleg's point, that there's usually
> no in-determinism, but rather just ignorance of what else is going on
> outside
> your program.
>
> In this case, this is a program that finishes ridiculously fast - on my
> low-end computer, it finishes in 0.3 milliseconds (!). But when you run
> this program, things like fork, disk read, system calls, dynamic linking,
> and a whole lot of other crap add much more than 0.3 milliseconds of
> overhead,
> and much of that other crap is undeterministic - e.g., if the executable
> needs to be read from disk, and something else is using the disk, you
> notice
> a slowdown.
>
> But still - my original point was that if you measure this program's
> runtime
> several times, the minimum, not the average, is often more representative
> of
> the "true" runtime (in this case 0.3 milliseconds). I measured this program
> with "time", and one time got 30 milliseconds, several times got 2
> milliseconds,
> and one time got 1 millisecond. Yes, that minimum of 1 millisecond is
> closer
> to the performance of the program than the average of 1, 2 and 30.
>
> Anyway, you piqued my curiousity, and I'll read that ASPLOS paper :-)
>
> --
> Nadav Har'El                        |   Wednesday, Feb  2 2011, 28 Shevat
> 5771
> nyh at math.technion.ac.il
> |-----------------------------------------
> Phone +972-523-790466, ICQ 13349191 |I saw a book titled "Die Microsoft
> http://nadav.harel.org.il           |Windows". Turns out it was in
> German...
>
> _______________________________________________
> Linux-il mailing list
> Linux-il at cs.huji.ac.il
> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.huji.ac.il/pipermail/linux-il/attachments/20140510/19f17d92/attachment.html>