<div dir="ltr"><br><br><div class="gmail_quote">On Mon, Jul 19, 2010 at 12:18 PM, Nadav Har'El <span dir="ltr"><<a href="mailto:nyh@math.technion.ac.il">nyh@math.technion.ac.il</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Imagine, for example, that you run a certain program 5 times and get the<br>
times: 20.0, 18.0, 18.1, 27.0, 18.1<br>
Evidently, the first run was slower because things were not in the cache,<br>
and the run that took 27.0 was delayed by some other process in the background<br>
taking up the CPU or disk. The minimum run time, 18.0, is the most interesting<br>
one - it is the time a run would take every time, if things were perfect.<br>
If you average the above numbers, or find the standard deviation, etc.,<br>
the numbers would not be very interesting...<br></blockquote><div><br></div><div><meta http-equiv="content-type" content="text/html; charset=utf-8"><div>Just heard my intuition against that claim recited by a master, Joshua Bloch of the "Effective Java" fame.</div>
<div><br></div><div>Long story short, he claims there that modern computers are now highly non-deterministic, he demonstrated 20% running time variation by the same JVM running the same code. He claims like I felt, you must employ statistics on benchmark to get a meaningful result, and I think it implies that minimum is not the way to go here. I recommend this 30 minutes talk without any relation to the discussion.</div>
<div><br></div><div>Video: <a href="http://parleys.com/#id=2103&sl=12&st=5">http://parleys.com/#id=2103&sl=12&st=5</a></div><div>slides: <a href="http://wiki.jvmlangsummit.com/images/1/1d/PerformanceAnxiety2010.pdf">http://wiki.jvmlangsummit.com/images/1/1d/PerformanceAnxiety2010.pdf</a></div>
</div><div><br></div></div></div>