Python question - first call is slower?

[Nadav Har'El]

On Wed, Jun 20, 2012, Meir Kriheli wrote about "Re: Python question - first call is slower?":
> > I considered, and discredited, the following attempted "explanations":
>...
> >   its code gets done in 6 milliseconds; It's not a 12 millisecond pause
> >   and then the rest of the function finishes in 2ms.
> >
> 
> Yeah, compiles to bytecode.

As far as I understand, the code is "compiled to bytecode" when it is
imported, *not* when it is first run, so it doesn't explain why the
first run of a function is slower. If you think it's otherwise, please
let me know.

> Nitpick: CPython doesn't (the one you're referring to as Python), other
> implementations may and will (PyPy, Jython, etc)

Yes, I'm talking about the "python" executable, which I guess is
CPython.

> > 3. "If class A imports B which imports C which imports D, some of these
> >   classes are only read when the code is actually used for the first
> >   time". Again, I couldn't find any evidence that this is true in Python
> >   (unlike, e.g., Java). An "import" would read the whole class hierarchy
> > into
> >   memory. Right?
> A module is loaded only once (see also: sys.modules)

Yes, this I know. But at first I thought maybe it is loaded lazily
somehow, so some things are loaded only when a function from the module
is actually run. But I can find no evidence of that - every bit of
documentation I can find suggests that the module, and every module it
refers to, recursively, are fully loaded at the time of the "import" of
the top module.

> Can you post some code ? Without it this may be tough.

Yes, sorry. The specific code I saw this problem with is free software, so I
could theoretically post it, but it takes quite a bit of setup to get
started with so I don't think it would be very useful for others. I'll try
to see if I can recreate this problem with a much simpler piece of code
and report back.

> Do you have imports inside the function ?

I looked for these, and didn't find any.

> Do you access and affect globals ?

Why does that matter? Anyway, like I said the function doesn't keep any
state, so it's not like the second run notices the first run has already
done the job, and does nothing - or anything like that. Unless, of
course, I'm misunderstanding something in the code ;-)

> Is it a generator ?

What does this mean? Sorry, but my level of knowledge of Python is well
below my level in other programming languages... I guess I felt that I
already know one language too many :(

> Do you have default parameters ? Are they mutable ? Do they require some
> computation ?

This is a complicated setup, with one function calling another calling
..., so the whole thing probably uses every imaginable Python construct,
but probably not in a heavy way which can explain a 12ms (6-fold)
slowdown of the top function. I'll investigate.

> Is there a difference in the dataset between the runs (python caches small
> ints).

No, it's the same computation again. I'd be surprised if the number of
integers in this code explains a 12ms (6-fold) slowdown in that
function, but I'll investigate.

> Does it manipulate files (will be cached by the OS),

Yes, but I run it over and over with the same files, so they will
already be cached in all the runs (including the "first", which is the
first only in this python process).

Thanks,
Nadav.

-- 
Nadav Har'El                        |                 Wednesday, Jun 20 2012, 
nyh at math.technion.ac.il             |-----------------------------------------
Phone +972-523-790466, ICQ 13349191 |If marriage was illegal, only outlaws
http://nadav.harel.org.il           |would have in-laws.