Who's counting jiffies when all CPUs idle in NO_HZ mode?

[Elazar Leibovich]

(If you've been at the office yesterday, maybe I wouldn't have send the
question ;-)

You've followed my line of thought (thinking that there's always one CPU
active). In some cases I think all CPUs must be asleep (otherwise, my
Android's battery would be draining even after I press the power button).
Still, two consecutive calls to clock_getttime should give a reasonable
result.

However, I can't find the place where the time is adjusted through the
hardware clock.

The thing is, the only architecture I can see clocksource other than
jiffies is s390[0], so I assume hardware internal clock is not used in many
important architectures. We can only count on the jiffies counter.

The jiffies counter is incremented only by do_timer[1], which is scheduled
to run do_timer, which runs on xtime_update, which seems to eventually runs
inside an IRQ handler called in a certain frequency in some systems.

While looking at it, it seems that I found the mechanism I was looking for.
It looks like the no_sched mechanism, simply lowers the CPU frequency , and
calculates the jiffies according to the new frequency while the CPU is
idle[2].

I still need to have a second look, but it looks like the CPU is always
woken up to count jiffies, but when it's idle it woken up in lower
frequencies, is aware to the fact it's idle, and thus is counting more
jiffies per tick.

[0] clocksource_register usage:
http://lxr.linux.no/linux+v3.13.5/+ident=61690701
[1] jiffies_64 usage: http://lxr.linux.no/linux+v3.13.5/+ident=59277722
[2] tick_nohz_* -
http://lxr.linux.no/linux+v3.13.5/kernel/time/tick-sched.c#L52


On Wed, Mar 26, 2014 at 6:45 AM, Shachar Shemesh <shachar at shemesh.biz>wrote:

>  Oops!
>
> The "one cpu always active" relates to the "full" tickless, not to the
> idle tickless. Please disregard that answer, except the use of the hardware
> clocks. I believe it is still valid, but I'll need to look at the source
> code.
>
> Shachar
>
>
> On 26/03/14 06:38, Shachar Shemesh wrote:
>
> So I answer this here, and then I get a visit in the office with the same
> question... :-)
>
> On 25/03/14 23:04, Elazar Leibovich wrote:
>
>  (I'm talking now about MONOTONIC_CLOCK_RAW, not taking NTP adjustment
> into account)
>
>  To my understanding, the basic time counting mechanism at the Linux
> kernel, is the jiffies counter. The way it counts time, is by leveraging a
> CPU interrupt happening at a certain known frequency. Every time this
> interrupt occurs, the interrupt handler would increment a counter. By
> multiplying this counter with the IRQ's frequency, we can estimate how much
> time passed.
>
>  Now, to my understanding, the NO_HZ_IDLE=y configuration, would prevent
> any interrupt on idle CPUs.
>
> At least for the time being, this does not mean what you think it means.
> See http://lwn.net/Articles/549580/
>
> Even in full tickless mode (properly referred to as "full" tickless mode),
> the boot CPU is still on a counter. Full ticklessness is still some way
> away.
>
> Even when we do achieve that lofty goal, most CPUs have a hardware counter
> that counts the time. Just like NAPI for network moved from an interrupt
> mode to polling mode for performance's sake, so we can do here.
>
>
> So to my understanding, if all CPUs are idle, nothing is going to run on
> any CPU.
>
>   No.
>
>  Who would count jiffies then? How can we be sure how much time passed
> with no hardware clock (as it is the case in some systems), and all CPUs
> asleep?
>
> When we finally move into the utopian full tickless mode, jiffies will
> lose their original meaning. In full tickless mode, there is no base
> frequency for the clock. As such, jiffies become an arbitrary number based
> on which you can decide how much time has passed, and can be set to
> whatever. How low to set them becomes a question of power management, then.
>
> Shachar
>
>
> _______________________________________________
> Linux-il mailing listLinux-il at cs.huji.ac.ilhttp://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
>
>
>
> _______________________________________________
> Linux-il mailing list
> Linux-il at cs.huji.ac.il
> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.huji.ac.il/pipermail/linux-il/attachments/20140326/b9045aab/attachment.html>