Ubuntu – Ubuntu 19 kidle_inject process

19.04cpu load

i have a friend of mine which have a brand new ASUS laptop.
I tried to use Ubuntu 18 which works fine on my ASUS laptop, but on his laptop, the screen was freezing maybe because of not supported graphics drivers (i suppose but not sure of that).

I read on ASUS forums that installing Ubuntu 19 can make things better.
This is the case. We installed it and its better now except one thing.

We have 4 kidle_inject processes that uses each 50% of CPU which make computer really slow.
How can i disable or lower the impact of kidle_inject processes ?

Thanks

Best Answer

  • The kidle_inj processes, one per CPU, are one way to achieve thermal throttling to help keep your processor cooler. Typically, invoked by thermald, the actual method used is a function of the CPU frequency scaling driver, with possible overides via the /etc/thermald/thermal-cpu-cdev-order.xml file.

    Let's work through a couple of examples. Even under 100% load on all CPUs, my test server doesn't overheat, so I will set a low thermal trip point of 55 degrees for this.

    First using the intel_pstate CPU frequency scaling driver and the powersave governor:

    doug@s15:~$ cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_driver
    intel_pstate
    intel_pstate
    intel_pstate
    intel_pstate
    intel_pstate
    intel_pstate
    intel_pstate
    intel_pstate
    doug@s15:~$ cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
    powersave
    powersave
    powersave
    powersave
    powersave
    powersave
    powersave
    powersave
    

    Now use turbostat to monitor things, and also watch the maximum performance setting (maximum allowed CPU frequency as a percent). It starts un-throttled:

    doug@s15:~$ cat /sys/devices/system/cpu/intel_pstate/max_perf_pct
    100
    

    .

    doug@s15:~$ sudo turbostat --quiet --Summary --show Busy%,Bzy_MHz,PkgTmp,PkgWatt --interval 5
    Busy%   Bzy_MHz PkgTmp  PkgWatt
    0.02    1600    26      3.70
    0.03    1600    26      3.70
    2.21    3737    38      6.87
    38.89   3564    48      42.70
    94.64   3500    50      58.54 <<< Load being ramped up.
    100.00  3500    52      58.49 <<< Processor package temperature going up.
    100.00  3500    53      58.78
    100.00  3500    56      59.04
    100.00  3500    56      59.27
    100.00  3123    53      51.18 <<< Notice throttling via clock frequency
    100.00  2969    56      47.32
    100.00  2693    52      41.90
    100.00  2009    53      28.98
    100.00  2489    55      37.90
    100.00  2431    54      36.82
    100.00  2620    54      40.50
    100.00  2409    55      36.39
    100.00  2511    54      38.47
    100.00  2569    57      39.61
    100.00  2301    53      34.57
    100.00  1682    53      23.64
    100.00  2089    54      30.52
    100.00  2569    56      39.59
    100.00  2301    52      34.55
    87.08   1671    53      22.98
    48.70   2037    52      24.04
    5.58    2318    44      7.50
    0.02    1603    40      3.88
    0.03    1600    40      3.87
    0.02    1600    39      3.85
    ^C0.04  1600    38      3.86
    

    And during the above the max percent was reduced, until after the load was removed, and the processor temperature dropped:

    doug@s15:~$ cat /sys/devices/system/cpu/intel_pstate/max_perf_pct
    60
    doug@s15:~$ cat /sys/devices/system/cpu/intel_pstate/max_perf_pct
    60
    doug@s15:~$ cat /sys/devices/system/cpu/intel_pstate/max_perf_pct
    100
    

    Second, use the acpi-cpufreq CPU scaling driver and ondemand governor:

    doug@s15:~$ cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_driver
    acpi-cpufreq
    acpi-cpufreq
    acpi-cpufreq
    acpi-cpufreq
    acpi-cpufreq
    acpi-cpufreq
    acpi-cpufreq
    acpi-cpufreq
    doug@s15:~$ cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
    ondemand
    ondemand
    ondemand
    ondemand
    ondemand
    ondemand
    ondemand
    ondemand
    

    Now use turbostat to monitor things, and also watch the kidle_inject threads. Notice that the turbostat output has an added column , %C6 which is the deepest idle state my processor goes into (done via "hide" instead of via "show" because "show" method doesn't work).

    doug@s15:~$ sudo turbostat --Summary --quiet --hide IRQ,Avg_MHz,SMI,GFXMHz,TSC_MHz,GFXWatt,CorWatt,POLL%,CPU%c1,CPU%c3,CPU%c7,CoreTmp,GFX%rc6,Pkg%pc2,Pkg%pc3,Pkg%pc6,POLL,C1,C1E,C3,C6,C1%,C1E%,C3%,C6% --interval 5
    Busy%   Bzy_MHz CPU%c6  PkgTmp  PkgWatt
    0.05    1602    99.83   26      3.71
    0.04    1600    99.87   28      3.71
    0.05    1600    99.83   26      3.71
    0.05    1601    99.84   26      3.71
    24.67   3591    52.79   45      30.24
    93.87   3500    0.00    47      58.30  <<< Load ramped up
    100.00  3500    0.00    50      58.42
    100.00  3500    0.00    53      58.70
    100.00  3500    0.00    55      58.99
    100.00  3500    0.00    56      59.23
    93.72   3424    6.18    56      54.44  <<< Now some C6 idle time is forced via kidle_inj
    81.41   3223    18.32   54      44.49
    77.81   3179    21.82   56      42.02
    83.82   3348    15.90   57      48.14
    78.87   3278    20.78   54      44.52
    66.34   3061    33.15   54      35.02
    62.65   2898    36.78   54      30.80
    61.20   2856    38.20   53      29.63
    63.71   3051    35.73   54      33.36
    61.67   2938    37.76   54      30.90
    61.92   2929    37.53   52      30.95
    63.47   3039    35.97   55      33.17
    60.87   2862    38.52   56      29.60
    62.90   3073    36.56   53      33.40
    62.36   2964    37.09   55      31.61
    61.16   2866    38.24   53      29.78
    63.98   3099    35.43   55      34.28
    56.37   2708    43.01   52      25.80
    52.01   2616    47.29   53      23.07
    58.24   2738    41.15   53      26.86
    65.39   3143    34.05   56      35.60
    68.01   3209    31.50   56      38.09
    58.62   2949    40.79   53      29.83
    58.43   2730    40.95   53      26.88
    48.87   3158    36.84   53      33.68
    14.74   2642    70.22   43      14.77
    0.37    1602    99.10   42      4.02
    0.29    1601    99.30   40      3.97
    0.23    1602    99.43   40      3.94
    0.17    1601    99.58   39      3.91
    0.17    1686    99.56   38      3.91
    0.06    1601    99.79   38      3.85
    0.04    1602    99.87   36      3.83
    0.04    1600    99.88   36      3.83
    0.09    1750    99.75   35      3.85
    0.04    1600    99.89   35      3.82
    0.04    1600    99.85   36      3.82
    0.04    1600    99.88   34      3.81
    0.04    1600    99.86   35      3.80
    ^C0.04  1600    99.87   33      3.80
    

    And during throttling, the kidle_inj tasks force the deep idle state and the processor power goes down.

    doug@s15:~$ ps aux | grep kidle
    USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
    root      3005 33.2  0.0      0     0 ?        S    00:19   0:32 [kidle_inj/0]
    root      3006 33.2  0.0      0     0 ?        S    00:19   0:32 [kidle_inj/1]
    root      3007 33.2  0.0      0     0 ?        S    00:19   0:32 [kidle_inj/2]
    root      3008 33.2  0.0      0     0 ?        S    00:19   0:32 [kidle_inj/3]
    root      3009 33.2  0.0      0     0 ?        S    00:19   0:32 [kidle_inj/4]
    root      3010 33.2  0.0      0     0 ?        S    00:19   0:32 [kidle_inj/5]
    root      3011 33.2  0.0      0     0 ?        S    00:19   0:32 [kidle_inj/6]
    root      3012 33.2  0.0      0     0 ?        S    00:19   0:32 [kidle_inj/7]
    
    doug@s15:~$ ps aux | grep kidle
    USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
    root      3005 33.5  0.0      0     0 ?        S    00:19   0:35 [kidle_inj/0]
    root      3006 33.5  0.0      0     0 ?        S    00:19   0:35 [kidle_inj/1]
    root      3007 33.5  0.0      0     0 ?        S    00:19   0:35 [kidle_inj/2]
    root      3008 33.5  0.0      0     0 ?        S    00:19   0:35 [kidle_inj/3]
    root      3009 33.5  0.0      0     0 ?        S    00:19   0:35 [kidle_inj/4]
    root      3010 33.5  0.0      0     0 ?        S    00:19   0:35 [kidle_inj/5]
    root      3011 33.5  0.0      0     0 ?        S    00:19   0:35 [kidle_inj/6]
    root      3012 33.5  0.0      0     0 ?        S    00:19   0:35 [kidle_inj/7]
    
    doug@s15:~$ ps aux | grep kidle
    USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
    root      3005 29.1  0.0      0     0 ?        S    00:19   0:44 [kidle_inj/0]
    root      3006 29.1  0.0      0     0 ?        S    00:19   0:44 [kidle_inj/1]
    root      3007 29.1  0.0      0     0 ?        S    00:19   0:44 [kidle_inj/2]
    root      3008 29.1  0.0      0     0 ?        S    00:19   0:44 [kidle_inj/3]
    root      3009 29.1  0.0      0     0 ?        S    00:19   0:44 [kidle_inj/4]
    root      3010 29.1  0.0      0     0 ?        S    00:19   0:44 [kidle_inj/5]
    root      3011 29.1  0.0      0     0 ?        S    00:19   0:44 [kidle_inj/6]
    root      3012 29.1  0.0      0     0 ?        S    00:19   0:44 [kidle_inj/7]
    
    doug@s15:~$ ps aux | grep kidle
    ... throttling over, processes gone ...
    

    You should not disable whatever thermal protection method you are using, but you could clean your fans and such to help keep the processor cooler. Also, if you have the option to use the pstate method of throttling, the remaining performance is typically higher than the kidle_inj method. For example, and for the workflow used for the above, the pstate method outperforms the intel_powerclamp kidle_inj method by 33%.

    Now, if for whatever reason your processor is capable of using the intel_pstate CPU frequency scaling driver, but you have chosen not to, then the suggestion is to use the intel_cpufreq driver (which is just the intel_pstate driver in passive mode) and the ondemand governor. Why? Because then the pstate throttling method will be used. On my system, resulting in about 28% performance improvement over the kidle_inject method under the same throttling conditions.

    How to change from intel_pstate to intel_cpufreq?

    doug@s15:~$ cat /sys/devices/system/cpu/intel_pstate/status
    active
    doug@s15:~$ echo passive | sudo tee /sys/devices/system/cpu/intel_pstate/status
    passive
    

    And set the governor:

    doug@s15:~$ sudo su
    root@s15:/home/doug# for file in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do echo "ondemand" > $file; done
    root@s15:/home/doug# exit
    exit
    doug@s15:~$ cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_driver
    intel_cpufreq
    intel_cpufreq
    intel_cpufreq
    intel_cpufreq
    intel_cpufreq
    intel_cpufreq
    intel_cpufreq
    intel_cpufreq
    doug@s15:~$ cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
    ondemand
    ondemand
    ondemand
    ondemand
    ondemand
    ondemand
    ondemand
    ondemand
    

    Why is there such a significant difference in performance? Because the kidle_inj method wastes a lot of time and energy going into and exiting the deep idle state, whereas the pstate method does not.

    And for users seeing "idle_inject" instead of, or in addition to, "kidle_inj":

    doug@s15:~$ ps aux | grep idle
        USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
    root        12  0.0  0.0      0     0 ?        S    09:07   0:00 [idle_inject/0]
    root        16  0.0  0.0      0     0 ?        S    09:07   0:00 [idle_inject/1]
    root        22  0.0  0.0      0     0 ?        S    09:07   0:00 [idle_inject/2]
    root        28  0.0  0.0      0     0 ?        S    09:07   0:00 [idle_inject/3]
    root        34  0.0  0.0      0     0 ?        S    09:07   0:00 [idle_inject/4]
    root        40  0.0  0.0      0     0 ?        S    09:07   0:00 [idle_inject/5]
    root        46  0.0  0.0      0     0 ?        S    09:07   0:00 [idle_inject/6]
    root        52  0.0  0.0      0     0 ?        S    09:07   0:00 [idle_inject/7]
    

    That is relatively recent, as of kernel 4.19, and the kernel configuration parameter is "CONFIG_IDLE_INJECT", which is set for Ubuntu kernels, but I don't yet know the purpose.

    EDIT (2019.08.09, Aug 9th):

    Readers, please be aware that for the intel_cpufreq CPU frequency scaling driver (intel_pstate in passive mode), and the acpi-cpufreq driver and schedutil governor some bugs may cause thermald to not work correctly. A reduction in the maximum allowed CPU frequency might not be honored by the system. Patches to fix these issues are in progress, but will take awhile to propagate.