sched: DL replenish lagged too much INFO: task udevd:593 blocked for more than 140 seconds. Not tainted 4.9.170+ #48 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. udevd D28008 593 470 0x00000000 ffff8801cfc9c740 ffff8801d0674200 ffff8801db621000 ffff8801cfc9df00 ffff8801db621018 ffff8801cfd1f990 ffffffff82801a4e 0000000000000246 ffff8801cfc9c740 0000000000000000 ffff8801db6218f0 1ffff10039fa3f21 Call Trace: [<00000000e2563114>] schedule+0x92/0x1c0 kernel/sched/core.c:3546 [<000000008ab7af94>] __rt_mutex_slowlock+0x99/0x290 kernel/locking/rtmutex.c:1221 [<00000000bc72faec>] rt_mutex_slowlock+0x199/0x4c0 kernel/locking/rtmutex.c:1293 [<000000007e411455>] rt_mutex_fastlock kernel/locking/rtmutex.c:1439 [inline] [<000000007e411455>] rt_mutex_lock+0x2d/0x40 kernel/locking/rtmutex.c:1499 [<0000000068a9a96a>] process_notifier drivers/misc/uid_sys_stats.c:636 [inline] [<0000000068a9a96a>] process_notifier+0x96/0x670 drivers/misc/uid_sys_stats.c:625 [<000000002dadb8f0>] notifier_call_chain+0xb4/0x1d0 kernel/notifier.c:93 [<00000000c2f53b56>] __blocking_notifier_call_chain kernel/notifier.c:317 [inline] [<00000000c2f53b56>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] [<00000000c2f53b56>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] [<00000000c2f53b56>] blocking_notifier_call_chain+0x80/0xa0 kernel/notifier.c:325 [<00000000e9735fcd>] profile_task_exit+0x1f/0x30 kernel/profile.c:140 [<000000000e10954f>] do_exit+0x90/0x2aa0 kernel/exit.c:744 [<00000000d625ffce>] do_group_exit+0x111/0x300 kernel/exit.c:945 [<0000000056b13968>] SYSC_exit_group kernel/exit.c:956 [inline] [<0000000056b13968>] SyS_exit_group+0x1d/0x20 kernel/exit.c:954 [<00000000a9c7ab51>] do_syscall_64+0x1ad/0x570 arch/x86/entry/common.c:285 [<000000006018cd48>] entry_SYSCALL_64_after_swapgs+0x5d/0xdb Showing all locks held in the system: 2 locks held by kworker/u4:0/5: #0: ("events_unbound"){.+.+.+}, at: [<00000000a82aa46e>] process_one_work+0x790/0x1600 kernel/workqueue.c:2107 #1: ((&sub_info->work)){+.+.+.}, at: [<000000000b369dbd>] process_one_work+0x7ce/0x1600 kernel/workqueue.c:2111 2 locks held by khungtaskd/24: #0: (rcu_read_lock){......}, at: [<0000000054176316>] check_hung_uninterruptible_tasks kernel/hung_task.c:169 [inline] #0: (rcu_read_lock){......}, at: [<0000000054176316>] watchdog+0x14b/0xaf0 kernel/hung_task.c:263 #1: (tasklist_lock){.+.+..}, at: [<00000000f621120a>] debug_show_all_locks+0x7f/0x21f kernel/locking/lockdep.c:4339 1 lock held by udevd/593: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by udevd/594: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by rsyslogd/1908: #0: (&f->f_pos_lock){+.+.+.}, at: [<00000000b97cf3f0>] __fdget_pos+0xa8/0xd0 fs/file.c:782 2 locks held by getty/2035: #0: (&tty->ldisc_sem){++++++}, at: [<00000000d7b595b9>] ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:377 #1: (&ldata->atomic_read_lock){+.+...}, at: [<00000000c21c0047>] n_tty_read+0x1fe/0x1820 drivers/tty/n_tty.c:2156 1 lock held by syz-executor.3/2118: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by syz-executor.1/2119: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by syz-executor.0/2120: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by syz-executor.4/2121: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 2 locks held by kworker/u4:4/2150: #0: ("events_unbound"){.+.+.+}, at: [<00000000a82aa46e>] process_one_work+0x790/0x1600 kernel/workqueue.c:2107 #1: ((&sub_info->work)){+.+.+.}, at: [<000000000b369dbd>] process_one_work+0x7ce/0x1600 kernel/workqueue.c:2111 1 lock held by udevd/3039: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by udevd/3041: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by syz-executor.0/3996: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by syz-executor.0/4005: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by syz-executor.0/4046: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by syz-executor.0/4047: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by syz-executor.3/3991: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by syz-executor.3/3994: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by syz-executor.3/4050: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by syz-executor.3/4053: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by syz-executor.1/4003: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by syz-executor.1/4006: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by syz-executor.1/4048: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by syz-executor.1/4049: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by syz-executor.4/4014: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by syz-executor.4/4015: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by syz-executor.4/4023: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by syz-executor.4/4025: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by modprobe/4051: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 1 lock held by modprobe/4052: #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:316 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] __blocking_notifier_call_chain kernel/notifier.c:304 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain kernel/notifier.c:328 [inline] #0: ((task_exit_notifier).rwsem){.+.+..}, at: [<000000006ce95df5>] blocking_notifier_call_chain+0x6a/0xa0 kernel/notifier.c:325 ============================================= NMI backtrace for cpu 0 CPU: 0 PID: 24 Comm: khungtaskd Not tainted 4.9.170+ #48 ffff8801d98d7cc8 ffffffff81b4fb21 0000000000000000 0000000000000000 0000000000000000 ffffffff81097401 dffffc0000000000 ffff8801d98d7d00 ffffffff81b5adbc 0000000000000000 0000000000000000 0000000000000000 Call Trace: [<0000000065700262>] __dump_stack lib/dump_stack.c:15 [inline] [<0000000065700262>] dump_stack+0xc1/0x120 lib/dump_stack.c:51 [<00000000fc2f5962>] nmi_cpu_backtrace.cold+0x47/0x87 lib/nmi_backtrace.c:99 [<00000000df235471>] nmi_trigger_cpumask_backtrace+0x124/0x155 lib/nmi_backtrace.c:60 [<000000008a722ad6>] arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:37 [<0000000066a5bcec>] trigger_all_cpu_backtrace include/linux/nmi.h:58 [inline] [<0000000066a5bcec>] check_hung_task kernel/hung_task.c:126 [inline] [<0000000066a5bcec>] check_hung_uninterruptible_tasks kernel/hung_task.c:183 [inline] [<0000000066a5bcec>] watchdog+0x670/0xaf0 kernel/hung_task.c:263 [<0000000087610a3f>] kthread+0x278/0x310 kernel/kthread.c:211 [<000000004578518f>] ret_from_fork+0x5c/0x70 arch/x86/entry/entry_64.S:373 Sending NMI from CPU 0 to CPUs 1: NMI backtrace for cpu 1 skipped: idling at pc 0xffffffff82810ac2