INFO: task kworker/0:5:5289 blocked for more than 143 seconds. Not tainted 5.1.0-rc4-319354-g9a33b36 #3 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kworker/0:5 D22736 5289 2 0x80000000 Workqueue: usb_hub_wq hub_event Call Trace: schedule+0x8f/0x180 kernel/sched/core.c:3562 schedule_timeout+0x477/0xdb0 kernel/time/timer.c:1803 usb_start_wait_urb.constprop.0+0x2a2/0x2e0 drivers/net/usb/kaweth.c:1250 kaweth_internal_control_msg drivers/net/usb/kaweth.c:1286 [inline] kaweth_control.constprop.0+0x365/0x4f0 drivers/net/usb/kaweth.c:281 kaweth_read_configuration drivers/net/usb/kaweth.c:299 [inline] kaweth_probe.cold+0xb1/0x1359 drivers/net/usb/kaweth.c:1077 usb_probe_interface+0x31d/0x820 drivers/usb/core/driver.c:361 really_probe+0x2da/0xb10 drivers/base/dd.c:509 driver_probe_device+0x21d/0x350 drivers/base/dd.c:671 __device_attach_driver+0x1d8/0x290 drivers/base/dd.c:778 bus_for_each_drv+0x163/0x1e0 drivers/base/bus.c:454 __device_attach+0x223/0x3a0 drivers/base/dd.c:844 bus_probe_device+0x1f1/0x2a0 drivers/base/bus.c:514 device_add+0xad2/0x16e0 drivers/base/core.c:2106 usb_set_configuration+0xdf7/0x1740 drivers/usb/core/message.c:2021 generic_probe+0xa2/0xda drivers/usb/core/generic.c:210 usb_probe_device+0xc0/0x150 drivers/usb/core/driver.c:266 really_probe+0x2da/0xb10 drivers/base/dd.c:509 driver_probe_device+0x21d/0x350 drivers/base/dd.c:671 __device_attach_driver+0x1d8/0x290 drivers/base/dd.c:778 bus_for_each_drv+0x163/0x1e0 drivers/base/bus.c:454 __device_attach+0x223/0x3a0 drivers/base/dd.c:844 bus_probe_device+0x1f1/0x2a0 drivers/base/bus.c:514 device_add+0xad2/0x16e0 drivers/base/core.c:2106 usb_new_device.cold+0x537/0xccf drivers/usb/core/hub.c:2534 hub_port_connect drivers/usb/core/hub.c:5089 [inline] hub_port_connect_change drivers/usb/core/hub.c:5204 [inline] port_event drivers/usb/core/hub.c:5350 [inline] hub_event+0x138e/0x3b00 drivers/usb/core/hub.c:5432 process_one_work+0x90f/0x1580 kernel/workqueue.c:2269 worker_thread+0x9b/0xe20 kernel/workqueue.c:2415 kthread+0x313/0x420 kernel/kthread.c:253 ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:352 Showing all locks held in the system: 1 lock held by khungtaskd/23: #0: 0000000069181d1b (rcu_read_lock){....}, at: debug_show_all_locks+0x53/0x269 kernel/locking/lockdep.c:5059 5 locks held by kworker/1:2/532: #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: __write_once_size include/linux/compiler.h:220 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:855 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:40 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: set_work_data kernel/workqueue.c:619 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: process_one_work+0x81f/0x1580 kernel/workqueue.c:2240 #1: 000000000718afc6 ((work_completion)(&hub->events)){+.+.}, at: process_one_work+0x853/0x1580 kernel/workqueue.c:2244 #2: 0000000088855365 (&dev->mutex){....}, at: device_lock include/linux/device.h:1207 [inline] #2: 0000000088855365 (&dev->mutex){....}, at: hub_event+0x18a/0x3b00 drivers/usb/core/hub.c:5378 #3: 00000000d9f1f8aa (&port_dev->status_lock){+.+.}, at: usb_lock_port drivers/usb/core/hub.c:2994 [inline] #3: 00000000d9f1f8aa (&port_dev->status_lock){+.+.}, at: hub_port_connect drivers/usb/core/hub.c:5020 [inline] #3: 00000000d9f1f8aa (&port_dev->status_lock){+.+.}, at: hub_port_connect_change drivers/usb/core/hub.c:5204 [inline] #3: 00000000d9f1f8aa (&port_dev->status_lock){+.+.}, at: port_event drivers/usb/core/hub.c:5350 [inline] #3: 00000000d9f1f8aa (&port_dev->status_lock){+.+.}, at: hub_event+0x11a3/0x3b00 drivers/usb/core/hub.c:5432 #4: 00000000cf884ab5 (hcd->address0_mutex){+.+.}, at: hub_port_init+0x1bb/0x2d30 drivers/usb/core/hub.c:4529 1 lock held by rsyslogd/5072: #0: 0000000027ac3d34 (&f->f_pos_lock){+.+.}, at: __fdget_pos+0xe8/0x100 fs/file.c:801 2 locks held by getty/5162: #0: 000000000def69d3 (&tty->ldisc_sem){++++}, at: tty_ldisc_ref_wait+0x27/0x80 drivers/tty/tty_ldisc.c:272 #1: 000000001a2dd258 (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x21c/0x1a60 drivers/tty/n_tty.c:2156 2 locks held by getty/5163: #0: 0000000060cc791c (&tty->ldisc_sem){++++}, at: tty_ldisc_ref_wait+0x27/0x80 drivers/tty/tty_ldisc.c:272 #1: 000000009c3913ed (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x21c/0x1a60 drivers/tty/n_tty.c:2156 2 locks held by getty/5164: #0: 00000000e0488faa (&tty->ldisc_sem){++++}, at: tty_ldisc_ref_wait+0x27/0x80 drivers/tty/tty_ldisc.c:272 #1: 000000000678d50c (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x21c/0x1a60 drivers/tty/n_tty.c:2156 2 locks held by getty/5165: #0: 000000004eb0e683 (&tty->ldisc_sem){++++}, at: tty_ldisc_ref_wait+0x27/0x80 drivers/tty/tty_ldisc.c:272 #1: 00000000ba771a43 (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x21c/0x1a60 drivers/tty/n_tty.c:2156 2 locks held by getty/5166: #0: 0000000074fd1a3b (&tty->ldisc_sem){++++}, at: tty_ldisc_ref_wait+0x27/0x80 drivers/tty/tty_ldisc.c:272 #1: 000000006905c687 (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x21c/0x1a60 drivers/tty/n_tty.c:2156 2 locks held by getty/5167: #0: 00000000db370967 (&tty->ldisc_sem){++++}, at: tty_ldisc_ref_wait+0x27/0x80 drivers/tty/tty_ldisc.c:272 #1: 000000006a517507 (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x21c/0x1a60 drivers/tty/n_tty.c:2156 2 locks held by getty/5168: #0: 0000000094a5e36f (&tty->ldisc_sem){++++}, at: tty_ldisc_ref_wait+0x27/0x80 drivers/tty/tty_ldisc.c:272 #1: 000000005c840ecb (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x21c/0x1a60 drivers/tty/n_tty.c:2156 5 locks held by kworker/0:4/5268: #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: __write_once_size include/linux/compiler.h:220 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:855 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:40 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: set_work_data kernel/workqueue.c:619 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: process_one_work+0x81f/0x1580 kernel/workqueue.c:2240 #1: 0000000087ae2882 ((work_completion)(&hub->events)){+.+.}, at: process_one_work+0x853/0x1580 kernel/workqueue.c:2244 #2: 0000000081d23cc8 (&dev->mutex){....}, at: device_lock include/linux/device.h:1207 [inline] #2: 0000000081d23cc8 (&dev->mutex){....}, at: hub_event+0x18a/0x3b00 drivers/usb/core/hub.c:5378 #3: 00000000b4791037 (&dev->mutex){....}, at: device_lock include/linux/device.h:1207 [inline] #3: 00000000b4791037 (&dev->mutex){....}, at: __device_attach+0x80/0x3a0 drivers/base/dd.c:821 #4: 00000000951aab5c (&dev->mutex){....}, at: device_lock include/linux/device.h:1207 [inline] #4: 00000000951aab5c (&dev->mutex){....}, at: __device_attach+0x80/0x3a0 drivers/base/dd.c:821 5 locks held by kworker/0:5/5289: #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: __write_once_size include/linux/compiler.h:220 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:855 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:40 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: set_work_data kernel/workqueue.c:619 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: process_one_work+0x81f/0x1580 kernel/workqueue.c:2240 #1: 000000006dacbce5 ((work_completion)(&hub->events)){+.+.}, at: process_one_work+0x853/0x1580 kernel/workqueue.c:2244 #2: 000000003d085337 (&dev->mutex){....}, at: device_lock include/linux/device.h:1207 [inline] #2: 000000003d085337 (&dev->mutex){....}, at: hub_event+0x18a/0x3b00 drivers/usb/core/hub.c:5378 #3: 00000000413e1510 (&dev->mutex){....}, at: device_lock include/linux/device.h:1207 [inline] #3: 00000000413e1510 (&dev->mutex){....}, at: __device_attach+0x80/0x3a0 drivers/base/dd.c:821 #4: 0000000023650cb0 (&dev->mutex){....}, at: device_lock include/linux/device.h:1207 [inline] #4: 0000000023650cb0 (&dev->mutex){....}, at: __device_attach+0x80/0x3a0 drivers/base/dd.c:821 3 locks held by kworker/1:4/5305: #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: __write_once_size include/linux/compiler.h:220 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:855 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:40 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: set_work_data kernel/workqueue.c:619 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: process_one_work+0x81f/0x1580 kernel/workqueue.c:2240 #1: 00000000a29f723e ((work_completion)(&hub->events)){+.+.}, at: process_one_work+0x853/0x1580 kernel/workqueue.c:2244 #2: 00000000f3e16779 (&dev->mutex){....}, at: device_lock include/linux/device.h:1207 [inline] #2: 00000000f3e16779 (&dev->mutex){....}, at: hub_event+0x18a/0x3b00 drivers/usb/core/hub.c:5378 6 locks held by kworker/0:6/5375: #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: __write_once_size include/linux/compiler.h:220 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:855 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:40 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: set_work_data kernel/workqueue.c:619 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: process_one_work+0x81f/0x1580 kernel/workqueue.c:2240 #1: 0000000036f7f6c1 ((work_completion)(&hub->events)){+.+.}, at: process_one_work+0x853/0x1580 kernel/workqueue.c:2244 #2: 00000000767f3e53 (&dev->mutex){....}, at: device_lock include/linux/device.h:1207 [inline] #2: 00000000767f3e53 (&dev->mutex){....}, at: hub_event+0x18a/0x3b00 drivers/usb/core/hub.c:5378 #3: 0000000003b47fdb (&dev->mutex){....}, at: device_lock include/linux/device.h:1207 [inline] #3: 0000000003b47fdb (&dev->mutex){....}, at: __device_attach+0x80/0x3a0 drivers/base/dd.c:821 #4: 000000009711a749 (&dev->mutex){....}, at: device_lock include/linux/device.h:1207 [inline] #4: 000000009711a749 (&dev->mutex){....}, at: __device_attach+0x80/0x3a0 drivers/base/dd.c:821 #5: 000000005cba2f71 (&dev->mutex){....}, at: device_lock include/linux/device.h:1207 [inline] #5: 000000005cba2f71 (&dev->mutex){....}, at: __device_attach+0x80/0x3a0 drivers/base/dd.c:821 6 locks held by kworker/1:5/5532: #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: __write_once_size include/linux/compiler.h:220 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:855 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:40 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: set_work_data kernel/workqueue.c:619 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline] #0: 000000004a979eed ((wq_completion)usb_hub_wq){+.+.}, at: process_one_work+0x81f/0x1580 kernel/workqueue.c:2240 #1: 00000000b8668093 ((work_completion)(&hub->events)){+.+.}, at: process_one_work+0x853/0x1580 kernel/workqueue.c:2244 #2: 000000005d9cd013 (&dev->mutex){....}, at: device_lock include/linux/device.h:1207 [inline] #2: 000000005d9cd013 (&dev->mutex){....}, at: hub_event+0x18a/0x3b00 drivers/usb/core/hub.c:5378 #3: 00000000055800bd (&dev->mutex){....}, at: device_lock include/linux/device.h:1207 [inline] #3: 00000000055800bd (&dev->mutex){....}, at: __device_attach+0x80/0x3a0 drivers/base/dd.c:821 #4: 000000000d215861 (&dev->mutex){....}, at: device_lock include/linux/device.h:1207 [inline] #4: 000000000d215861 (&dev->mutex){....}, at: __device_attach+0x80/0x3a0 drivers/base/dd.c:821 #5: 00000000ef616325 (&dev->mutex){....}, at: device_lock include/linux/device.h:1207 [inline] #5: 00000000ef616325 (&dev->mutex){....}, at: __device_attach+0x80/0x3a0 drivers/base/dd.c:821 ============================================= NMI backtrace for cpu 1 CPU: 1 PID: 23 Comm: khungtaskd Not tainted 5.1.0-rc4-319354-g9a33b36 #3 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0xe8/0x16e lib/dump_stack.c:113 nmi_cpu_backtrace.cold+0x48/0x87 lib/nmi_backtrace.c:101 nmi_trigger_cpumask_backtrace+0x1a6/0x1bd lib/nmi_backtrace.c:62 trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline] check_hung_uninterruptible_tasks kernel/hung_task.c:204 [inline] watchdog+0x98e/0xe20 kernel/hung_task.c:288 kthread+0x313/0x420 kernel/kthread.c:253 ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:352 Sending NMI from CPU 1 to CPUs 0: NMI backtrace for cpu 0 skipped: idling at native_safe_halt+0x2/0x10 arch/x86/include/asm/irqflags.h:57