syzbot


INFO: task hung in process_one_work (2)

Status: auto-closed as invalid on 2019/10/25 10:51
Subsystems: net bpf
[Documentation on labels]
First crash: 1745d, last: 1745d
Similar bugs (3)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: task hung in process_one_work trace 1 2181d 2181d 0/26 auto-closed as invalid on 2019/02/22 10:22
upstream INFO: task hung in process_one_work (4) net syz error error 49 30d 1316d 0/26 upstream: reported syz repro on 2020/08/21 00:08
upstream INFO: task hung in process_one_work (3) net 1 1581d 1581d 0/26 closed as invalid on 2019/11/30 16:54

Sample crash report:
INFO: task kworker/1:0:17 blocked for more than 143 seconds.
      Not tainted 5.2.0-rc2+ #24
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kworker/1:0     D26112    17      2 0x80004000
Workqueue: events cpu_map_kthread_stop
Call Trace:
 context_switch kernel/sched/core.c:2818 [inline]
 __schedule+0x7cb/0x1560 kernel/sched/core.c:3445
 schedule+0xa8/0x260 kernel/sched/core.c:3509
 schedule_timeout+0x717/0xc50 kernel/time/timer.c:1783
 do_wait_for_common kernel/sched/completion.c:83 [inline]
 __wait_for_common kernel/sched/completion.c:104 [inline]
 wait_for_common kernel/sched/completion.c:115 [inline]
 wait_for_completion+0x29c/0x440 kernel/sched/completion.c:136
 kthread_stop+0x10b/0x6c0 kernel/kthread.c:559
 cpu_map_kthread_stop+0x36/0x40 kernel/bpf/cpumap.c:158
 process_one_work+0x989/0x1790 kernel/workqueue.c:2269
 worker_thread+0x98/0xe40 kernel/workqueue.c:2415
 kthread+0x354/0x420 kernel/kthread.c:255
 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352

Showing all locks held in the system:
2 locks held by kworker/0:1/12:
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: __write_once_size include/linux/compiler.h:221 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:855 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:40 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: set_work_data kernel/workqueue.c:620 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:647 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: process_one_work+0x87e/0x1790 kernel/workqueue.c:2240
 #1: 000000007304179b ((work_completion)(&map->work)){+.+.}, at: process_one_work+0x8b4/0x1790 kernel/workqueue.c:2244
2 locks held by kworker/1:0/17:
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: __write_once_size include/linux/compiler.h:221 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:855 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:40 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: set_work_data kernel/workqueue.c:620 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:647 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: process_one_work+0x87e/0x1790 kernel/workqueue.c:2240
 #1: 00000000cd22c4b3 ((work_completion)(&old_rcpu->kthread_stop_wq)){+.+.}, at: process_one_work+0x8b4/0x1790 kernel/workqueue.c:2244
1 lock held by khungtaskd/1042:
 #0: 0000000066a2615d (rcu_read_lock){....}, at: debug_show_all_locks+0x5f/0x27e kernel/locking/lockdep.c:5149
3 locks held by kworker/1:2/2501:
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: __write_once_size include/linux/compiler.h:221 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:855 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:40 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: set_work_data kernel/workqueue.c:620 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:647 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: process_one_work+0x87e/0x1790 kernel/workqueue.c:2240
 #1: 0000000085a00f40 ((work_completion)(&map->work)){+.+.}, at: process_one_work+0x8b4/0x1790 kernel/workqueue.c:2244
 #2: 0000000092c8674e (rcu_state.barrier_mutex){+.+.}, at: rcu_barrier+0x47/0x2f0 kernel/rcu/tree.c:2719
6 locks held by udevd/3879:
1 lock held by rsyslogd/8312:
2 locks held by getty/8402:
 #0: 000000008690d1d2 (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341
 #1: 00000000249bc56a (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x232/0x1b70 drivers/tty/n_tty.c:2156
2 locks held by getty/8403:
 #0: 00000000dc89196f (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341
 #1: 000000004b5282e6 (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x232/0x1b70 drivers/tty/n_tty.c:2156
2 locks held by getty/8404:
 #0: 00000000c1365e79 (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341
 #1: 000000000168a3f5 (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x232/0x1b70 drivers/tty/n_tty.c:2156
2 locks held by getty/8405:
 #0: 00000000d6a8da76 (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341
 #1: 000000002aece6bd (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x232/0x1b70 drivers/tty/n_tty.c:2156
2 locks held by getty/8406:
 #0: 000000009cb9de3d (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341
 #1: 00000000833a3db1 (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x232/0x1b70 drivers/tty/n_tty.c:2156
2 locks held by getty/8407:
 #0: 00000000cc8124cf (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341
 #1: 0000000003c61715 (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x232/0x1b70 drivers/tty/n_tty.c:2156
2 locks held by getty/8408:
 #0: 0000000093b436f7 (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341
 #1: 000000000192e824 (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x232/0x1b70 drivers/tty/n_tty.c:2156
2 locks held by kworker/0:3/8451:
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: __write_once_size include/linux/compiler.h:221 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:855 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:40 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: set_work_data kernel/workqueue.c:620 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:647 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: process_one_work+0x87e/0x1790 kernel/workqueue.c:2240
 #1: 00000000d3f04f00 ((work_completion)(&map->work)){+.+.}, at: process_one_work+0x8b4/0x1790 kernel/workqueue.c:2244
3 locks held by kworker/1:5/22047:
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: __write_once_size include/linux/compiler.h:221 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:855 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:40 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: set_work_data kernel/workqueue.c:620 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:647 [inline]
 #0: 000000005c0a70a3 ((wq_completion)events){+.+.}, at: process_one_work+0x87e/0x1790 kernel/workqueue.c:2240
 #1: 00000000d9fe4b62 ((work_completion)(&map->work)){+.+.}, at: process_one_work+0x8b4/0x1790 kernel/workqueue.c:2244
 #2: 0000000092c8674e (rcu_state.barrier_mutex){+.+.}, at: rcu_barrier+0x47/0x2f0 kernel/rcu/tree.c:2719
1 lock held by syz-executor.4/22101:
 #0: 000000003b76c6a6 (event_mutex){+.+.}, at: perf_trace_destroy+0x28/0x100 kernel/trace/trace_event_perf.c:236

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 1042 Comm: khungtaskd Not tainted 5.2.0-rc2+ #24
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x172/0x1f0 lib/dump_stack.c:113
 nmi_cpu_backtrace.cold+0x63/0xa4 lib/nmi_backtrace.c:101
 nmi_trigger_cpumask_backtrace+0x1be/0x236 lib/nmi_backtrace.c:62
 arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38
 trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:205 [inline]
 watchdog+0x9b7/0xec0 kernel/hung_task.c:289
 kthread+0x354/0x420 kernel/kthread.c:255
 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 9528 Comm: kworker/u4:6 Not tainted 5.2.0-rc2+ #24
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: bat_events batadv_nc_worker
RIP: 0010:preempt_count arch/x86/include/asm/preempt.h:26 [inline]
RIP: 0010:check_kcov_mode kernel/kcov.c:68 [inline]
RIP: 0010:write_comp_data+0x9/0x70 kernel/kcov.c:123
Code: 12 00 00 8b 80 e4 12 00 00 48 8b 11 48 83 c2 01 48 39 d0 76 07 48 89 34 d1 48 89 11 5d c3 0f 1f 00 65 4c 8b 04 25 c0 fd 01 00 <65> 8b 05 68 57 91 7e a9 00 01 1f 00 75 51 41 8b 80 e0 12 00 00 83
RSP: 0018:ffff88809ad1fcc8 EFLAGS: 00000246
RAX: 1ffff11011d20381 RBX: ffff8880a7bcfac0 RCX: ffffffff87016daa
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000007
RBP: ffff88809ad1fcd0 R08: ffff88809b4fa140 R09: ffffed1015d26be8
R10: ffffed1015d26be7 R11: ffff8880ae935f3b R12: 0000000000000001
R13: 00000000000000d9 R14: 0000000000000000 R15: dffffc0000000000
FS:  0000000000000000(0000) GS:ffff8880ae900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000159a9b0 CR3: 0000000095170000 CR4: 00000000001406e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
Call Trace:
 batadv_nc_purge_orig_hash net/batman-adv/network-coding.c:408 [inline]
 batadv_nc_worker+0x1ba/0x760 net/batman-adv/network-coding.c:718
 process_one_work+0x989/0x1790 kernel/workqueue.c:2269
 worker_thread+0x98/0xe40 kernel/workqueue.c:2415
 kthread+0x354/0x420 kernel/kthread.c:255
 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352

Crashes (1):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2019/06/18 16:13 bpf-next 4d18f6de6ac1 e3f76baa .config console log report ci-upstream-bpf-next-kasan-gce
* Struck through repros no longer work on HEAD.