syzbot


INFO: task hung in flush_workqueue (2)

Status: auto-closed as invalid on 2019/09/24 07:48
Subsystems: afs net
[Documentation on labels]
First crash: 1837d, last: 1828d
Similar bugs (1)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: task hung in flush_workqueue media C 293 1923d 1974d 11/26 fixed on 2019/01/11 01:22

Sample crash report:
INFO: task kworker/u4:12:24198 blocked for more than 143 seconds.
      Not tainted 5.1.0-rc2+ #40
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kworker/u4:12   D24616 24198      2 0x80000000
Workqueue: netns cleanup_net
Call Trace:
 context_switch kernel/sched/core.c:2877 [inline]
 __schedule+0x817/0x1cc0 kernel/sched/core.c:3518
 schedule+0x92/0x180 kernel/sched/core.c:3562
 schedule_timeout+0x8ca/0xfd0 kernel/time/timer.c:1779
 do_wait_for_common kernel/sched/completion.c:83 [inline]
 __wait_for_common kernel/sched/completion.c:104 [inline]
 wait_for_common kernel/sched/completion.c:115 [inline]
 wait_for_completion+0x29c/0x440 kernel/sched/completion.c:136
 flush_workqueue+0x40f/0x14c0 kernel/workqueue.c:2826
 rxrpc_destroy_all_connections+0x103/0x487 net/rxrpc/conn_object.c:472
 rxrpc_exit_net+0x1c2/0x310 net/rxrpc/net_ns.c:126
 ops_exit_list.isra.0+0xb0/0x160 net/core/net_namespace.c:153
 cleanup_net+0x3fb/0x960 net/core/net_namespace.c:551
 process_one_work+0x98e/0x1790 kernel/workqueue.c:2269
 worker_thread+0x98/0xe40 kernel/workqueue.c:2415
 kthread+0x357/0x430 kernel/kthread.c:253
 ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:352
INFO: task syz-executor.0:10189 blocked for more than 143 seconds.
      Not tainted 5.1.0-rc2+ #40
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
syz-executor.0  D28248 10189   8071 0x00000004
Call Trace:
 context_switch kernel/sched/core.c:2877 [inline]
 __schedule+0x817/0x1cc0 kernel/sched/core.c:3518
 schedule+0x92/0x180 kernel/sched/core.c:3562
 synchronize_rcu_expedited+0x431/0x5b0 kernel/rcu/tree_exp.h:775
 synchronize_net+0x3b/0x60 net/core/dev.c:9247
 packet_release+0x806/0xbf0 net/packet/af_packet.c:3013
 __sock_release+0xd3/0x2b0 net/socket.c:599
 sock_close+0x1b/0x30 net/socket.c:1247
 __fput+0x2e5/0x8d0 fs/file_table.c:278
 ____fput+0x16/0x20 fs/file_table.c:309
 task_work_run+0x14a/0x1c0 kernel/task_work.c:113
 tracehook_notify_resume include/linux/tracehook.h:188 [inline]
 exit_to_usermode_loop+0x273/0x2c0 arch/x86/entry/common.c:166
 prepare_exit_to_usermode arch/x86/entry/common.c:197 [inline]
 syscall_return_slowpath arch/x86/entry/common.c:268 [inline]
 do_syscall_64+0x52d/0x610 arch/x86/entry/common.c:293
 entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x411fc1
Code: ff 85 c0 75 0c 48 c7 03 c0 22 41 00 e9 a6 fe ff ff be 4c 37 44 00 48 89 ef e8 1b 94 ff ff 85 c0 75 0c 48 c7 03 f0 22 41 00 e9 <89> fe ff ff 31 c0 48 89 ee bf 30 31 44 00 e8 5c f0 00 00 b8 14 fc
RSP: 002b:00007fffab471bd0 EFLAGS: 00000293 ORIG_RAX: 0000000000000003
RAX: 0000000000000000 RBX: 0000000000000005 RCX: 0000000000411fc1
RDX: 0000000000000000 RSI: 0000000000740338 RDI: 0000000000000004
RBP: 0000000000000000 R08: 000000000011702f R09: 000000000011702f
R10: 00007fffab471b00 R11: 0000000000000293 R12: 0000000000000001
R13: 00007fffab471c10 R14: 0000000000000000 R15: 00007fffab471c20

Showing all locks held in the system:
3 locks held by kworker/1:0/17:
1 lock held by khungtaskd/1042:
 #0: 0000000041b6cb70 (rcu_read_lock){....}, at: debug_show_all_locks+0x5f/0x27e kernel/locking/lockdep.c:5061
1 lock held by rsyslogd/7920:
 #0: 00000000f66fc987 (&f->f_pos_lock){+.+.}, at: __fdget_pos+0xee/0x110 fs/file.c:801
2 locks held by getty/8032:
 #0: 00000000d8f4be5e (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341
 #1: 00000000b5ef590a (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x232/0x1b70 drivers/tty/n_tty.c:2156
2 locks held by getty/8033:
 #0: 000000006c69525e (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341
 #1: 000000002cabe4ee (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x232/0x1b70 drivers/tty/n_tty.c:2156
2 locks held by getty/8034:
 #0: 000000000ded5bbe (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341
 #1: 0000000077aabd92 (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x232/0x1b70 drivers/tty/n_tty.c:2156
2 locks held by getty/8035:
 #0: 000000004bc1fc8a (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341
 #1: 000000008e98c34d (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x232/0x1b70 drivers/tty/n_tty.c:2156
2 locks held by getty/8036:
 #0: 0000000002d01d95 (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341
 #1: 000000002c9ce229 (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x232/0x1b70 drivers/tty/n_tty.c:2156
2 locks held by getty/8037:
 #0: 000000007d5593cb (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341
 #1: 000000001665675a (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x232/0x1b70 drivers/tty/n_tty.c:2156
2 locks held by getty/8038:
 #0: 00000000c10f3b0a (&tty->ldisc_sem){++++}, at: ldsem_down_read+0x33/0x40 drivers/tty/tty_ldsem.c:341
 #1: 00000000e14dca32 (&ldata->atomic_read_lock){+.+.}, at: n_tty_read+0x232/0x1b70 drivers/tty/n_tty.c:2156
2 locks held by syz-executor.0/15426:
 #0: 000000002ebf64ed (&bdev->bd_mutex){+.+.}, at: __blkdev_put+0xbb/0x810 fs/block_dev.c:1833
 #1: 00000000b001bb95 (&nbd->config_lock){+.+.}, at: refcount_dec_and_mutex_lock lib/refcount.c:319 [inline]
 #1: 00000000b001bb95 (&nbd->config_lock){+.+.}, at: refcount_dec_and_mutex_lock+0x56/0x90 lib/refcount.c:314
2 locks held by syz-executor.0/15685:
 #0: 000000004e0a12a2 (rtnl_mutex){+.+.}, at: rtnl_lock+0x17/0x20 net/core/rtnetlink.c:76
 #1: 00000000c451bac8 (rcu_state.exp_mutex){+.+.}, at: exp_funnel_lock kernel/rcu/tree_exp.h:317 [inline]
 #1: 00000000c451bac8 (rcu_state.exp_mutex){+.+.}, at: synchronize_rcu_expedited+0x26f/0x5b0 kernel/rcu/tree_exp.h:758
3 locks held by kworker/u4:12/24198:
 #0: 000000005bb42399 ((wq_completion)netns){+.+.}, at: __write_once_size include/linux/compiler.h:220 [inline]
 #0: 000000005bb42399 ((wq_completion)netns){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
 #0: 000000005bb42399 ((wq_completion)netns){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:855 [inline]
 #0: 000000005bb42399 ((wq_completion)netns){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:40 [inline]
 #0: 000000005bb42399 ((wq_completion)netns){+.+.}, at: set_work_data kernel/workqueue.c:619 [inline]
 #0: 000000005bb42399 ((wq_completion)netns){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:646 [inline]
 #0: 000000005bb42399 ((wq_completion)netns){+.+.}, at: process_one_work+0x87e/0x1790 kernel/workqueue.c:2240
 #1: 000000005d369d79 (net_cleanup_work){+.+.}, at: process_one_work+0x8b4/0x1790 kernel/workqueue.c:2244
 #2: 000000007b9f6bed (pernet_ops_rwsem){++++}, at: cleanup_net+0xae/0x960 net/core/net_namespace.c:518
2 locks held by syz-executor.0/10189:
 #0: 000000002d39be9f (&sb->s_type->i_mutex_key#11){+.+.}, at: inode_lock include/linux/fs.h:769 [inline]
 #0: 000000002d39be9f (&sb->s_type->i_mutex_key#11){+.+.}, at: __sock_release+0x89/0x2b0 net/socket.c:598
 #1: 00000000c451bac8 (rcu_state.exp_mutex){+.+.}, at: exp_funnel_lock kernel/rcu/tree_exp.h:285 [inline]
 #1: 00000000c451bac8 (rcu_state.exp_mutex){+.+.}, at: synchronize_rcu_expedited+0x4ab/0x5b0 kernel/rcu/tree_exp.h:758

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 1042 Comm: khungtaskd Not tainted 5.1.0-rc2+ #40
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x172/0x1f0 lib/dump_stack.c:113
 nmi_cpu_backtrace.cold+0x63/0xa4 lib/nmi_backtrace.c:101
 nmi_trigger_cpumask_backtrace+0x1be/0x236 lib/nmi_backtrace.c:62
 arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38
 trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:204 [inline]
 watchdog+0x9b7/0xec0 kernel/hung_task.c:288
 kthread+0x357/0x430 kernel/kthread.c:253
 ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:352
Sending NMI from CPU 0 to CPUs 1:

Crashes (2):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2019/03/28 07:34 upstream 1a9df9e29c2a f94f56fe .config console log report ci-upstream-kasan-gce-root
2019/03/19 12:47 upstream 9e98c678c2d6 e4549234 .config console log report ci-upstream-kasan-gce-root
* Struck through repros no longer work on HEAD.