syzbot


possible deadlock in flush_workqueue

Status: auto-closed as invalid on 2020/02/25 05:02
Reported-by: syzbot+91a7242c038b5c070eb9@syzkaller.appspotmail.com
First crash: 1702d, last: 1691d
Similar bugs (7)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-5.15 possible deadlock in flush_workqueue 1 430d 430d 0/3 auto-obsoleted due to no activity on 2023/08/09 18:38
android-414 possible deadlock in flush_workqueue 1 1677d 1677d 0/1 auto-closed as invalid on 2020/03/10 11:29
linux-5.15 possible deadlock in flush_workqueue (2) 16 10h13m 93d 0/3 upstream: reported on 2024/03/13 19:04
upstream possible deadlock in flush_workqueue (2) C done done 256 1630d 2064d 15/27 fixed on 2020/01/31 18:49
linux-4.14 possible deadlock in flush_workqueue C done 15 1670d 1763d 1/1 fixed on 2019/12/18 17:48
linux-4.14 possible deadlock in flush_workqueue (2) 3 1634d 1638d 0/1 auto-closed as invalid on 2020/04/22 20:54
upstream possible deadlock in flush_workqueue net C 73762 2080d 2122d 11/27 fixed on 2018/10/11 14:33

Sample crash report:
block nbd4: Receive control failed (result -22)
block nbd4: shutting down sockets
============================================
WARNING: possible recursive locking detected
4.19.80 #0 Not tainted
--------------------------------------------
kworker/u5:1/8224 is trying to acquire lock:
000000007b22e084 ((wq_completion)"knbd%d-recv"nbd->index){+.+.}, at: flush_workqueue+0xf7/0x14b0 kernel/workqueue.c:2652

but task is already holding lock:
000000007b22e084 ((wq_completion)"knbd%d-recv"nbd->index){+.+.}, at: __write_once_size include/linux/compiler.h:220 [inline]
000000007b22e084 ((wq_completion)"knbd%d-recv"nbd->index){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
000000007b22e084 ((wq_completion)"knbd%d-recv"nbd->index){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:40 [inline]
000000007b22e084 ((wq_completion)"knbd%d-recv"nbd->index){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:59 [inline]
000000007b22e084 ((wq_completion)"knbd%d-recv"nbd->index){+.+.}, at: set_work_data kernel/workqueue.c:617 [inline]
000000007b22e084 ((wq_completion)"knbd%d-recv"nbd->index){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:644 [inline]
000000007b22e084 ((wq_completion)"knbd%d-recv"nbd->index){+.+.}, at: process_one_work+0x87e/0x1750 kernel/workqueue.c:2124

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock((wq_completion)"knbd%d-recv"nbd->index);
  lock((wq_completion)"knbd%d-recv"nbd->index);

 *** DEADLOCK ***

 May be due to missing lock nesting notation

3 locks held by kworker/u5:1/8224:
 #0: 000000007b22e084 ((wq_completion)"knbd%d-recv"nbd->index){+.+.}, at: __write_once_size include/linux/compiler.h:220 [inline]
 #0: 000000007b22e084 ((wq_completion)"knbd%d-recv"nbd->index){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
 #0: 000000007b22e084 ((wq_completion)"knbd%d-recv"nbd->index){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:40 [inline]
 #0: 000000007b22e084 ((wq_completion)"knbd%d-recv"nbd->index){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:59 [inline]
 #0: 000000007b22e084 ((wq_completion)"knbd%d-recv"nbd->index){+.+.}, at: set_work_data kernel/workqueue.c:617 [inline]
 #0: 000000007b22e084 ((wq_completion)"knbd%d-recv"nbd->index){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:644 [inline]
 #0: 000000007b22e084 ((wq_completion)"knbd%d-recv"nbd->index){+.+.}, at: process_one_work+0x87e/0x1750 kernel/workqueue.c:2124
 #1: 00000000459f9595 ((work_completion)(&args->work)){+.+.}, at: process_one_work+0x8b4/0x1750 kernel/workqueue.c:2128
 #2: 000000007f0bb026 (&nbd->config_lock){+.+.}, at: refcount_dec_and_mutex_lock lib/refcount.c:311 [inline]
 #2: 000000007f0bb026 (&nbd->config_lock){+.+.}, at: refcount_dec_and_mutex_lock+0x56/0x90 lib/refcount.c:306

stack backtrace:
CPU: 1 PID: 8224 Comm: kworker/u5:1 Not tainted 4.19.80 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: knbd4-recv recv_work
Call Trace:
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x172/0x1f0 lib/dump_stack.c:113
 print_deadlock_bug kernel/locking/lockdep.c:1759 [inline]
 check_deadlock kernel/locking/lockdep.c:1803 [inline]
 validate_chain kernel/locking/lockdep.c:2399 [inline]
 __lock_acquire.cold+0x20f/0x4a7 kernel/locking/lockdep.c:3411
 lock_acquire+0x16f/0x3f0 kernel/locking/lockdep.c:3903
 flush_workqueue+0x126/0x14b0 kernel/workqueue.c:2655
 drain_workqueue+0x1b4/0x470 kernel/workqueue.c:2820
 destroy_workqueue+0x21/0x6b0 kernel/workqueue.c:4158
 nbd_config_put+0x3cf/0x860 drivers/block/nbd.c:1132
 recv_work+0x19b/0x200 drivers/block/nbd.c:740
 process_one_work+0x989/0x1750 kernel/workqueue.c:2153
 worker_thread+0x98/0xe40 kernel/workqueue.c:2296
 kthread+0x354/0x420 kernel/kthread.c:246
 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:415
kobject: 'loop1' (00000000ed1e4b23): kobject_uevent_env
kobject: 'loop1' (00000000ed1e4b23): fill_kobj_path: path = '/devices/virtual/block/loop1'
kobject: 'loop2' (0000000096c35d3e): kobject_uevent_env
kobject: 'loop2' (0000000096c35d3e): fill_kobj_path: path = '/devices/virtual/block/loop2'
netlink: 'syz-executor.3': attribute type 2 has an invalid length.
kobject: 'loop0' (0000000075bee5ea): kobject_uevent_env
kobject: 'loop0' (0000000075bee5ea): fill_kobj_path: path = '/devices/virtual/block/loop0'
overlayfs: unrecognized mount option ";Ì·-†u¬¸p" or missing value
nf_conntrack: default automatic helper assignment has been turned off for security reasons and CT-based  firewall rule not found. Use the iptables CT target to attach helpers instead.
TCP: request_sock_TCP: Possible SYN flooding on port 20000. Sending cookies.  Check SNMP counters.
kobject: 'loop2' (0000000096c35d3e): kobject_uevent_env
kobject: 'loop2' (0000000096c35d3e): fill_kobj_path: path = '/devices/virtual/block/loop2'
audit: type=1804 audit(1572238871.517:56): pid=8262 uid=0 auid=4294967295 ses=4294967295 subj=unconfined_u:system_r:insmod_t:s0-s0:c0.c1023 op=invalid_pcr cause=open_writers comm="syz-executor.1" name="/root/syzkaller-testdir978770137/syzkaller.b7Owe0/19/bus" dev="sda1" ino=16606 res=1
overlayfs: unrecognized mount option ";Ì·-†u¬¸p" or missing value
kobject: 'loop3' (000000009f250bac): kobject_uevent_env
kobject: 'loop3' (000000009f250bac): fill_kobj_path: path = '/devices/virtual/block/loop3'
kobject: 'loop5' (000000000a7cc420): kobject_uevent_env
kobject: 'loop5' (000000000a7cc420): fill_kobj_path: path = '/devices/virtual/block/loop5'
netlink: 'syz-executor.3': attribute type 2 has an invalid length.
kobject: 'loop2' (0000000096c35d3e): kobject_uevent_env
TCP: request_sock_TCP: Possible SYN flooding on port 20000. Sending cookies.  Check SNMP counters.
kobject: 'loop2' (0000000096c35d3e): fill_kobj_path: path = '/devices/virtual/block/loop2'
kobject: 'loop0' (0000000075bee5ea): kobject_uevent_env
kobject: 'loop0' (0000000075bee5ea): fill_kobj_path: path = '/devices/virtual/block/loop0'
overlayfs: unrecognized mount option ";Ì·-†u¬¸p" or missing value
kobject: 'loop1' (00000000ed1e4b23): kobject_uevent_env
kobject: 'loop1' (00000000ed1e4b23): fill_kobj_path: path = '/devices/virtual/block/loop1'
audit: type=1804 audit(1572238871.987:57): pid=8284 uid=0 auid=4294967295 ses=4294967295 subj=unconfined_u:system_r:insmod_t:s0-s0:c0.c1023 op=invalid_pcr cause=open_writers comm="syz-executor.1" name="/root/syzkaller-testdir978770137/syzkaller.b7Owe0/20/bus" dev="sda1" ino=16597 res=1
kobject: 'loop2' (0000000096c35d3e): kobject_uevent_env
kobject: 'loop2' (0000000096c35d3e): fill_kobj_path: path = '/devices/virtual/block/loop2'
kobject: 'loop0' (0000000075bee5ea): kobject_uevent_env
kobject: 'loop0' (0000000075bee5ea): fill_kobj_path: path = '/devices/virtual/block/loop0'
kobject: 'loop3' (000000009f250bac): kobject_uevent_env
kobject: 'loop3' (000000009f250bac): fill_kobj_path: path = '/devices/virtual/block/loop3'
kobject: 'loop1' (00000000ed1e4b23): kobject_uevent_env
kobject: 'loop1' (00000000ed1e4b23): fill_kobj_path: path = '/devices/virtual/block/loop1'

Crashes (3):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2019/10/28 05:01 linux-4.19.y c3038e718a19 25bb509e .config console log report ci2-linux-4-19
2019/10/20 20:34 linux-4.19.y c3038e718a19 8c88c9c1 .config console log report ci2-linux-4-19
2019/10/17 11:57 linux-4.19.y dafd634415a7 8c88c9c1 .config console log report ci2-linux-4-19
* Struck through repros no longer work on HEAD.