syzbot


INFO: task hung in rdma_dev_exit_net (5)

Status: auto-obsoleted due to no activity on 2024/04/21 00:36
Subsystems: rdma
[Documentation on labels]
First crash: 121d, last: 107d
Similar bugs (4)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: task hung in rdma_dev_exit_net (2) rdma 6 1183d 1317d 0/26 auto-closed as invalid on 2021/05/10 22:49
upstream INFO: task hung in rdma_dev_exit_net (3) rdma 5 931d 966d 0/26 auto-closed as invalid on 2022/01/17 19:05
upstream INFO: task hung in rdma_dev_exit_net rdma 1 1476d 1476d 0/26 auto-closed as invalid on 2020/07/21 12:21
upstream INFO: task hung in rdma_dev_exit_net (4) rdma 7 305d 467d 0/26 auto-obsoleted due to no activity on 2023/10/05 19:47

Sample crash report:
INFO: task kworker/u4:17:13842 blocked for more than 143 seconds.
      Not tainted 6.7.0-syzkaller-13004-g4fbbed787267 #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/u4:17   state:D stack:25024 pid:13842 tgid:13842 ppid:2      flags:0x00004000
Workqueue: netns cleanup_net
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5400 [inline]
 __schedule+0xf12/0x5c00 kernel/sched/core.c:6727
 __schedule_loop kernel/sched/core.c:6802 [inline]
 schedule+0xe9/0x270 kernel/sched/core.c:6817
 schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:6874
 rwsem_down_write_slowpath kernel/locking/rwsem.c:1178 [inline]
 __down_write_common+0x948/0x1400 kernel/locking/rwsem.c:1306
 rdma_dev_exit_net+0x178/0x590 drivers/infiniband/core/device.c:1123
 ops_exit_list+0xb0/0x170 net/core/net_namespace.c:170
 cleanup_net+0x505/0xb20 net/core/net_namespace.c:618
 process_one_work+0x886/0x15d0 kernel/workqueue.c:2633
 process_scheduled_works kernel/workqueue.c:2706 [inline]
 worker_thread+0x8b9/0x1290 kernel/workqueue.c:2787
 kthread+0x2c6/0x3a0 kernel/kthread.c:388
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242
 </TASK>

Showing all locks held in the system:
1 lock held by khungtaskd/29:
 #0: ffffffff8d1a9120 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:298 [inline]
 #0: ffffffff8d1a9120 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:750 [inline]
 #0: ffffffff8d1a9120 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x75/0x340 kernel/locking/lockdep.c:6614
2 locks held by getty/4807:
 #0: ffff888028a600a0 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x24/0x80 drivers/tty/tty_ldisc.c:243
 #1: ffffc900015b02f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0xfc6/0x1490 drivers/tty/n_tty.c:2201
4 locks held by kworker/u4:17/13842:
 #0: ffff88801308bd38 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0x789/0x15d0 kernel/workqueue.c:2608
 #1: ffffc90003abfd80 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0x7eb/0x15d0 kernel/workqueue.c:2609
 #2: ffffffff8eca15d0 (pernet_ops_rwsem){++++}-{3:3}, at: cleanup_net+0x9f/0xb20 net/core/net_namespace.c:580
 #3: ffffffff8e97cf30 (rdma_nets_rwsem){++++}-{3:3}, at: rdma_dev_exit_net+0x178/0x590 drivers/infiniband/core/device.c:1123
2 locks held by kworker/u4:14/26658:
3 locks held by kworker/u4:20/26686:
6 locks held by syz-executor.3/27626:
3 locks held by syz-executor.2/27664:
 #0: ffffffff8eca15d0 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x2cc/0x660 net/core/net_namespace.c:491
 #1: ffffffff8e97d1b0 (devices_rwsem){++++}-{3:3}, at: rdma_dev_init_net+0x234/0x520 drivers/infiniband/core/device.c:1179
 #2: ffffffff8e97cf30 (rdma_nets_rwsem){++++}-{3:3}, at: rdma_dev_init_net+0x2a7/0x520 drivers/infiniband/core/device.c:1184
3 locks held by syz-executor.5/27672:
 #0: ffffffff8eca15d0 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x2cc/0x660 net/core/net_namespace.c:491
 #1: ffffffff8e97d1b0 (devices_rwsem){++++}-{3:3}, at: rdma_dev_init_net+0x234/0x520 drivers/infiniband/core/device.c:1179
 #2: ffffffff8e97cf30 (rdma_nets_rwsem){++++}-{3:3}, at: rdma_dev_init_net+0x2a7/0x520 drivers/infiniband/core/device.c:1184
3 locks held by syz-executor.5/27713:
 #0: ffffffff8eca15d0 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x2cc/0x660 net/core/net_namespace.c:491
 #1: ffffffff8e97d1b0 (devices_rwsem){++++}-{3:3}, at: rdma_dev_init_net+0x234/0x520 drivers/infiniband/core/device.c:1179
 #2: ffffffff8e97cf30 (rdma_nets_rwsem){++++}-{3:3}, at: rdma_dev_init_net+0x2a7/0x520 drivers/infiniband/core/device.c:1184
3 locks held by syz-executor.2/27721:
 #0: ffffffff8eca15d0 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x2cc/0x660 net/core/net_namespace.c:491
 #1: ffffffff8e97d1b0 (devices_rwsem){++++}-{3:3}, at: rdma_dev_init_net+0x234/0x520 drivers/infiniband/core/device.c:1179
 #2: ffffffff8e97cf30 (rdma_nets_rwsem){++++}-{3:3}, at: rdma_dev_init_net+0x2a7/0x520 drivers/infiniband/core/device.c:1184
3 locks held by syz-executor.3/27725:
 #0: ffffffff8eca15d0 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x2cc/0x660 net/core/net_namespace.c:491
 #1: ffffffff8e97d1b0 (devices_rwsem){++++}-{3:3}, at: rdma_dev_init_net+0x234/0x520 drivers/infiniband/core/device.c:1179
 #2: ffffffff8e97cf30 (rdma_nets_rwsem){++++}-{3:3}, at: rdma_dev_init_net+0x2a7/0x520 drivers/infiniband/core/device.c:1184
3 locks held by syz-executor.5/27858:
 #0: ffffffff8eca15d0 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x2cc/0x660 net/core/net_namespace.c:491
 #1: ffffffff8e97d1b0 (devices_rwsem){++++}-{3:3}, at: rdma_dev_init_net+0x234/0x520 drivers/infiniband/core/device.c:1179
 #2: ffffffff8e97cf30 (rdma_nets_rwsem){++++}-{3:3}, at: rdma_dev_init_net+0x2a7/0x520 drivers/infiniband/core/device.c:1184
3 locks held by syz-executor.2/27871:
 #0: ffffffff8eca15d0 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x2cc/0x660 net/core/net_namespace.c:491
 #1: ffffffff8e97d1b0 (devices_rwsem){++++}-{3:3}, at: rdma_dev_init_net+0x234/0x520 drivers/infiniband/core/device.c:1179
 #2: ffffffff8e97cf30 (rdma_nets_rwsem){++++}-{3:3}, at: rdma_dev_init_net+0x2a7/0x520 drivers/infiniband/core/device.c:1184
3 locks held by syz-executor.3/27873:
 #0: ffffffff8eca15d0 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x2cc/0x660 net/core/net_namespace.c:491
 #1: ffffffff8e97d1b0 (devices_rwsem){++++}-{3:3}, at: rdma_dev_init_net+0x234/0x520 drivers/infiniband/core/device.c:1179
 #2: ffffffff8e97cf30 (rdma_nets_rwsem){++++}-{3:3}, at: rdma_dev_init_net+0x2a7/0x520 drivers/infiniband/core/device.c:1184

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 29 Comm: khungtaskd Not tainted 6.7.0-syzkaller-13004-g4fbbed787267 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xd9/0x1b0 lib/dump_stack.c:106
 nmi_cpu_backtrace+0x277/0x390 lib/nmi_backtrace.c:113
 nmi_trigger_cpumask_backtrace+0x299/0x300 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:160 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:222 [inline]
 watchdog+0xf87/0x1210 kernel/hung_task.c:379
 kthread+0x2c6/0x3a0 kernel/kthread.c:388
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:242
 </TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 25290 Comm: syz-executor.3 Not tainted 6.7.0-syzkaller-13004-g4fbbed787267 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/17/2023
RIP: 0010:bytes_is_nonzero mm/kasan/generic.c:86 [inline]
RIP: 0010:memory_is_nonzero mm/kasan/generic.c:104 [inline]
RIP: 0010:memory_is_poisoned_n mm/kasan/generic.c:129 [inline]
RIP: 0010:memory_is_poisoned mm/kasan/generic.c:161 [inline]
RIP: 0010:check_region_inline mm/kasan/generic.c:180 [inline]
RIP: 0010:kasan_check_range+0xf7/0x190 mm/kasan/generic.c:189
Code: 00 00 48 85 d2 74 1e 41 83 e2 07 49 39 d1 75 0a 45 3a 11 b8 01 00 00 00 7c 0b 44 89 c2 e8 21 ed ff ff 83 f0 01 5b 5d 41 5c c3 <48> 85 d2 74 4f 48 01 ea eb 09 48 83 c0 01 48 39 d0 74 41 80 38 00
RSP: 0018:ffffc900044f7a18 EFLAGS: 00000083
RAX: fffffbfff2428812 RBX: fffffbfff2428813 RCX: ffffffff81667d2e
RDX: 0000000000000001 RSI: 0000000000000008 RDI: ffffffff92144090
RBP: fffffbfff2428812 R08: 0000000000000000 R09: fffffbfff2428812
R10: ffffffff92144097 R11: 0000000000000000 R12: ffff88803ceea880
R13: 0000000000000000 R14: 0000000000000a92 R15: 0000000000000004
FS:  00007f5f4d7e36c0(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c002e3f000 CR3: 000000001ded4000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <TASK>
 instrument_atomic_read include/linux/instrumented.h:68 [inline]
 _test_bit include/asm-generic/bitops/instrumented-non-atomic.h:141 [inline]
 hlock_class+0x4e/0x130 kernel/locking/lockdep.c:228
 lookup_chain_cache_add kernel/locking/lockdep.c:3781 [inline]
 validate_chain kernel/locking/lockdep.c:3837 [inline]
 __lock_acquire+0x1502/0x3b30 kernel/locking/lockdep.c:5137
 lock_acquire kernel/locking/lockdep.c:5754 [inline]
 lock_acquire+0x1ae/0x520 kernel/locking/lockdep.c:5719
 __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
 _raw_spin_lock_irqsave+0x3a/0x50 kernel/locking/spinlock.c:162
 __wake_up_common_lock kernel/sched/wait.c:105 [inline]
 __wake_up+0x1c/0x50 kernel/sched/wait.c:127
 io_cqring_wake io_uring/io_uring.h:248 [inline]
 io_cqring_wake io_uring/io_uring.h:235 [inline]
 io_cq_unlock_post+0xed/0x190 io_uring/io_uring.c:669
 __io_cqring_overflow_flush+0x3e2/0x570 io_uring/io_uring.c:719
 io_cqring_do_overflow_flush io_uring/io_uring.c:727 [inline]
 io_cqring_wait io_uring/io_uring.c:2670 [inline]
 __do_sys_io_uring_enter+0x1e48/0x2450 io_uring/io_uring.c:3701
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xd3/0x250 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x63/0x6b
RIP: 0033:0x7f5f4ca7cda9
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 e1 20 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f5f4d7e30c8 EFLAGS: 00000246 ORIG_RAX: 00000000000001aa
RAX: ffffffffffffffda RBX: 00007f5f4cbabf80 RCX: 00007f5f4ca7cda9
RDX: 0000000000400000 RSI: 0000000000000000 RDI: 0000000000000005
RBP: 00007f5f4cac947a R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000000
R13: 000000000000000b R14: 00007f5f4cbabf80 R15: 00007ffeec5bd668
 </TASK>

Crashes (4):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/01/22 00:20 upstream 4fbbed787267 9bd8dcda .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: task hung in rdma_dev_exit_net
2024/01/11 04:00 upstream acc657692aed 04815ef1 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: task hung in rdma_dev_exit_net
2024/01/07 06:44 net 3f14b377d01d d0304e9c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in rdma_dev_exit_net
2024/01/16 09:11 net-next 3e7aeb78ab01 2a7bcc7f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_exit_net
* Struck through repros no longer work on HEAD.