syzbot


INFO: task hung in rdma_dev_change_netns

Status: upstream: reported on 2024/09/03 23:37
Subsystems: rdma
[Documentation on labels]
Reported-by: syzbot+73c5eab674c7e1e7012e@syzkaller.appspotmail.com
First crash: 196d, last: 2d05h
Discussions (5)
Title Replies (including bot) Last reply
[syzbot] Monthly rdma report (Jan 2025) 0 (1) 2025/01/30 13:12
[syzbot] Monthly rdma report (Nov 2024) 2 (3) 2024/12/02 14:46
[syzbot] Monthly rdma report (Oct 2024) 0 (1) 2024/10/28 20:45
[syzbot] Monthly rdma report (Sep 2024) 0 (1) 2024/09/27 13:28
[syzbot] [rdma?] INFO: task hung in rdma_dev_change_netns 0 (1) 2024/09/03 23:37

Sample crash report:
INFO: task kworker/u8:28:11047 blocked for more than 143 seconds.
      Not tainted 6.13.0-syzkaller-09760-g69e858e0b8b2 #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/u8:28   state:D stack:25808 pid:11047 tgid:11047 ppid:2      task_flags:0x4208060 flags:0x00004000
Workqueue: netns cleanup_net
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5377 [inline]
 __schedule+0xf43/0x5890 kernel/sched/core.c:6764
 __schedule_loop kernel/sched/core.c:6841 [inline]
 schedule+0xe7/0x350 kernel/sched/core.c:6856
 schedule_preempt_disabled+0x13/0x30 kernel/sched/core.c:6913
 __mutex_lock_common kernel/locking/mutex.c:662 [inline]
 __mutex_lock+0x6bd/0xb10 kernel/locking/mutex.c:730
 rdma_dev_change_netns+0x30/0x320 drivers/infiniband/core/device.c:1650
 rdma_dev_exit_net+0x2de/0x590 drivers/infiniband/core/device.c:1135
 ops_exit_list+0xb0/0x180 net/core/net_namespace.c:172
 cleanup_net+0x5c6/0xbf0 net/core/net_namespace.c:652
 process_one_work+0x9c5/0x1ba0 kernel/workqueue.c:3236
 process_scheduled_works kernel/workqueue.c:3317 [inline]
 worker_thread+0x6c8/0xf00 kernel/workqueue.c:3398
 kthread+0x3af/0x750 kernel/kthread.c:464
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:148
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
INFO: task syz.7.1244:11214 blocked for more than 144 seconds.
      Not tainted 6.13.0-syzkaller-09760-g69e858e0b8b2 #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz.7.1244      state:D stack:23728 pid:11214 tgid:11213 ppid:8992   task_flags:0x400140 flags:0x00004004
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5377 [inline]
 __schedule+0xf43/0x5890 kernel/sched/core.c:6764
 __schedule_loop kernel/sched/core.c:6841 [inline]
 schedule+0xe7/0x350 kernel/sched/core.c:6856
 schedule_timeout+0x244/0x280 kernel/time/sleep_timeout.c:75
 do_wait_for_common kernel/sched/completion.c:95 [inline]
 __wait_for_common+0x3e1/0x600 kernel/sched/completion.c:116
 disable_device+0x170/0x280 drivers/infiniband/core/device.c:1279
 __ib_unregister_device+0x2b4/0x480 drivers/infiniband/core/device.c:1502
 ib_unregister_device_and_put+0x5a/0x80 drivers/infiniband/core/device.c:1567
 nldev_dellink+0x211/0x300 drivers/infiniband/core/nldev.c:1825
 rdma_nl_rcv_msg+0x388/0x6e0 drivers/infiniband/core/netlink.c:195
 rdma_nl_rcv_skb.constprop.0.isra.0+0x2e6/0x450 drivers/infiniband/core/netlink.c:239
 netlink_unicast_kernel net/netlink/af_netlink.c:1322 [inline]
 netlink_unicast+0x53c/0x7f0 net/netlink/af_netlink.c:1348
 netlink_sendmsg+0x8b8/0xd70 net/netlink/af_netlink.c:1892
 sock_sendmsg_nosec net/socket.c:713 [inline]
 __sock_sendmsg net/socket.c:728 [inline]
 ____sys_sendmsg+0xaaf/0xc90 net/socket.c:2568
 ___sys_sendmsg+0x135/0x1e0 net/socket.c:2622
 __sys_sendmsg+0x16e/0x220 net/socket.c:2654
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xcd/0x250 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f4f6958cda9
RSP: 002b:00007f4f6a36d038 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 00007f4f697a5fa0 RCX: 00007f4f6958cda9
RDX: 0000000000000000 RSI: 00000000200002c0 RDI: 000000000000000b
RBP: 00007f4f6960e2a0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000000 R14: 00007f4f697a5fa0 R15: 00007ffcc45a23b8
 </TASK>

Showing all locks held in the system:
1 lock held by khungtaskd/30:
 #0: ffffffff8e1bcbc0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire include/linux/rcupdate.h:337 [inline]
 #0: ffffffff8e1bcbc0 (rcu_read_lock){....}-{1:3}, at: rcu_read_lock include/linux/rcupdate.h:849 [inline]
 #0: ffffffff8e1bcbc0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x7f/0x390 kernel/locking/lockdep.c:6746
2 locks held by getty/5583:
 #0: ffff8880321610a0 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x24/0x80 drivers/tty/tty_ldisc.c:243
 #1: ffffc90002fde2f0 (&ldata->atomic_read_lock){+.+.}-{4:4}, at: n_tty_read+0xfba/0x1480 drivers/tty/n_tty.c:2211
4 locks held by kworker/u8:28/11047:
 #0: ffff88801beeb148 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0x1293/0x1ba0 kernel/workqueue.c:3211
 #1: ffffc9000d4f7d18 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0x921/0x1ba0 kernel/workqueue.c:3212
 #2: ffffffff8feddb10 (pernet_ops_rwsem){++++}-{4:4}, at: cleanup_net+0xca/0xbf0 net/core/net_namespace.c:606
 #3: ffff88807d2286c0 (&device->unregistration_lock){+.+.}-{4:4}, at: rdma_dev_change_netns+0x30/0x320 drivers/infiniband/core/device.c:1650
2 locks held by syz.7.1244/11214:
 #0: ffffffff9ac03178 (&rdma_nl_types[idx].sem){.+.+}-{4:4}, at: rdma_nl_rcv_msg+0x16a/0x6e0 drivers/infiniband/core/netlink.c:164
 #1: ffff88807d2286c0 (&device->unregistration_lock){+.+.}-{4:4}, at: __ib_unregister_device+0x23b/0x480 drivers/infiniband/core/device.c:1498
2 locks held by kworker/u8:42/11490:
 #0: ffff88801cfbd148 ((wq_completion)iou_exit){+.+.}-{0:0}, at: process_one_work+0x1293/0x1ba0 kernel/workqueue.c:3211
 #1: ffffc9000be87d18 ((work_completion)(&ctx->exit_work)){+.+.}-{0:0}, at: process_one_work+0x921/0x1ba0 kernel/workqueue.c:3212
2 locks held by syz.5.1365/11674:
 #0: ffffffff8feddb10 (pernet_ops_rwsem){++++}-{4:4}, at: copy_net_ns+0x292/0x6c0 net/core/net_namespace.c:512
 #1: ffff88807d2286c0 (&device->unregistration_lock){+.+.}-{4:4}, at: rdma_dev_change_netns+0x30/0x320 drivers/infiniband/core/device.c:1650
2 locks held by syz.9.1510/12227:
 #0: ffffffff8feddb10 (pernet_ops_rwsem){++++}-{4:4}, at: copy_net_ns+0x292/0x6c0 net/core/net_namespace.c:512
 #1: ffff88807d2286c0 (&device->unregistration_lock){+.+.}-{4:4}, at: rdma_dev_change_netns+0x30/0x320 drivers/infiniband/core/device.c:1650
2 locks held by syz.8.1577/12488:
 #0: ffffffff8feddb10 (pernet_ops_rwsem){++++}-{4:4}, at: copy_net_ns+0x292/0x6c0 net/core/net_namespace.c:512
 #1: ffff88807d2286c0 (&device->unregistration_lock){+.+.}-{4:4}, at: rdma_dev_change_netns+0x30/0x320 drivers/infiniband/core/device.c:1650

=============================================

NMI backtrace for cpu 0
CPU: 0 UID: 0 PID: 30 Comm: khungtaskd Not tainted 6.13.0-syzkaller-09760-g69e858e0b8b2 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 12/27/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:94 [inline]
 dump_stack_lvl+0x116/0x1f0 lib/dump_stack.c:120
 nmi_cpu_backtrace+0x27b/0x390 lib/nmi_backtrace.c:113
 nmi_trigger_cpumask_backtrace+0x29c/0x300 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:162 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:236 [inline]
 watchdog+0xf62/0x12b0 kernel/hung_task.c:399
 kthread+0x3af/0x750 kernel/kthread.c:464
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:148
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1 skipped: idling at native_safe_halt arch/x86/include/asm/irqflags.h:48 [inline]
NMI backtrace for cpu 1 skipped: idling at arch_safe_halt arch/x86/include/asm/irqflags.h:106 [inline]
NMI backtrace for cpu 1 skipped: idling at acpi_safe_halt+0x1a/0x20 drivers/acpi/processor_idle.c:111

Crashes (276):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2025/02/02 07:30 upstream 69e858e0b8b2 0dff8567 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: task hung in rdma_dev_change_netns
2025/02/02 04:22 upstream 69e858e0b8b2 0dff8567 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: task hung in rdma_dev_change_netns
2025/02/01 22:51 upstream 69e858e0b8b2 0dff8567 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: task hung in rdma_dev_change_netns
2025/01/31 17:40 upstream 69e858e0b8b2 aa47157c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: task hung in rdma_dev_change_netns
2025/01/19 15:38 upstream fda5e3f28400 f2cb035c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: task hung in rdma_dev_change_netns
2025/01/03 19:00 upstream 0bc21e701a6f f3558dbf .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: task hung in rdma_dev_change_netns
2024/12/31 14:59 upstream ccb98ccef0e5 d3ccff63 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: task hung in rdma_dev_change_netns
2024/12/23 12:57 upstream 4bbf9020becb 444551c4 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: task hung in rdma_dev_change_netns
2024/12/23 11:03 upstream 4bbf9020becb 444551c4 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: task hung in rdma_dev_change_netns
2024/12/08 00:03 upstream 7503345ac5f5 9ac0fdc6 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: task hung in rdma_dev_change_netns
2024/12/03 00:32 upstream cdd30ebb1b9f bb326ffb .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: task hung in rdma_dev_change_netns
2024/11/27 07:17 upstream 445d9f05fa14 52b38cc1 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: task hung in rdma_dev_change_netns
2025/02/05 15:12 net a1300691aed9 5896748e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in rdma_dev_change_netns
2025/01/31 02:50 net dfffaccffc53 4c6ac32f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in rdma_dev_change_netns
2025/01/12 09:34 net 47e55e4b410f 6dbc6a9b .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in rdma_dev_change_netns
2025/01/05 09:21 net 8ce4f287524c f3558dbf .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/12/13 03:02 net 150b567e0d57 3547e30f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/09/03 15:42 net 5517ae241919 512328ba .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/08/30 23:28 net 98d4435efcbf f885a8ff .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/08/22 15:14 net af688a99eb1f 295a4b50 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in rdma_dev_change_netns
2025/02/04 15:54 net-next c2933b2befe2 8f267cef .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2025/01/28 19:28 net-next 0ad9617c78ac f5427d7c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2025/01/28 07:22 net-next 0ad9617c78ac 18070896 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2025/01/01 17:18 net-next 9268abe611b0 d3ccff63 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/12/31 01:13 net-next 9268abe611b0 d3ccff63 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/12/30 16:22 net-next 9268abe611b0 d3ccff63 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/12/29 10:57 net-next 9268abe611b0 d3ccff63 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/12/29 08:58 net-next 9268abe611b0 d3ccff63 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/12/28 15:55 net-next 9268abe611b0 d3ccff63 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/12/27 03:52 net-next 9268abe611b0 d3ccff63 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/12/26 23:19 net-next 9268abe611b0 d3ccff63 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/12/25 15:49 net-next 9268abe611b0 444551c4 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/12/24 10:28 net-next 9268abe611b0 444551c4 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/12/22 15:56 net-next ae418e95dd93 d7f584ee .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/12/14 19:39 net-next 9bc5c9515b48 7cbfbb3a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/12/13 17:21 net-next 2c27c7663390 3547e30f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/12/12 07:23 net-next c0b8980e6041 ff949d25 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/12/11 23:55 net-next 65fb414c93f4 ff949d25 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/12/11 21:37 net-next 65fb414c93f4 ff949d25 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/12/11 11:34 net-next 65fb414c93f4 ff949d25 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/12/11 04:43 net-next a0e1fc921cb0 cfc402b4 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/12/09 00:07 net-next 7ea2745766d7 9ac0fdc6 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/12/08 14:49 net-next 3ca459eaba1b 9ac0fdc6 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/12/06 20:54 net-next 51db5c894300 9ac0fdc6 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/11/28 13:36 net-next fcc79e1714e8 5df23865 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/11/22 08:58 net-next fcc79e1714e8 4b25d554 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/11/21 17:59 net-next dd7207838d38 4b25d554 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/11/21 13:10 net-next dd7207838d38 4b25d554 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/11/21 09:08 net-next dd7207838d38 4b25d554 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/11/19 18:16 net-next dd7207838d38 571351cb .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/11/19 16:31 net-next e867ed3ac8aa 571351cb .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
2024/11/19 15:00 net-next e867ed3ac8aa 571351cb .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in rdma_dev_change_netns
* Struck through repros no longer work on HEAD.