syzbot


INFO: task hung in __kernfs_remove

Status: auto-closed as invalid on 2022/10/02 20:33
Subsystems: kernfs
[Documentation on labels]
First crash: 661d, last: 661d
Similar bugs (1)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.19 INFO: task hung in __kernfs_remove 1 1757d 1757d 0/1 auto-closed as invalid on 2019/11/01 17:57

Sample crash report:
INFO: task kworker/u4:9:4110 blocked for more than 143 seconds.
      Not tainted 5.19.0-rc5-syzkaller-00049-gc1084b6c5620 #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/u4:9    state:D stack:26304 pid: 4110 ppid:     2 flags:0x00004000
Workqueue: netns cleanup_net
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5146 [inline]
 __schedule+0xa00/0x4b50 kernel/sched/core.c:6458
 schedule+0xd2/0x1f0 kernel/sched/core.c:6530
 rwsem_down_write_slowpath+0x68a/0x11a0 kernel/locking/rwsem.c:1172
 __down_write_common kernel/locking/rwsem.c:1287 [inline]
 __down_write_common kernel/locking/rwsem.c:1284 [inline]
 __down_write kernel/locking/rwsem.c:1296 [inline]
 down_write+0x135/0x150 kernel/locking/rwsem.c:1543
 kernfs_drain fs/kernfs/dir.c:494 [inline]
 __kernfs_remove+0x5fc/0xb20 fs/kernfs/dir.c:1383
 kernfs_remove_by_name_ns+0xa8/0x110 fs/kernfs/dir.c:1586
 kernfs_remove_by_name include/linux/kernfs.h:558 [inline]
 remove_files+0x96/0x1c0 fs/sysfs/group.c:28
 sysfs_remove_group+0x87/0x170 fs/sysfs/group.c:288
 sysfs_remove_groups fs/sysfs/group.c:312 [inline]
 sysfs_remove_groups+0x5c/0xa0 fs/sysfs/group.c:304
 destroy_port drivers/infiniband/core/sysfs.c:1285 [inline]
 ib_free_port_attrs+0x162/0x460 drivers/infiniband/core/sysfs.c:1410
 remove_one_compat_dev drivers/infiniband/core/device.c:1002 [inline]
 rdma_dev_exit_net+0x2b2/0x550 drivers/infiniband/core/device.c:1140
 ops_exit_list+0xb0/0x170 net/core/net_namespace.c:162
 cleanup_net+0x4ea/0xb00 net/core/net_namespace.c:594
 process_one_work+0x996/0x1610 kernel/workqueue.c:2289
 worker_thread+0x665/0x1080 kernel/workqueue.c:2436
 kthread+0x2e9/0x3a0 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:302
 </TASK>
INFO: task syz-executor.1:23547 blocked for more than 143 seconds.
      Not tainted 5.19.0-rc5-syzkaller-00049-gc1084b6c5620 #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor.1  state:D stack:27048 pid:23547 ppid:  3638 flags:0x00004004
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5146 [inline]
 __schedule+0xa00/0x4b50 kernel/sched/core.c:6458
 schedule+0xd2/0x1f0 kernel/sched/core.c:6530
 schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6589
 __mutex_lock_common kernel/locking/mutex.c:679 [inline]
 __mutex_lock+0xa70/0x1350 kernel/locking/mutex.c:747
 add_one_compat_dev drivers/infiniband/core/device.c:943 [inline]
 add_one_compat_dev+0xea/0x7f0 drivers/infiniband/core/device.c:920
 rdma_dev_init_net+0x28b/0x480 drivers/infiniband/core/device.c:1185
 ops_init+0xaf/0x470 net/core/net_namespace.c:134
 setup_net+0x5d1/0xc50 net/core/net_namespace.c:325
 copy_net_ns+0x318/0x760 net/core/net_namespace.c:471
 create_new_namespaces+0x3f6/0xb20 kernel/nsproxy.c:110
 copy_namespaces+0x391/0x450 kernel/nsproxy.c:178
 copy_process+0x2ffd/0x7020 kernel/fork.c:2243
 kernel_clone+0xe7/0xab0 kernel/fork.c:2655
 __do_sys_clone3+0x1cd/0x2e0 kernel/fork.c:2947
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x46/0xb0
RIP: 0033:0x7f9f33089109
RSP: 002b:00007f9f3424b038 EFLAGS: 00000246 ORIG_RAX: 00000000000001b3
RAX: ffffffffffffffda RBX: 00007f9f3319c1d0 RCX: 00007f9f33089109
RDX: 0000000000000000 RSI: 0000000000000058 RDI: 00007f9f3424b050
RBP: 00007f9f330e305d R08: 0000000000000000 R09: 0000000000000058
R10: 00007f9f3424b050 R11: 0000000000000246 R12: 0000000000000058
R13: 00007ffee918a8ef R14: 00007f9f3424b300 R15: 0000000000022000
 </TASK>

Showing all locks held in the system:
1 lock held by khungtaskd/28:
 #0: ffffffff8bd83b60 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x53/0x260 kernel/locking/lockdep.c:6491
2 locks held by getty/3292:
 #0: ffff888026b57098 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x22/0x80 drivers/tty/tty_ldisc.c:244
 #1: ffffc90001c282e8 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0xe50/0x13c0 drivers/tty/n_tty.c:2124
4 locks held by kworker/u4:9/4110:
 #0: ffff8881400a3138 ((wq_completion)netns){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
 #0: ffff8881400a3138 ((wq_completion)netns){+.+.}-{0:0}, at: arch_atomic_long_set include/linux/atomic/atomic-long.h:41 [inline]
 #0: ffff8881400a3138 ((wq_completion)netns){+.+.}-{0:0}, at: atomic_long_set include/linux/atomic/atomic-instrumented.h:1280 [inline]
 #0: ffff8881400a3138 ((wq_completion)netns){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:636 [inline]
 #0: ffff8881400a3138 ((wq_completion)netns){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:663 [inline]
 #0: ffff8881400a3138 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0x87a/0x1610 kernel/workqueue.c:2260
 #1: ffffc90004397da8 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0x8ae/0x1610 kernel/workqueue.c:2264
 #2: ffffffff8d5730d0 (pernet_ops_rwsem){++++}-{3:3}, at: cleanup_net+0x9b/0xb00 net/core/net_namespace.c:556
 #3: ffff888012153d48 (&root->kernfs_rwsem){++++}-{3:3}, at: kernfs_drain fs/kernfs/dir.c:494 [inline]
 #3: ffff888012153d48 (&root->kernfs_rwsem){++++}-{3:3}, at: __kernfs_remove+0x5fc/0xb20 fs/kernfs/dir.c:1383
2 locks held by kworker/u4:26/13949:
5 locks held by syz-executor.0/23533:
6 locks held by syz-executor.5/23532:
 #0: ffffffff8d5730d0 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x2f5/0x760 net/core/net_namespace.c:467
 #1: ffffffff8d2972f0 (devices_rwsem){++++}-{3:3}, at: rdma_dev_init_net+0x20d/0x480 drivers/infiniband/core/device.c:1179
 #2: ffffffff8d297070 (rdma_nets_rwsem){++++}-{3:3}, at: rdma_dev_init_net+0x280/0x480 drivers/infiniband/core/device.c:1184
 #3: ffff888070eb0f18 (&device->compat_devs_mutex){+.+.}-{3:3}, at: add_one_compat_dev drivers/infiniband/core/device.c:943 [inline]
 #3: ffff888070eb0f18 (&device->compat_devs_mutex){+.+.}-{3:3}, at: add_one_compat_dev+0xea/0x7f0 drivers/infiniband/core/device.c:920
 #4: ffff888012153d48 (&root->kernfs_rwsem){++++}-{3:3}, at: kernfs_add_one+0xa4/0x550 fs/kernfs/dir.c:734
 #5: ffff888012153d08 (&sem->wait_lock){....}-{2:2}, at: rwsem_down_write_slowpath+0x6e1/0x11a0 kernel/locking/rwsem.c:1176
4 locks held by syz-executor.1/23547:
 #0: ffffffff8d5730d0 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x2f5/0x760 net/core/net_namespace.c:467
 #1: ffffffff8d2972f0 (devices_rwsem){++++}-{3:3}, at: rdma_dev_init_net+0x20d/0x480 drivers/infiniband/core/device.c:1179
 #2: ffffffff8d297070 (rdma_nets_rwsem){++++}-{3:3}, at: rdma_dev_init_net+0x280/0x480 drivers/infiniband/core/device.c:1184
 #3: ffff88804d450f18 (&device->compat_devs_mutex){+.+.}-{3:3}, at: add_one_compat_dev drivers/infiniband/core/device.c:943 [inline]
 #3: ffff88804d450f18 (&device->compat_devs_mutex){+.+.}-{3:3}, at: add_one_compat_dev+0xea/0x7f0 drivers/infiniband/core/device.c:920
2 locks held by dhcpcd/23571:
 #0: ffff888074b6e130 (sk_lock-AF_PACKET){+.+.}-{0:0}, at: lock_sock include/net/sock.h:1677 [inline]
 #0: ffff888074b6e130 (sk_lock-AF_PACKET){+.+.}-{0:0}, at: packet_do_bind+0x2f/0xdc0 net/packet/af_packet.c:3194
 #1: ffffffff8bd8de60 (rcu_state.exp_mutex){+.+.}-{3:3}, at: exp_funnel_lock kernel/rcu/tree_exp.h:290 [inline]
 #1: ffffffff8bd8de60 (rcu_state.exp_mutex){+.+.}-{3:3}, at: synchronize_rcu_expedited+0x4f8/0x610 kernel/rcu/tree_exp.h:927

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 28 Comm: khungtaskd Not tainted 5.19.0-rc5-syzkaller-00049-gc1084b6c5620 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/29/2022
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:106
 nmi_cpu_backtrace.cold+0x47/0x144 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x1e6/0x230 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:212 [inline]
 watchdog+0xc1d/0xf50 kernel/hung_task.c:369
 kthread+0x2e9/0x3a0 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:302
 </TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 23532 Comm: syz-executor.5 Not tainted 5.19.0-rc5-syzkaller-00049-gc1084b6c5620 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/29/2022
RIP: 0010:bytes_is_nonzero mm/kasan/generic.c:84 [inline]
RIP: 0010:memory_is_nonzero mm/kasan/generic.c:102 [inline]
RIP: 0010:memory_is_poisoned_n mm/kasan/generic.c:128 [inline]
RIP: 0010:memory_is_poisoned mm/kasan/generic.c:159 [inline]
RIP: 0010:check_region_inline mm/kasan/generic.c:180 [inline]
RIP: 0010:kasan_check_range+0xc8/0x180 mm/kasan/generic.c:189
Code: 38 00 74 ed 48 8d 50 08 eb 09 48 83 c0 01 48 39 d0 74 7a 80 38 00 74 f2 48 89 c2 b8 01 00 00 00 48 85 d2 75 56 5b 5d 41 5c c3 <48> 85 d2 74 5e 48 01 ea eb 09 48 83 c0 01 48 39 d0 74 50 80 38 00
RSP: 0018:ffffc900033f71d8 EFLAGS: 00000083
RAX: ffffed100242a79e RBX: ffffed100242a79f RCX: ffffffff815dfad1
RDX: 0000000000000001 RSI: 0000000000000004 RDI: ffff888012153cf0
RBP: ffffed100242a79e R08: 0000000000000001 R09: ffff888012153cf3
R10: ffffed100242a79e R11: 0000000000000001 R12: ffff888012153cf8
R13: ffff888012153d00 R14: ffff888012153cf0 R15: ffffc900038d7198
FS:  00007fdd7b7bc700(0000) GS:ffff8880b9b00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4e7f56b998 CR3: 000000004791f000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <TASK>
 instrument_atomic_read_write include/linux/instrumented.h:101 [inline]
 atomic_try_cmpxchg_acquire include/linux/atomic/atomic-instrumented.h:541 [inline]
 queued_spin_lock include/asm-generic/qspinlock.h:111 [inline]
 do_raw_spin_lock+0x111/0x2a0 kernel/locking/spinlock_debug.c:115
 rwsem_down_write_slowpath+0x6e1/0x11a0 kernel/locking/rwsem.c:1176
 __down_write_common kernel/locking/rwsem.c:1287 [inline]
 __down_write_common kernel/locking/rwsem.c:1284 [inline]
 __down_write kernel/locking/rwsem.c:1296 [inline]
 down_write+0x135/0x150 kernel/locking/rwsem.c:1543
 kernfs_add_one+0xa4/0x550 fs/kernfs/dir.c:734
 __kernfs_create_file+0x29c/0x350 fs/kernfs/file.c:1016
 sysfs_add_file_mode_ns+0x20f/0x3f0 fs/sysfs/file.c:294
 create_files fs/sysfs/group.c:64 [inline]
 internal_create_group+0x322/0xb10 fs/sysfs/group.c:148
 internal_create_groups.part.0+0x90/0x140 fs/sysfs/group.c:188
 internal_create_groups fs/sysfs/group.c:184 [inline]
 sysfs_create_groups+0x25/0x50 fs/sysfs/group.c:214
 setup_gid_attrs drivers/infiniband/core/sysfs.c:1169 [inline]
 ib_setup_port_attrs+0x3ec/0x5b0 drivers/infiniband/core/sysfs.c:1441
 add_one_compat_dev drivers/infiniband/core/device.c:969 [inline]
 add_one_compat_dev+0x517/0x7f0 drivers/infiniband/core/device.c:920
 rdma_dev_init_net+0x28b/0x480 drivers/infiniband/core/device.c:1185
 ops_init+0xaf/0x470 net/core/net_namespace.c:134
 setup_net+0x5d1/0xc50 net/core/net_namespace.c:325
 copy_net_ns+0x318/0x760 net/core/net_namespace.c:471
 create_new_namespaces+0x3f6/0xb20 kernel/nsproxy.c:110
 copy_namespaces+0x391/0x450 kernel/nsproxy.c:178
 copy_process+0x2ffd/0x7020 kernel/fork.c:2243
 kernel_clone+0xe7/0xab0 kernel/fork.c:2655
 __do_sys_clone3+0x1cd/0x2e0 kernel/fork.c:2947
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x46/0xb0
RIP: 0033:0x7fdd7c889109
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fdd7b7bc038 EFLAGS: 00000246 ORIG_RAX: 00000000000001b3
RAX: ffffffffffffffda RBX: 00007fdd7c99c100 RCX: 00007fdd7c889109
RDX: 0000000000000000 RSI: 0000000000000058 RDI: 00007fdd7b7bc050
RBP: 00007fdd7c8e305d R08: 0000000000000000 R09: 0000000000000058
R10: 00007fdd7b7bc050 R11: 0000000000000246 R12: 0000000000000058
R13: 00007ffc7eb6546f R14: 00007fdd7b7bc300 R15: 0000000000022000
 </TASK>
----------------
Code disassembly (best guess):
   0:	38 00                	cmp    %al,(%rax)
   2:	74 ed                	je     0xfffffff1
   4:	48 8d 50 08          	lea    0x8(%rax),%rdx
   8:	eb 09                	jmp    0x13
   a:	48 83 c0 01          	add    $0x1,%rax
   e:	48 39 d0             	cmp    %rdx,%rax
  11:	74 7a                	je     0x8d
  13:	80 38 00             	cmpb   $0x0,(%rax)
  16:	74 f2                	je     0xa
  18:	48 89 c2             	mov    %rax,%rdx
  1b:	b8 01 00 00 00       	mov    $0x1,%eax
  20:	48 85 d2             	test   %rdx,%rdx
  23:	75 56                	jne    0x7b
  25:	5b                   	pop    %rbx
  26:	5d                   	pop    %rbp
  27:	41 5c                	pop    %r12
  29:	c3                   	retq
* 2a:	48 85 d2             	test   %rdx,%rdx <-- trapping instruction
  2d:	74 5e                	je     0x8d
  2f:	48 01 ea             	add    %rbp,%rdx
  32:	eb 09                	jmp    0x3d
  34:	48 83 c0 01          	add    $0x1,%rax
  38:	48 39 d0             	cmp    %rdx,%rax
  3b:	74 50                	je     0x8d
  3d:	80 38 00             	cmpb   $0x0,(%rax)

Crashes (1):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2022/07/04 20:27 upstream c1084b6c5620 bff65f44 .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in __kernfs_remove
* Struck through repros no longer work on HEAD.