syzbot


possible deadlock in static_key_slow_inc (2)

Status: upstream: reported C repro on 2022/11/02 10:36
Reported-by: syzbot+c39682e86c9d84152f93@syzkaller.appspotmail.com
First crash: 28d, last: 40m

Cause bisection: failed (bisect log)
similar bugs (2):
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream possible deadlock in static_key_slow_inc 5 1916d 1852d 0/24 closed as invalid on 2017/10/31 11:27
linux-4.14 possible deadlock in static_key_slow_inc 1 299d 299d 0/1 auto-closed as invalid on 2022/05/31 03:42
Patch testing requests:
Created Duration User Patch Repo Result
2022/11/03 12:08 20m hdanton@sina.com patch https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git a2c65a9d0568 OK log
2022/11/03 00:15 6m hdanton@sina.com patch https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git a2c65a9d0568 error
2022/11/02 13:53 6m hdanton@sina.com patch https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git a2c65a9d0568 error

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
6.1.0-rc6-syzkaller-00015-gc3eb11fbb826 #0 Not tainted
------------------------------------------------------
syz-executor392/3641 is trying to acquire lock:
ffffffff8cfc4470 (cpu_hotplug_lock){++++}-{0:0}, at: static_key_slow_inc+0xe/0x20 kernel/jump_label.c:158

but task is already holding lock:
ffffffff8d15a7e8 (freezer_mutex){+.+.}-{3:3}, at: freezer_change_state kernel/cgroup/legacy_freezer.c:387 [inline]
ffffffff8d15a7e8 (freezer_mutex){+.+.}-{3:3}, at: freezer_write+0x9f/0x3b0 kernel/cgroup/legacy_freezer.c:426

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #2 (freezer_mutex){+.+.}-{3:3}:
       lock_acquire+0x182/0x3c0 kernel/locking/lockdep.c:5668
       __mutex_lock_common+0x1bd/0x26e0 kernel/locking/mutex.c:603
       __mutex_lock kernel/locking/mutex.c:747 [inline]
       mutex_lock_nested+0x17/0x20 kernel/locking/mutex.c:799
       freezer_attach+0x82/0x290 kernel/cgroup/legacy_freezer.c:163
       cgroup_migrate_execute+0x7e9/0x1020 kernel/cgroup/cgroup.c:2615
       cgroup_attach_task+0x54d/0x6a0 kernel/cgroup/cgroup.c:2906
       __cgroup1_procs_write+0x2a9/0x400 kernel/cgroup/cgroup-v1.c:523
       cgroup_file_write+0x2ab/0x730 kernel/cgroup/cgroup.c:4057
       kernfs_fop_write_iter+0x3ac/0x500 fs/kernfs/file.c:330
       call_write_iter include/linux/fs.h:2191 [inline]
       new_sync_write fs/read_write.c:491 [inline]
       vfs_write+0x7dc/0xc50 fs/read_write.c:584
       ksys_write+0x177/0x2a0 fs/read_write.c:637
       do_syscall_x64 arch/x86/entry/common.c:50 [inline]
       do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80
       entry_SYSCALL_64_after_hwframe+0x63/0xcd

-> #1 (cgroup_threadgroup_rwsem){++++}-{0:0}:
       lock_acquire+0x182/0x3c0 kernel/locking/lockdep.c:5668
       percpu_down_write+0x50/0x300 kernel/locking/percpu-rwsem.c:227
       cgroup_attach_lock kernel/cgroup/cgroup.c:2431 [inline]
       cgroup_procs_write_start+0x1c8/0x640 kernel/cgroup/cgroup.c:2935
       __cgroup_procs_write+0xe3/0x490 kernel/cgroup/cgroup.c:5135
       cgroup_procs_write+0x23/0x40 kernel/cgroup/cgroup.c:5171
       cgroup_file_write+0x2ab/0x730 kernel/cgroup/cgroup.c:4057
       kernfs_fop_write_iter+0x3ac/0x500 fs/kernfs/file.c:330
       call_write_iter include/linux/fs.h:2191 [inline]
       new_sync_write fs/read_write.c:491 [inline]
       vfs_write+0x7dc/0xc50 fs/read_write.c:584
       ksys_write+0x177/0x2a0 fs/read_write.c:637
       do_syscall_x64 arch/x86/entry/common.c:50 [inline]
       do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80
       entry_SYSCALL_64_after_hwframe+0x63/0xcd

-> #0 (cpu_hotplug_lock){++++}-{0:0}:
       check_prev_add kernel/locking/lockdep.c:3097 [inline]
       check_prevs_add kernel/locking/lockdep.c:3216 [inline]
       validate_chain+0x1898/0x6ae0 kernel/locking/lockdep.c:3831
       __lock_acquire+0x1292/0x1f60 kernel/locking/lockdep.c:5055
       lock_acquire+0x182/0x3c0 kernel/locking/lockdep.c:5668
       percpu_down_read include/linux/percpu-rwsem.h:51 [inline]
       cpus_read_lock+0x3e/0x130 kernel/cpu.c:310
       static_key_slow_inc+0xe/0x20 kernel/jump_label.c:158
       freezer_apply_state+0x1bb/0x400 kernel/cgroup/legacy_freezer.c:353
       freezer_write+0x24a/0x3b0 kernel/cgroup/legacy_freezer.c:426
       cgroup_file_write+0x2ab/0x730 kernel/cgroup/cgroup.c:4057
       kernfs_fop_write_iter+0x3ac/0x500 fs/kernfs/file.c:330
       call_write_iter include/linux/fs.h:2191 [inline]
       new_sync_write fs/read_write.c:491 [inline]
       vfs_write+0x7dc/0xc50 fs/read_write.c:584
       ksys_write+0x177/0x2a0 fs/read_write.c:637
       do_syscall_x64 arch/x86/entry/common.c:50 [inline]
       do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80
       entry_SYSCALL_64_after_hwframe+0x63/0xcd

other info that might help us debug this:

Chain exists of:
  cpu_hotplug_lock --> cgroup_threadgroup_rwsem --> freezer_mutex

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(freezer_mutex);
                               lock(cgroup_threadgroup_rwsem);
                               lock(freezer_mutex);
  lock(cpu_hotplug_lock);

 *** DEADLOCK ***

4 locks held by syz-executor392/3641:
 #0: ffff8880278ce460 (sb_writers#10){.+.+}-{0:0}, at: vfs_write+0x287/0xc50 fs/read_write.c:580
 #1: ffff88801ebc9488 (&of->mutex){+.+.}-{3:3}, at: kernfs_fop_write_iter+0x1f3/0x500 fs/kernfs/file.c:321
 #2: ffff88814567f2c0 (kn->active#56){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x217/0x500 fs/kernfs/file.c:322
 #3: ffffffff8d15a7e8 (freezer_mutex){+.+.}-{3:3}, at: freezer_change_state kernel/cgroup/legacy_freezer.c:387 [inline]
 #3: ffffffff8d15a7e8 (freezer_mutex){+.+.}-{3:3}, at: freezer_write+0x9f/0x3b0 kernel/cgroup/legacy_freezer.c:426

stack backtrace:
CPU: 1 PID: 3641 Comm: syz-executor392 Not tainted 6.1.0-rc6-syzkaller-00015-gc3eb11fbb826 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/26/2022
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1b1/0x28e lib/dump_stack.c:106
 check_noncircular+0x2cc/0x390 kernel/locking/lockdep.c:2177
 check_prev_add kernel/locking/lockdep.c:3097 [inline]
 check_prevs_add kernel/locking/lockdep.c:3216 [inline]
 validate_chain+0x1898/0x6ae0 kernel/locking/lockdep.c:3831
 __lock_acquire+0x1292/0x1f60 kernel/locking/lockdep.c:5055
 lock_acquire+0x182/0x3c0 kernel/locking/lockdep.c:5668
 percpu_down_read include/linux/percpu-rwsem.h:51 [inline]
 cpus_read_lock+0x3e/0x130 kernel/cpu.c:310
 static_key_slow_inc+0xe/0x20 kernel/jump_label.c:158
 freezer_apply_state+0x1bb/0x400 kernel/cgroup/legacy_freezer.c:353
 freezer_write+0x24a/0x3b0 kernel/cgroup/legacy_freezer.c:426
 cgroup_file_write+0x2ab/0x730 kernel/cgroup/cgroup.c:4057
 kernfs_fop_write_iter+0x3ac/0x500 fs/kernfs/file.c:330
 call_write_iter include/linux/fs.h:2191 [inline]
 new_sync_write fs/read_write.c:491 [inline]
 vfs_write+0x7dc/0xc50 fs/read_write.c:584
 ksys_write+0x177/0x2a0 fs/read_write.c:637
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7f468b95add9
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 51 15 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 c0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fff9d0a5938 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007f468b95add9
RDX: 0000000000000007 RSI: 0000000020000040 RDI: 0000000000000004
RBP: 0000000000000000 R08: 00007fff9d0a5960 R09: 00007fff9d0a5960
R10: 00007fff9d0a5960 R11: 0000000000000246 R12: 00007fff9d0a595c
R13: 00007fff9d0a5970 R14: 00007fff9d0a59b0 R15: 0000000000000000
 </TASK>

Crashes (4954):
Manager Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Title
ci2-upstream-fs 2022/11/25 12:49 upstream c3eb11fbb826 74a66371 .config log report syz C possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce 2022/11/25 11:40 upstream 08ad43d554ba 74a66371 .config log report syz C possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce-root 2022/11/07 22:13 upstream f0c4d9fc9cc9 a779b11a .config log report syz C possible deadlock in static_key_slow_inc
ci-upstream-bpf-kasan-gce 2022/11/25 13:53 bpf 8be602dadb2f 74a66371 .config log report syz C possible deadlock in static_key_slow_inc
ci-upstream-net-this-kasan-gce 2022/11/25 10:29 net 661e5ebbafd2 74a66371 .config log report syz C possible deadlock in static_key_slow_inc
ci-upstream-net-this-kasan-gce 2022/10/29 15:38 net a2c65a9d0568 899d812a .config log report syz C possible deadlock in static_key_slow_inc
ci-upstream-bpf-next-kasan-gce 2022/11/25 11:59 bpf-next 2b3e8f6f5b93 74a66371 .config log report syz C possible deadlock in static_key_slow_inc
ci-upstream-net-kasan-gce 2022/11/25 10:28 net-next 14e5f71e31ff 74a66371 .config log report syz C possible deadlock in static_key_slow_inc
ci-upstream-gce-arm64 2022/11/25 09:59 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 65762d97e6fa 74a66371 .config log report syz C possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce 2022/11/27 01:32 upstream 644e9524388a 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce-root 2022/11/26 21:58 upstream 644e9524388a 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce 2022/11/26 20:49 upstream 644e9524388a 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci2-upstream-fs 2022/11/26 15:30 upstream 644e9524388a f4470a7b .config log report info possible deadlock in static_key_slow_inc
ci2-upstream-fs 2022/11/26 14:28 upstream 08ad43d554ba f4470a7b .config log report info possible deadlock in static_key_slow_inc
ci2-upstream-fs 2022/11/26 10:17 upstream 08ad43d554ba f4470a7b .config log report info possible deadlock in static_key_slow_inc
ci2-upstream-fs 2022/11/26 06:22 upstream 08ad43d554ba f4470a7b .config log report info possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce-smack-root 2022/11/26 05:49 upstream 0b1dcc2cf55a 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-qemu-upstream 2022/11/26 04:23 upstream 0b1dcc2cf55a f4470a7b .config log report info possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce-selinux-root 2022/11/26 02:56 upstream 0b1dcc2cf55a 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce-root 2022/11/26 00:29 upstream 0b1dcc2cf55a 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce-smack-root 2022/11/25 23:54 upstream 0b1dcc2cf55a 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce 2022/11/25 22:50 upstream 0b1dcc2cf55a 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce 2022/11/25 21:50 upstream 0b1dcc2cf55a 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce 2022/11/25 21:37 upstream 0b1dcc2cf55a 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce-smack-root 2022/11/25 20:23 upstream 0b1dcc2cf55a 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce 2022/11/25 19:02 upstream 08ad43d554ba 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-qemu-upstream 2022/11/25 18:02 upstream 08ad43d554ba 0d68fcb4 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce 2022/11/25 16:14 upstream 08ad43d554ba 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce-selinux-root 2022/11/25 07:40 upstream 08ad43d554ba 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-qemu-upstream 2022/11/25 00:56 upstream 08ad43d554ba 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce-386 2022/11/26 23:08 upstream 644e9524388a 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce-386 2022/11/26 22:58 upstream 644e9524388a 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce-386 2022/11/26 02:43 upstream 0b1dcc2cf55a 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce-386 2022/11/25 14:58 upstream 08ad43d554ba 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-qemu-upstream-386 2022/11/25 07:13 upstream 08ad43d554ba 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce-386 2022/11/24 23:28 upstream c3eb11fbb826 62e26685 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-kasan-gce-386 2022/11/24 22:03 upstream c3eb11fbb826 62e26685 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-bpf-kasan-gce 2022/11/27 00:11 bpf 8be602dadb2f 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-bpf-kasan-gce 2022/11/26 19:35 bpf 8be602dadb2f 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-bpf-kasan-gce 2022/11/26 18:10 bpf 8be602dadb2f 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-bpf-kasan-gce 2022/11/26 07:22 bpf 8be602dadb2f 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-net-this-kasan-gce 2022/11/25 13:11 net 661e5ebbafd2 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-net-this-kasan-gce 2022/11/25 08:53 net 661e5ebbafd2 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-bpf-kasan-gce 2022/11/25 00:34 bpf 8be602dadb2f 62e26685 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-net-this-kasan-gce 2022/10/29 10:29 net a2c65a9d0568 899d812a .config log report info possible deadlock in static_key_slow_inc
ci-upstream-bpf-next-kasan-gce 2022/11/26 19:12 bpf-next 2b3e8f6f5b93 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-net-kasan-gce 2022/11/26 17:45 net-next a6e3d86ece0b 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-net-kasan-gce 2022/11/26 16:42 net-next a6e3d86ece0b 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-net-kasan-gce 2022/11/26 14:07 net-next a6e3d86ece0b 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-net-kasan-gce 2022/11/26 13:06 net-next a6e3d86ece0b 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-net-kasan-gce 2022/11/26 11:26 net-next a6e3d86ece0b 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-net-kasan-gce 2022/11/26 10:01 net-next a6e3d86ece0b 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-net-kasan-gce 2022/11/26 01:41 net-next a6e3d86ece0b 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-net-kasan-gce 2022/11/25 20:32 net-next a6e3d86ece0b 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-net-kasan-gce 2022/11/25 19:22 net-next a6e3d86ece0b 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-bpf-next-kasan-gce 2022/11/25 11:32 bpf-next 2b3e8f6f5b93 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-bpf-next-kasan-gce 2022/11/25 11:29 bpf-next 2b3e8f6f5b93 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-net-kasan-gce 2022/11/25 08:45 net-next 14e5f71e31ff 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-linux-next-kasan-gce-root 2022/11/24 19:29 linux-next c35bd4e42885 62e26685 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-gce-arm64 2022/11/25 16:50 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 6d464646530f 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-gce-arm64 2022/11/25 09:33 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 65762d97e6fa 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-gce-arm64 2022/11/25 06:03 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 65762d97e6fa 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-gce-arm64 2022/11/25 04:56 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 65762d97e6fa 74a66371 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-gce-arm64 2022/11/25 03:29 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 65762d97e6fa 62e26685 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-gce-arm64 2022/11/25 02:11 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 65762d97e6fa 62e26685 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-gce-arm64 2022/11/24 21:02 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 65762d97e6fa 62e26685 .config log report info possible deadlock in static_key_slow_inc
ci-upstream-gce-arm64 2022/11/24 20:30 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 65762d97e6fa 62e26685 .config log report info possible deadlock in static_key_slow_inc
* Struck through repros no longer work on HEAD.