bisecting fixing commit since 13d2ce42de8cb98ff952f8de6307f896203854c2 building syzkaller on 821e0b09046a2f972ace26fbdc02aef1116792d4 testing commit 13d2ce42de8cb98ff952f8de6307f896203854c2 compiler: gcc version 8.4.1 20210217 (GCC) kernel signature: 5eb3b3e6d1da15a93509f617da26ac94641067ca8aa6968a822f4a34ac0a4c9d run #0: crashed: BUG: corrupted list in kobject_add_internal run #1: crashed: BUG: corrupted list in kobject_add_internal run #2: crashed: BUG: corrupted list in kobject_add_internal run #3: crashed: BUG: corrupted list in kobject_add_internal run #4: crashed: BUG: corrupted list in kobject_add_internal run #5: crashed: BUG: corrupted list in kobject_add_internal run #6: crashed: BUG: corrupted list in kobject_add_internal run #7: crashed: BUG: corrupted list in kobject_add_internal run #8: crashed: BUG: corrupted list in kobject_add_internal run #9: crashed: BUG: corrupted list in kobject_add_internal run #10: crashed: BUG: corrupted list in kobject_add_internal run #11: crashed: BUG: corrupted list in kobject_add_internal run #12: crashed: BUG: corrupted list in kobject_add_internal run #13: crashed: BUG: corrupted list in kobject_add_internal run #14: crashed: BUG: corrupted list in kobject_add_internal run #15: crashed: BUG: corrupted list in kobject_add_internal run #16: crashed: BUG: corrupted list in kobject_add_internal run #17: crashed: BUG: corrupted list in kobject_add_internal run #18: crashed: general protection fault in klist_next run #19: crashed: general protection fault in klist_next testing current HEAD c2276d585654e8d573366c29c565043ec36adf63 testing commit c2276d585654e8d573366c29c565043ec36adf63 compiler: gcc version 8.4.1 20210217 (GCC) kernel signature: 9ae1f9c87da97f3795f110408367fa549af8b571bbab21a88edb7742e8e35dad all runs: OK # git bisect start c2276d585654e8d573366c29c565043ec36adf63 13d2ce42de8cb98ff952f8de6307f896203854c2 Bisecting: 1814 revisions left to test after this (roughly 11 steps) [84eac67bcb414603783bbb70cbb3c3f55f094e59] ARM: dts: exynos: correct PMIC interrupt trigger level on Odroid X/U3 family testing commit 84eac67bcb414603783bbb70cbb3c3f55f094e59 compiler: gcc version 8.4.1 20210217 (GCC) kernel signature: 3c74b60a88dfbec0c5b59b14296a8ab95f6d4022acb4a536600175b15cfcc505 run #0: crashed: BUG: corrupted list in kobject_add_internal run #1: crashed: BUG: corrupted list in kobject_add_internal run #2: crashed: BUG: corrupted list in kobject_add_internal run #3: crashed: BUG: corrupted list in kobject_add_internal run #4: crashed: BUG: corrupted list in kobject_add_internal run #5: crashed: BUG: corrupted list in kobject_add_internal run #6: crashed: BUG: corrupted list in kobject_add_internal run #7: crashed: BUG: corrupted list in kobject_add_internal run #8: crashed: general protection fault in klist_next run #9: crashed: general protection fault in klist_next # git bisect good 84eac67bcb414603783bbb70cbb3c3f55f094e59 Bisecting: 907 revisions left to test after this (roughly 10 steps) [06a65c0d8aaf2c010d6dab7f8358e4d6692dcc35] extcon: sm5502: Drop invalid register write in sm5502_reg_data testing commit 06a65c0d8aaf2c010d6dab7f8358e4d6692dcc35 compiler: gcc version 8.4.1 20210217 (GCC) kernel signature: d04c2290871864416aebe6616fcf742c8d89190674515c1c4bae0aaaebfeb966 run #0: crashed: BUG: corrupted list in kobject_add_internal run #1: crashed: BUG: corrupted list in kobject_add_internal run #2: crashed: BUG: corrupted list in kobject_add_internal run #3: crashed: BUG: corrupted list in kobject_add_internal run #4: crashed: BUG: corrupted list in kobject_add_internal run #5: crashed: BUG: corrupted list in kobject_add_internal run #6: crashed: BUG: corrupted list in kobject_add_internal run #7: crashed: BUG: corrupted list in kobject_add_internal run #8: crashed: BUG: corrupted list in kobject_add_internal run #9: basic kernel testing failed: failed to copy test binary to VM: timedout ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "/tmp/syz-executor522961309" "root@10.128.0.44:./syz-executor522961309"] Warning: Permanently added '10.128.0.44' (ECDSA) to the list of known hosts. # git bisect good 06a65c0d8aaf2c010d6dab7f8358e4d6692dcc35 Bisecting: 453 revisions left to test after this (roughly 9 steps) [9df311b2e743642c5427ecf563c5050ceb355d1d] bpf: Fix leakage under speculation on mispredicted branches testing commit 9df311b2e743642c5427ecf563c5050ceb355d1d compiler: gcc version 8.4.1 20210217 (GCC) kernel signature: d1de7a14a0d83e3acca5d6eb88fb6296ef148253805896d4cfc140bb0fdc56c1 run #0: crashed: BUG: corrupted list in kobject_add_internal run #1: crashed: BUG: corrupted list in kobject_add_internal run #2: crashed: BUG: corrupted list in kobject_add_internal run #3: crashed: BUG: corrupted list in kobject_add_internal run #4: crashed: BUG: corrupted list in kobject_add_internal run #5: crashed: BUG: corrupted list in kobject_add_internal run #6: crashed: BUG: corrupted list in kobject_add_internal run #7: crashed: BUG: corrupted list in kobject_add_internal run #8: crashed: BUG: corrupted list in kobject_add_internal run #9: crashed: general protection fault in klist_next # git bisect good 9df311b2e743642c5427ecf563c5050ceb355d1d Bisecting: 226 revisions left to test after this (roughly 8 steps) [5129766a9797ea212087314fafc68268f1bf8515] net: sched: Fix qdisc_rate_table refcount leak when get tcf_block failed testing commit 5129766a9797ea212087314fafc68268f1bf8515 compiler: gcc version 8.4.1 20210217 (GCC) kernel signature: 1dea5f15db02a8c2122f5ee071d001a7039f8520cb7281c42ae1bdb130d34d57 run #0: crashed: BUG: corrupted list in kobject_add_internal run #1: crashed: BUG: corrupted list in kobject_add_internal run #2: crashed: BUG: corrupted list in kobject_add_internal run #3: crashed: BUG: corrupted list in kobject_add_internal run #4: crashed: BUG: corrupted list in kobject_add_internal run #5: crashed: BUG: corrupted list in kobject_add_internal run #6: crashed: BUG: corrupted list in kobject_add_internal run #7: crashed: BUG: corrupted list in kobject_add_internal run #8: crashed: kernel panic: Fatal exception run #9: crashed: general protection fault in klist_next # git bisect good 5129766a9797ea212087314fafc68268f1bf8515 Bisecting: 113 revisions left to test after this (roughly 7 steps) [3f7b869c1b44108a8cbf3e4a763ddac9df548d73] Bluetooth: avoid circular locks in sco_sock_connect testing commit 3f7b869c1b44108a8cbf3e4a763ddac9df548d73 compiler: gcc version 8.4.1 20210217 (GCC) kernel signature: 75fe79d6b936b9461047228ffd591bda4425ef908af504fda8fa32c912b7d902 all runs: OK # git bisect bad 3f7b869c1b44108a8cbf3e4a763ddac9df548d73 Bisecting: 56 revisions left to test after this (roughly 6 steps) [78bbca0bf3f4f4780515872e8286c84296d04cdb] RDMA/iwcm: Release resources if iw_cm module initialization fails testing commit 78bbca0bf3f4f4780515872e8286c84296d04cdb compiler: gcc version 8.4.1 20210217 (GCC) kernel signature: 7e5dfcf65dc03617a1cedaea01d54a817f6e7859428997aa531e1f768fe0dda6 run #0: crashed: BUG: corrupted list in kobject_add_internal run #1: crashed: BUG: corrupted list in kobject_add_internal run #2: crashed: BUG: corrupted list in kobject_add_internal run #3: crashed: BUG: corrupted list in kobject_add_internal run #4: crashed: BUG: corrupted list in kobject_add_internal run #5: crashed: BUG: corrupted list in kobject_add_internal run #6: crashed: BUG: corrupted list in kobject_add_internal run #7: crashed: BUG: corrupted list in kobject_add_internal run #8: crashed: BUG: corrupted list in kobject_add_internal run #9: crashed: kernel panic: Fatal exception # git bisect good 78bbca0bf3f4f4780515872e8286c84296d04cdb Bisecting: 28 revisions left to test after this (roughly 5 steps) [84bf0fb189d512cb890381e6af855bc26b7b2294] tty: serial: jsm: hold port lock when reporting modem line changes testing commit 84bf0fb189d512cb890381e6af855bc26b7b2294 compiler: gcc version 8.4.1 20210217 (GCC) kernel signature: 3922324ec8740f5e7825530d5acea6161e89bb6fe4f5b653e8d494774c7202f1 run #0: crashed: BUG: corrupted list in kobject_add_internal run #1: crashed: BUG: corrupted list in kobject_add_internal run #2: crashed: BUG: corrupted list in kobject_add_internal run #3: crashed: BUG: corrupted list in kobject_add_internal run #4: crashed: BUG: corrupted list in kobject_add_internal run #5: crashed: BUG: corrupted list in kobject_add_internal run #6: crashed: BUG: corrupted list in kobject_add_internal run #7: crashed: BUG: corrupted list in kobject_add_internal run #8: crashed: general protection fault in klist_next run #9: crashed: kernel panic: Fatal exception # git bisect good 84bf0fb189d512cb890381e6af855bc26b7b2294 Bisecting: 14 revisions left to test after this (roughly 4 steps) [962ee5d1ff986bbc0f0370337db32fdfc7f2e7ed] staging: ks7010: Fix the initialization of the 'sleep_status' structure testing commit 962ee5d1ff986bbc0f0370337db32fdfc7f2e7ed compiler: gcc version 8.4.1 20210217 (GCC) kernel signature: bf400673eb5dec84b809177bd96bf65e41d4db825679798c1d5dfccb1e60ddc9 run #0: crashed: BUG: corrupted list in kobject_add_internal run #1: crashed: BUG: corrupted list in kobject_add_internal run #2: crashed: BUG: corrupted list in kobject_add_internal run #3: crashed: BUG: corrupted list in kobject_add_internal run #4: crashed: BUG: corrupted list in kobject_add_internal run #5: crashed: BUG: corrupted list in kobject_add_internal run #6: crashed: BUG: corrupted list in kobject_add_internal run #7: crashed: BUG: corrupted list in kobject_add_internal run #8: crashed: BUG: corrupted list in kobject_add_internal run #9: crashed: general protection fault in klist_next # git bisect good 962ee5d1ff986bbc0f0370337db32fdfc7f2e7ed Bisecting: 7 revisions left to test after this (roughly 3 steps) [4c0307b0607e5af0a5c1525085d95069770fadcd] media: imx258: Limit the max analogue gain to 480 testing commit 4c0307b0607e5af0a5c1525085d95069770fadcd compiler: gcc version 8.4.1 20210217 (GCC) kernel signature: 23fea43cd34c6300040069b17fa49b419bd2ee7b02abe27d40da606aa77ac035 all runs: crashed: inconsistent lock state in sco_sock_timeout # git bisect good 4c0307b0607e5af0a5c1525085d95069770fadcd Bisecting: 3 revisions left to test after this (roughly 2 steps) [0214e26442be85a054a1af8bb66db094ba26bc00] ARM: dts: imx53-ppd: Fix ACHC entry testing commit 0214e26442be85a054a1af8bb66db094ba26bc00 compiler: gcc version 8.4.1 20210217 (GCC) kernel signature: dda5bbbc72040875528f3a712aa01bfa8653e1d187b14eae7e5565f965178cd3 run #0: crashed: BUG: corrupted list in kobject_add_internal run #1: crashed: inconsistent lock state in sco_sock_timeout run #2: crashed: inconsistent lock state in sco_sock_timeout run #3: crashed: inconsistent lock state in sco_sock_timeout run #4: crashed: inconsistent lock state in sco_sock_timeout run #5: crashed: inconsistent lock state in sco_sock_timeout run #6: crashed: inconsistent lock state in sco_sock_timeout run #7: crashed: inconsistent lock state in sco_sock_timeout run #8: crashed: inconsistent lock state in sco_sock_timeout run #9: OK # git bisect good 0214e26442be85a054a1af8bb66db094ba26bc00 Bisecting: 1 revision left to test after this (roughly 1 step) [aca58859ee7254f195745d98f33192a008427835] net: ethernet: stmmac: Do not use unreachable() in ipq806x_gmac_probe() testing commit aca58859ee7254f195745d98f33192a008427835 compiler: gcc version 8.4.1 20210217 (GCC) kernel signature: dda5bbbc72040875528f3a712aa01bfa8653e1d187b14eae7e5565f965178cd3 run #0: crashed: BUG: corrupted list in kobject_add_internal run #1: crashed: inconsistent lock state in sco_sock_timeout run #2: crashed: inconsistent lock state in sco_sock_timeout run #3: crashed: inconsistent lock state in sco_sock_timeout run #4: crashed: inconsistent lock state in sco_sock_timeout run #5: crashed: inconsistent lock state in sco_sock_timeout run #6: crashed: inconsistent lock state in sco_sock_timeout run #7: crashed: inconsistent lock state in sco_sock_timeout run #8: crashed: inconsistent lock state in sco_sock_timeout run #9: crashed: inconsistent lock state in sco_sock_timeout # git bisect good aca58859ee7254f195745d98f33192a008427835 Bisecting: 0 revisions left to test after this (roughly 0 steps) [48669c81a65628ef234cbdd91b9395952c7c27fe] Bluetooth: schedule SCO timeouts with delayed_work testing commit 48669c81a65628ef234cbdd91b9395952c7c27fe compiler: gcc version 8.4.1 20210217 (GCC) kernel signature: 85c3d1f231dd3ce1eba643f4544d8c8ee872d21c537e5daa03d32f4347687040 run #0: crashed: BUG: corrupted list in kobject_add_internal run #1: OK run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK reproducer seems to be flaky # git bisect good 48669c81a65628ef234cbdd91b9395952c7c27fe 3f7b869c1b44108a8cbf3e4a763ddac9df548d73 is the first bad commit commit 3f7b869c1b44108a8cbf3e4a763ddac9df548d73 Author: Desmond Cheong Zhi Xi Date: Tue Aug 10 12:14:06 2021 +0800 Bluetooth: avoid circular locks in sco_sock_connect [ Upstream commit 734bc5ff783115aa3164f4e9dd5967ae78e0a8ab ] In a future patch, calls to bh_lock_sock in sco.c should be replaced by lock_sock now that none of the functions are run in IRQ context. However, doing so results in a circular locking dependency: ====================================================== WARNING: possible circular locking dependency detected 5.14.0-rc4-syzkaller #0 Not tainted ------------------------------------------------------ syz-executor.2/14867 is trying to acquire lock: ffff88803e3c1120 (sk_lock-AF_BLUETOOTH-BTPROTO_SCO){+.+.}-{0:0}, at: lock_sock include/net/sock.h:1613 [inline] ffff88803e3c1120 (sk_lock-AF_BLUETOOTH-BTPROTO_SCO){+.+.}-{0:0}, at: sco_conn_del+0x12a/0x2a0 net/bluetooth/sco.c:191 but task is already holding lock: ffffffff8d2dc7c8 (hci_cb_list_lock){+.+.}-{3:3}, at: hci_disconn_cfm include/net/bluetooth/hci_core.h:1497 [inline] ffffffff8d2dc7c8 (hci_cb_list_lock){+.+.}-{3:3}, at: hci_conn_hash_flush+0xda/0x260 net/bluetooth/hci_conn.c:1608 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #2 (hci_cb_list_lock){+.+.}-{3:3}: __mutex_lock_common kernel/locking/mutex.c:959 [inline] __mutex_lock+0x12a/0x10a0 kernel/locking/mutex.c:1104 hci_connect_cfm include/net/bluetooth/hci_core.h:1482 [inline] hci_remote_features_evt net/bluetooth/hci_event.c:3263 [inline] hci_event_packet+0x2f4d/0x7c50 net/bluetooth/hci_event.c:6240 hci_rx_work+0x4f8/0xd30 net/bluetooth/hci_core.c:5122 process_one_work+0x98d/0x1630 kernel/workqueue.c:2276 worker_thread+0x658/0x11f0 kernel/workqueue.c:2422 kthread+0x3e5/0x4d0 kernel/kthread.c:319 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295 -> #1 (&hdev->lock){+.+.}-{3:3}: __mutex_lock_common kernel/locking/mutex.c:959 [inline] __mutex_lock+0x12a/0x10a0 kernel/locking/mutex.c:1104 sco_connect net/bluetooth/sco.c:245 [inline] sco_sock_connect+0x227/0xa10 net/bluetooth/sco.c:601 __sys_connect_file+0x155/0x1a0 net/socket.c:1879 __sys_connect+0x161/0x190 net/socket.c:1896 __do_sys_connect net/socket.c:1906 [inline] __se_sys_connect net/socket.c:1903 [inline] __x64_sys_connect+0x6f/0xb0 net/socket.c:1903 do_syscall_x64 arch/x86/entry/common.c:50 [inline] do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80 entry_SYSCALL_64_after_hwframe+0x44/0xae -> #0 (sk_lock-AF_BLUETOOTH-BTPROTO_SCO){+.+.}-{0:0}: check_prev_add kernel/locking/lockdep.c:3051 [inline] check_prevs_add kernel/locking/lockdep.c:3174 [inline] validate_chain kernel/locking/lockdep.c:3789 [inline] __lock_acquire+0x2a07/0x54a0 kernel/locking/lockdep.c:5015 lock_acquire kernel/locking/lockdep.c:5625 [inline] lock_acquire+0x1ab/0x510 kernel/locking/lockdep.c:5590 lock_sock_nested+0xca/0x120 net/core/sock.c:3170 lock_sock include/net/sock.h:1613 [inline] sco_conn_del+0x12a/0x2a0 net/bluetooth/sco.c:191 sco_disconn_cfm+0x71/0xb0 net/bluetooth/sco.c:1202 hci_disconn_cfm include/net/bluetooth/hci_core.h:1500 [inline] hci_conn_hash_flush+0x127/0x260 net/bluetooth/hci_conn.c:1608 hci_dev_do_close+0x528/0x1130 net/bluetooth/hci_core.c:1778 hci_unregister_dev+0x1c0/0x5a0 net/bluetooth/hci_core.c:4015 vhci_release+0x70/0xe0 drivers/bluetooth/hci_vhci.c:340 __fput+0x288/0x920 fs/file_table.c:280 task_work_run+0xdd/0x1a0 kernel/task_work.c:164 exit_task_work include/linux/task_work.h:32 [inline] do_exit+0xbd4/0x2a60 kernel/exit.c:825 do_group_exit+0x125/0x310 kernel/exit.c:922 get_signal+0x47f/0x2160 kernel/signal.c:2808 arch_do_signal_or_restart+0x2a9/0x1c40 arch/x86/kernel/signal.c:865 handle_signal_work kernel/entry/common.c:148 [inline] exit_to_user_mode_loop kernel/entry/common.c:172 [inline] exit_to_user_mode_prepare+0x17d/0x290 kernel/entry/common.c:209 __syscall_exit_to_user_mode_work kernel/entry/common.c:291 [inline] syscall_exit_to_user_mode+0x19/0x60 kernel/entry/common.c:302 ret_from_fork+0x15/0x30 arch/x86/entry/entry_64.S:288 other info that might help us debug this: Chain exists of: sk_lock-AF_BLUETOOTH-BTPROTO_SCO --> &hdev->lock --> hci_cb_list_lock Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(hci_cb_list_lock); lock(&hdev->lock); lock(hci_cb_list_lock); lock(sk_lock-AF_BLUETOOTH-BTPROTO_SCO); *** DEADLOCK *** The issue is that the lock hierarchy should go from &hdev->lock --> hci_cb_list_lock --> sk_lock-AF_BLUETOOTH-BTPROTO_SCO. For example, one such call trace is: hci_dev_do_close(): hci_dev_lock(); hci_conn_hash_flush(): hci_disconn_cfm(): mutex_lock(&hci_cb_list_lock); sco_disconn_cfm(): sco_conn_del(): lock_sock(sk); However, in sco_sock_connect, we call lock_sock before calling hci_dev_lock inside sco_connect, thus inverting the lock hierarchy. We fix this by pulling the call to hci_dev_lock out from sco_connect. Signed-off-by: Desmond Cheong Zhi Xi Signed-off-by: Luiz Augusto von Dentz Signed-off-by: Sasha Levin net/bluetooth/sco.c | 39 ++++++++++++++++----------------------- 1 file changed, 16 insertions(+), 23 deletions(-) culprit signature: 75fe79d6b936b9461047228ffd591bda4425ef908af504fda8fa32c912b7d902 parent signature: 85c3d1f231dd3ce1eba643f4544d8c8ee872d21c537e5daa03d32f4347687040 Reproducer flagged being flaky revisions tested: 14, total time: 4h18m5.904675937s (build: 2h13m44.891331053s, test: 2h2m36.58176097s) first good commit: 3f7b869c1b44108a8cbf3e4a763ddac9df548d73 Bluetooth: avoid circular locks in sco_sock_connect recipients (to): ["desmondcheongzx@gmail.com" "luiz.von.dentz@intel.com" "sashal@kernel.org"] recipients (cc): []