ci starts bisection 2023-04-30 18:54:55.267139012 +0000 UTC m=+164363.829213535 bisecting fixing commit since 50fd82b3a9a9335df5d50c7ddcb81c81d358c4fc building syzkaller on eee80d3cd37ea7ebd6aab85c117564a4bae5bc4e ensuring issue is reproducible on original commit 50fd82b3a9a9335df5d50c7ddcb81c81d358c4fc testing commit 50fd82b3a9a9335df5d50c7ddcb81c81d358c4fc gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 15f6c0c310df470c044f0bca0b3160127e9001bc5f36fb0dd4af9f7b624662fa all runs: crashed: possible deadlock in j1939_sk_queue_drop_all testing current HEAD 825a0714d2b3883d4f8ff64f6933fb73ee3f1834 testing commit 825a0714d2b3883d4f8ff64f6933fb73ee3f1834 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: d7c3691cb32bb80e39736b887442d080ec314f820efef730872999872ae914a1 run #0: crashed: possible deadlock in j1939_sk_queue_drop_all run #1: crashed: possible deadlock in j1939_sk_queue_drop_all run #2: crashed: possible deadlock in j1939_sk_queue_drop_all run #3: crashed: possible deadlock in j1939_sk_queue_drop_all run #4: crashed: possible deadlock in j1939_sk_queue_drop_all run #5: crashed: possible deadlock in j1939_sk_queue_drop_all run #6: crashed: possible deadlock in j1939_sk_queue_drop_all run #7: crashed: possible deadlock in j1939_sk_queue_drop_all run #8: crashed: possible deadlock in j1939_session_activate run #9: crashed: possible deadlock in j1939_session_activate revisions tested: 2, total time: 26m57.510590875s (build: 13m25.890883023s, test: 12m24.882637079s) the crash still happens on HEAD commit msg: Merge tag 'efi-next-for-v6.4' of git://git.kernel.org/pub/scm/linux/kernel/git/efi/efi crash: possible deadlock in j1939_session_activate ====================================================== WARNING: possible circular locking dependency detected 6.3.0-syzkaller #0 Not tainted ------------------------------------------------------ ksoftirqd/1/21 is trying to acquire lock: ffff88802894d088 (&priv->active_session_list_lock){+.-.}-{2:2}, at: spin_lock_bh include/linux/spinlock.h:355 [inline] ffff88802894d088 (&priv->active_session_list_lock){+.-.}-{2:2}, at: j1939_session_list_lock net/can/j1939/transport.c:238 [inline] ffff88802894d088 (&priv->active_session_list_lock){+.-.}-{2:2}, at: j1939_session_activate+0x3e/0x3d0 net/can/j1939/transport.c:1564 but task is already holding lock: ffff8880783125c8 (&jsk->sk_session_queue_lock){+.-.}-{2:2}, at: spin_lock_bh include/linux/spinlock.h:355 [inline] ffff8880783125c8 (&jsk->sk_session_queue_lock){+.-.}-{2:2}, at: j1939_sk_queue_activate_next+0x4c/0x440 net/can/j1939/socket.c:207 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #2 (&jsk->sk_session_queue_lock){+.-.}-{2:2}: __raw_spin_lock_bh include/linux/spinlock_api_smp.h:126 [inline] _raw_spin_lock_bh+0x2f/0x40 kernel/locking/spinlock.c:178 spin_lock_bh include/linux/spinlock.h:355 [inline] j1939_sk_queue_drop_all+0x33/0x2f0 net/can/j1939/socket.c:139 j1939_sk_netdev_event_netdown+0x71/0x140 net/can/j1939/socket.c:1275 j1939_netdev_notify+0x121/0x160 net/can/j1939/main.c:379 notifier_call_chain+0x94/0x2a0 kernel/notifier.c:93 call_netdevice_notifiers_extack net/core/dev.c:1973 [inline] call_netdevice_notifiers net/core/dev.c:1987 [inline] __dev_notify_flags+0x156/0x240 net/core/dev.c:8607 dev_change_flags+0xf5/0x160 net/core/dev.c:8643 do_setlink+0x873/0x3050 net/core/rtnetlink.c:2836 __rtnl_newlink+0x94d/0x1520 net/core/rtnetlink.c:3626 rtnl_newlink+0x5a/0x90 net/core/rtnetlink.c:3673 rtnetlink_rcv_msg+0x32d/0x9f0 net/core/rtnetlink.c:6395 netlink_rcv_skb+0x126/0x380 net/netlink/af_netlink.c:2546 netlink_unicast_kernel net/netlink/af_netlink.c:1339 [inline] netlink_unicast+0x433/0x710 net/netlink/af_netlink.c:1365 netlink_sendmsg+0x78c/0xc40 net/netlink/af_netlink.c:1913 sock_sendmsg_nosec net/socket.c:724 [inline] sock_sendmsg+0xbc/0x150 net/socket.c:747 ____sys_sendmsg+0x5ff/0x8f0 net/socket.c:2503 ___sys_sendmsg+0xdb/0x160 net/socket.c:2557 __sys_sendmsg+0xc3/0x160 net/socket.c:2586 do_syscall_x64 arch/x86/entry/common.c:50 [inline] do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80 entry_SYSCALL_64_after_hwframe+0x63/0xcd -> #1 (&priv->j1939_socks_lock){+.-.}-{2:2}: __raw_spin_lock_bh include/linux/spinlock_api_smp.h:126 [inline] _raw_spin_lock_bh+0x2f/0x40 kernel/locking/spinlock.c:178 spin_lock_bh include/linux/spinlock.h:355 [inline] j1939_sk_errqueue+0x8e/0x160 net/can/j1939/socket.c:1081 j1939_session_destroy+0x1f4/0x3e0 net/can/j1939/transport.c:271 __j1939_session_release net/can/j1939/transport.c:294 [inline] kref_put include/linux/kref.h:65 [inline] j1939_session_put net/can/j1939/transport.c:299 [inline] j1939_session_deactivate_locked net/can/j1939/transport.c:1086 [inline] j1939_session_deactivate_locked+0x212/0x2a0 net/can/j1939/transport.c:1074 j1939_cancel_active_session+0x133/0x280 net/can/j1939/transport.c:2194 j1939_netdev_notify+0x119/0x160 net/can/j1939/main.c:378 notifier_call_chain+0x94/0x2a0 kernel/notifier.c:93 call_netdevice_notifiers_extack net/core/dev.c:1973 [inline] call_netdevice_notifiers net/core/dev.c:1987 [inline] __dev_notify_flags+0x156/0x240 net/core/dev.c:8607 dev_change_flags+0xf5/0x160 net/core/dev.c:8643 do_setlink+0x873/0x3050 net/core/rtnetlink.c:2836 __rtnl_newlink+0x94d/0x1520 net/core/rtnetlink.c:3626 rtnl_newlink+0x5a/0x90 net/core/rtnetlink.c:3673 rtnetlink_rcv_msg+0x32d/0x9f0 net/core/rtnetlink.c:6395 netlink_rcv_skb+0x126/0x380 net/netlink/af_netlink.c:2546 netlink_unicast_kernel net/netlink/af_netlink.c:1339 [inline] netlink_unicast+0x433/0x710 net/netlink/af_netlink.c:1365 netlink_sendmsg+0x78c/0xc40 net/netlink/af_netlink.c:1913 sock_sendmsg_nosec net/socket.c:724 [inline] sock_sendmsg+0xbc/0x150 net/socket.c:747 ____sys_sendmsg+0x5ff/0x8f0 net/socket.c:2503 ___sys_sendmsg+0xdb/0x160 net/socket.c:2557 __sys_sendmsg+0xc3/0x160 net/socket.c:2586 do_syscall_x64 arch/x86/entry/common.c:50 [inline] do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80 entry_SYSCALL_64_after_hwframe+0x63/0xcd -> #0 (&priv->active_session_list_lock){+.-.}-{2:2}: check_prev_add kernel/locking/lockdep.c:3108 [inline] check_prevs_add kernel/locking/lockdep.c:3227 [inline] validate_chain kernel/locking/lockdep.c:3842 [inline] __lock_acquire+0x2f21/0x5df0 kernel/locking/lockdep.c:5074 lock_acquire kernel/locking/lockdep.c:5691 [inline] lock_acquire+0x1ad/0x520 kernel/locking/lockdep.c:5656 __raw_spin_lock_bh include/linux/spinlock_api_smp.h:126 [inline] _raw_spin_lock_bh+0x2f/0x40 kernel/locking/spinlock.c:178 spin_lock_bh include/linux/spinlock.h:355 [inline] j1939_session_list_lock net/can/j1939/transport.c:238 [inline] j1939_session_activate+0x3e/0x3d0 net/can/j1939/transport.c:1564 j1939_sk_queue_activate_next_locked net/can/j1939/socket.c:181 [inline] j1939_sk_queue_activate_next+0x24d/0x440 net/can/j1939/socket.c:208 j1939_xtp_rx_eoma_one net/can/j1939/transport.c:1395 [inline] j1939_xtp_rx_eoma+0x21d/0x550 net/can/j1939/transport.c:1410 j1939_tp_cmd_recv net/can/j1939/transport.c:2099 [inline] j1939_tp_recv+0x699/0xc10 net/can/j1939/transport.c:2144 j1939_can_recv net/can/j1939/main.c:112 [inline] j1939_can_recv+0x60b/0x8a0 net/can/j1939/main.c:38 deliver net/can/af_can.c:572 [inline] can_rcv_filter+0x4ce/0x7b0 net/can/af_can.c:606 can_receive+0x2ae/0x4b0 net/can/af_can.c:663 can_rcv+0x15a/0x220 net/can/af_can.c:687 __netif_receive_skb_one_core+0x104/0x180 net/core/dev.c:5491 process_backlog+0xca/0x5c0 net/core/dev.c:5933 __napi_poll+0x9a/0x550 net/core/dev.c:6496 napi_poll net/core/dev.c:6563 [inline] net_rx_action+0x792/0xbb0 net/core/dev.c:6696 __do_softirq+0x1d0/0x901 kernel/softirq.c:571 run_ksoftirqd kernel/softirq.c:939 [inline] run_ksoftirqd+0x2d/0x60 kernel/softirq.c:931 smpboot_thread_fn+0x548/0x8c0 kernel/smpboot.c:164 kthread+0x2e6/0x3c0 kernel/kthread.c:379 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308 other info that might help us debug this: Chain exists of: &priv->active_session_list_lock --> &priv->j1939_socks_lock --> &jsk->sk_session_queue_lock Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(&jsk->sk_session_queue_lock); lock(&priv->j1939_socks_lock); lock(&jsk->sk_session_queue_lock); lock(&priv->active_session_list_lock); *** DEADLOCK *** 3 locks held by ksoftirqd/1/21: #0: ffffffff8b187a80 (rcu_read_lock){....}-{1:2}, at: __skb_unlink include/linux/skbuff.h:2352 [inline] #0: ffffffff8b187a80 (rcu_read_lock){....}-{1:2}, at: __skb_dequeue include/linux/skbuff.h:2367 [inline] #0: ffffffff8b187a80 (rcu_read_lock){....}-{1:2}, at: process_backlog+0x1cd/0x5c0 net/core/dev.c:5931 #1: ffffffff8b187a80 (rcu_read_lock){....}-{1:2}, at: can_receive+0x1c4/0x4b0 net/can/af_can.c:653 #2: ffff8880783125c8 (&jsk->sk_session_queue_lock){+.-.}-{2:2}, at: spin_lock_bh include/linux/spinlock.h:355 [inline] #2: ffff8880783125c8 (&jsk->sk_session_queue_lock){+.-.}-{2:2}, at: j1939_sk_queue_activate_next+0x4c/0x440 net/can/j1939/socket.c:207 stack backtrace: CPU: 1 PID: 21 Comm: ksoftirqd/1 Not tainted 6.3.0-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/14/2023 Call Trace: __dump_stack lib/dump_stack.c:88 [inline] dump_stack_lvl+0x60/0xa0 lib/dump_stack.c:106 check_noncircular+0x25f/0x2e0 kernel/locking/lockdep.c:2188 check_prev_add kernel/locking/lockdep.c:3108 [inline] check_prevs_add kernel/locking/lockdep.c:3227 [inline] validate_chain kernel/locking/lockdep.c:3842 [inline] __lock_acquire+0x2f21/0x5df0 kernel/locking/lockdep.c:5074 lock_acquire kernel/locking/lockdep.c:5691 [inline] lock_acquire+0x1ad/0x520 kernel/locking/lockdep.c:5656 __raw_spin_lock_bh include/linux/spinlock_api_smp.h:126 [inline] _raw_spin_lock_bh+0x2f/0x40 kernel/locking/spinlock.c:178 spin_lock_bh include/linux/spinlock.h:355 [inline] j1939_session_list_lock net/can/j1939/transport.c:238 [inline] j1939_session_activate+0x3e/0x3d0 net/can/j1939/transport.c:1564 j1939_sk_queue_activate_next_locked net/can/j1939/socket.c:181 [inline] j1939_sk_queue_activate_next+0x24d/0x440 net/can/j1939/socket.c:208 j1939_xtp_rx_eoma_one net/can/j1939/transport.c:1395 [inline] j1939_xtp_rx_eoma+0x21d/0x550 net/can/j1939/transport.c:1410 j1939_tp_cmd_recv net/can/j1939/transport.c:2099 [inline] j1939_tp_recv+0x699/0xc10 net/can/j1939/transport.c:2144 j1939_can_recv net/can/j1939/main.c:112 [inline] j1939_can_recv+0x60b/0x8a0 net/can/j1939/main.c:38 deliver net/can/af_can.c:572 [inline] can_rcv_filter+0x4ce/0x7b0 net/can/af_can.c:606 can_receive+0x2ae/0x4b0 net/can/af_can.c:663 can_rcv+0x15a/0x220 net/can/af_can.c:687 __netif_receive_skb_one_core+0x104/0x180 net/core/dev.c:5491 process_backlog+0xca/0x5c0 net/core/dev.c:5933 __napi_poll+0x9a/0x550 net/core/dev.c:6496 napi_poll net/core/dev.c:6563 [inline] net_rx_action+0x792/0xbb0 net/core/dev.c:6696 __do_softirq+0x1d0/0x901 kernel/softirq.c:571 run_ksoftirqd kernel/softirq.c:939 [inline] run_ksoftirqd+0x2d/0x60 kernel/softirq.c:931 smpboot_thread_fn+0x548/0x8c0 kernel/smpboot.c:164 kthread+0x2e6/0x3c0 kernel/kthread.c:379 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308