bisecting fixing commit since 205a42ce2861f2d0dea8eb5090d05262e1cfa049 building syzkaller on 8df85ed9883abc2a200858f44f22c11c602d218a testing commit 205a42ce2861f2d0dea8eb5090d05262e1cfa049 with gcc (GCC) 8.4.1 20210217 kernel signature: 50635e37810a2ccefd2f330be3bffb661167cb8dd60a29d43c378e8716d0862b all runs: crashed: possible deadlock in __dev_queue_xmit testing current HEAD 125222814e7b8f84df767d6ab622aff2a6d2f234 testing commit 125222814e7b8f84df767d6ab622aff2a6d2f234 with gcc (GCC) 8.4.1 20210217 kernel signature: 66c5b0a5d570737e6cba67586a6a36b50413ea9042ab7ee6a7014e1f146e0075 all runs: crashed: possible deadlock in __dev_queue_xmit revisions tested: 2, total time: 22m45.105073362s (build: 16m11.587822033s, test: 6m3.956703807s) the crash still happens on HEAD commit msg: Linux 4.19.182 crash: possible deadlock in __dev_queue_xmit Bluetooth: hci4: command 0x0409 tx timeout Bluetooth: hci3: command 0x0409 tx timeout Bluetooth: hci2: command 0x0409 tx timeout Bluetooth: hci5: command 0x0409 tx timeout ============================================ WARNING: possible recursive locking detected 4.19.182-syzkaller #0 Not tainted -------------------------------------------- syz-executor.0/7252 is trying to acquire lock: 00000000233cee82 (_xmit_ETHER#2){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline] 00000000233cee82 (_xmit_ETHER#2){+.-.}, at: __netif_tx_lock include/linux/netdevice.h:3838 [inline] 00000000233cee82 (_xmit_ETHER#2){+.-.}, at: __dev_queue_xmit+0x21bb/0x2900 net/core/dev.c:3834 but task is already holding lock: 000000006b3a355d (_xmit_ETHER#2){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline] 000000006b3a355d (_xmit_ETHER#2){+.-.}, at: __netif_tx_lock include/linux/netdevice.h:3838 [inline] 000000006b3a355d (_xmit_ETHER#2){+.-.}, at: sch_direct_xmit+0x1e8/0x1790 net/sched/sch_generic.c:330 other info that might help us debug this: Possible unsafe locking scenario: CPU0 ---- lock(_xmit_ETHER#2); lock(_xmit_ETHER#2); *** DEADLOCK *** May be due to missing lock nesting notation 10 locks held by syz-executor.0/7252: #0: 000000002495de1f (rcu_read_lock){....}, at: l3mdev_l3_out include/net/l3mdev.h:172 [inline] #0: 000000002495de1f (rcu_read_lock){....}, at: l3mdev_ip6_out include/net/l3mdev.h:193 [inline] #0: 000000002495de1f (rcu_read_lock){....}, at: rawv6_send_hdrinc net/ipv6/raw.c:682 [inline] #0: 000000002495de1f (rcu_read_lock){....}, at: rawv6_sendmsg+0x17e4/0x32f0 net/ipv6/raw.c:947 #1: 000000007a913885 (rcu_read_lock_bh){....}, at: lwtunnel_xmit_redirect include/net/lwtunnel.h:92 [inline] #1: 000000007a913885 (rcu_read_lock_bh){....}, at: ip6_finish_output2+0x1cc/0x2080 net/ipv6/ip6_output.c:106 #2: 000000007a913885 (rcu_read_lock_bh){....}, at: __dev_queue_xmit+0x20b/0x2900 net/core/dev.c:3773 #3: 000000008b5ae4f9 (&(&sch->seqlock)->rlock){+...}, at: spin_trylock include/linux/spinlock.h:339 [inline] #3: 000000008b5ae4f9 (&(&sch->seqlock)->rlock){+...}, at: qdisc_run_begin include/net/sch_generic.h:130 [inline] #3: 000000008b5ae4f9 (&(&sch->seqlock)->rlock){+...}, at: qdisc_run include/net/pkt_sched.h:119 [inline] #3: 000000008b5ae4f9 (&(&sch->seqlock)->rlock){+...}, at: __dev_xmit_skb net/core/dev.c:3451 [inline] #3: 000000008b5ae4f9 (&(&sch->seqlock)->rlock){+...}, at: __dev_queue_xmit+0x2109/0x2900 net/core/dev.c:3807 #4: 00000000500825ad (dev->qdisc_running_key ?: &qdisc_running_key){+...}, at: dev_queue_xmit+0xb/0x10 net/core/dev.c:3871 #5: 000000006b3a355d (_xmit_ETHER#2){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline] #5: 000000006b3a355d (_xmit_ETHER#2){+.-.}, at: __netif_tx_lock include/linux/netdevice.h:3838 [inline] #5: 000000006b3a355d (_xmit_ETHER#2){+.-.}, at: sch_direct_xmit+0x1e8/0x1790 net/sched/sch_generic.c:330 #6: 000000003a2e6cb6 (k-slock-AF_INET6){+...}, at: spin_trylock include/linux/spinlock.h:339 [inline] #6: 000000003a2e6cb6 (k-slock-AF_INET6){+...}, at: icmpv6_xmit_lock net/ipv6/icmp.c:119 [inline] #6: 000000003a2e6cb6 (k-slock-AF_INET6){+...}, at: icmp6_send+0xc42/0x1e50 net/ipv6/icmp.c:532 #7: 000000002495de1f (rcu_read_lock){....}, at: icmp6_send+0x1197/0x1e50 net/ipv6/icmp.c:565 #8: 000000007a913885 (rcu_read_lock_bh){....}, at: lwtunnel_xmit_redirect include/net/lwtunnel.h:92 [inline] #8: 000000007a913885 (rcu_read_lock_bh){....}, at: ip6_finish_output2+0x1cc/0x2080 net/ipv6/ip6_output.c:106 #9: 000000007a913885 (rcu_read_lock_bh){....}, at: __dev_queue_xmit+0x20b/0x2900 net/core/dev.c:3773 stack backtrace: CPU: 0 PID: 7252 Comm: syz-executor.0 Not tainted 4.19.182-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x17c/0x226 lib/dump_stack.c:118 print_deadlock_bug kernel/locking/lockdep.c:1763 [inline] check_deadlock kernel/locking/lockdep.c:1807 [inline] validate_chain kernel/locking/lockdep.c:2403 [inline] __lock_acquire.cold.40+0x344/0x744 kernel/locking/lockdep.c:3415 lock_acquire+0x180/0x3a0 kernel/locking/lockdep.c:3907 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline] _raw_spin_lock+0x2d/0x40 kernel/locking/spinlock.c:144 spin_lock include/linux/spinlock.h:329 [inline] __netif_tx_lock include/linux/netdevice.h:3838 [inline] __dev_queue_xmit+0x21bb/0x2900 net/core/dev.c:3834 dev_queue_xmit+0xb/0x10 net/core/dev.c:3871 neigh_resolve_output+0x4b5/0x830 net/core/neighbour.c:1374 neigh_output include/net/neighbour.h:501 [inline] ip6_finish_output2+0xf4e/0x2080 net/ipv6/ip6_output.c:120 ip6_finish_output+0x62e/0xc70 net/ipv6/ip6_output.c:192 NF_HOOK_COND include/linux/netfilter.h:278 [inline] ip6_output+0x1bb/0x610 net/ipv6/ip6_output.c:209 dst_output include/net/dst.h:455 [inline] ip6_local_out+0x74/0x110 net/ipv6/output_core.c:178 ip6_send_skb+0x92/0x2a0 net/ipv6/ip6_output.c:1737 ip6_push_pending_frames+0x94/0xb0 net/ipv6/ip6_output.c:1757 icmpv6_push_pending_frames+0x24a/0x480 net/ipv6/icmp.c:288 icmp6_send+0x17db/0x1e50 net/ipv6/icmp.c:584 __icmpv6_send include/linux/icmpv6.h:28 [inline] icmpv6_send include/linux/icmpv6.h:49 [inline] ip6_link_failure+0x24/0x370 net/ipv6/route.c:2297 dst_link_failure include/net/dst.h:438 [inline] ip_tunnel_xmit+0x1da1/0x2f30 net/ipv4/ip_tunnel.c:788 __gre_xmit+0x588/0x8e0 net/ipv4/ip_gre.c:447 erspan_xmit+0x9fd/0x2f20 net/ipv4/ip_gre.c:754 __netdev_start_xmit include/linux/netdevice.h:4345 [inline] netdev_start_xmit include/linux/netdevice.h:4359 [inline] xmit_one net/core/dev.c:3256 [inline] dev_hard_start_xmit+0x160/0x6e0 net/core/dev.c:3272 sch_direct_xmit+0x286/0x1790 net/sched/sch_generic.c:332 qdisc_restart net/sched/sch_generic.c:395 [inline] __qdisc_run+0x3ec/0x1640 net/sched/sch_generic.c:403 qdisc_run include/net/pkt_sched.h:120 [inline] __dev_xmit_skb net/core/dev.c:3451 [inline] __dev_queue_xmit+0x1aa2/0x2900 net/core/dev.c:3807 dev_queue_xmit+0xb/0x10 net/core/dev.c:3871 neigh_resolve_output+0x4b5/0x830 net/core/neighbour.c:1374 neigh_output include/net/neighbour.h:501 [inline] ip6_finish_output2+0xf4e/0x2080 net/ipv6/ip6_output.c:120 ip6_finish_output+0x62e/0xc70 net/ipv6/ip6_output.c:192 NF_HOOK_COND include/linux/netfilter.h:278 [inline] ip6_output+0x1bb/0x610 net/ipv6/ip6_output.c:209 dst_output include/net/dst.h:455 [inline] NF_HOOK include/linux/netfilter.h:289 [inline] rawv6_send_hdrinc net/ipv6/raw.c:692 [inline] rawv6_sendmsg+0x1991/0x32f0 net/ipv6/raw.c:947 inet_sendmsg+0x108/0x440 net/ipv4/af_inet.c:798 sock_sendmsg_nosec net/socket.c:622 [inline] sock_sendmsg+0xac/0xf0 net/socket.c:632 sock_write_iter+0x215/0x420 net/socket.c:901 call_write_iter include/linux/fs.h:1821 [inline] new_sync_write fs/read_write.c:474 [inline] __vfs_write+0x443/0x890 fs/read_write.c:487 vfs_write+0x150/0x4d0 fs/read_write.c:549 ksys_write+0x103/0x260 fs/read_write.c:599 __do_sys_write fs/read_write.c:611 [inline] __se_sys_write fs/read_write.c:608 [inline] __x64_sys_write+0x6e/0xb0 fs/read_write.c:608 do_syscall_64+0xd0/0x4e0 arch/x86/entry/common.c:293 entry_SYSCALL_64_after_hwframe+0x49/0xbe RIP: 0033:0x464709 Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 bc ff ff ff f7 d8 64 89 01 48 RSP: 002b:00007f8e34889188 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 RAX: ffffffffffffffda RBX: 000000000055bf00 RCX: 0000000000464709 RDX: 0000000000000028 RSI: 0000000020000140 RDI: 0000000000000005 RBP: 00000000004ae608 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 000000000055bf00 R13: 00007ffce610136f R14: 00007f8e34889300 R15: 0000000000022000 syz-executor.0 (7252) used greatest stack depth: 22056 bytes left Bluetooth: hci0: command 0x041b tx timeout Bluetooth: hci1: command 0x041b tx timeout Bluetooth: hci2: command 0x041b tx timeout Bluetooth: hci3: command 0x041b tx timeout Bluetooth: hci4: command 0x041b tx timeout Bluetooth: hci5: command 0x041b tx timeout Bluetooth: hci1: command 0x040f tx timeout Bluetooth: hci0: command 0x040f tx timeout Bluetooth: hci4: command 0x040f tx timeout Bluetooth: hci3: command 0x040f tx timeout Bluetooth: hci2: command 0x040f tx timeout Bluetooth: hci5: command 0x040f tx timeout Bluetooth: hci0: command 0x0419 tx timeout Bluetooth: hci1: command 0x0419 tx timeout Bluetooth: hci2: command 0x0419 tx timeout Bluetooth: hci3: command 0x0419 tx timeout Bluetooth: hci4: command 0x0419 tx timeout Bluetooth: hci5: command 0x0419 tx timeout