bisecting fixing commit since 205a42ce2861f2d0dea8eb5090d05262e1cfa049 building syzkaller on 8df85ed9883abc2a200858f44f22c11c602d218a testing commit 205a42ce2861f2d0dea8eb5090d05262e1cfa049 with gcc (GCC) 8.1.0 kernel signature: a87b6d41d88f4a01fb094a2254d698397c0c3208529a964e9cfbcf9367236443 all runs: crashed: possible deadlock in __dev_queue_xmit testing current HEAD 10ad6cfd57360760116cde00a8ef756e121367a9 testing commit 10ad6cfd57360760116cde00a8ef756e121367a9 with gcc (GCC) 8.1.0 kernel signature: 8764ea40877c4eed3b643ee2f54127d3a9e3c41802adb3ad3967351a4ad3c20e all runs: crashed: possible deadlock in __dev_queue_xmit revisions tested: 2, total time: 24m8.71585052s (build: 17m31.939390891s, test: 5m53.193670491s) the crash still happens on HEAD commit msg: Linux 4.19.148 crash: possible deadlock in __dev_queue_xmit batman_adv: batadv0: Interface activated: batadv_slave_1 IPv6: ADDRCONF(NETDEV_CHANGE): batadv_slave_1: link becomes ready IPv6: ADDRCONF(NETDEV_CHANGE): veth1_to_batadv: link becomes ready ============================================ WARNING: possible recursive locking detected 4.19.148-syzkaller #0 Not tainted -------------------------------------------- syz-executor.0/8074 is trying to acquire lock: 00000000962b59e9 (_xmit_ETHER#2){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline] 00000000962b59e9 (_xmit_ETHER#2){+.-.}, at: __netif_tx_lock include/linux/netdevice.h:3828 [inline] 00000000962b59e9 (_xmit_ETHER#2){+.-.}, at: __dev_queue_xmit+0x21bb/0x2900 net/core/dev.c:3834 but task is already holding lock: 0000000016b6923d (_xmit_ETHER#2){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline] 0000000016b6923d (_xmit_ETHER#2){+.-.}, at: __netif_tx_lock include/linux/netdevice.h:3828 [inline] 0000000016b6923d (_xmit_ETHER#2){+.-.}, at: sch_direct_xmit+0x235/0x1790 net/sched/sch_generic.c:330 other info that might help us debug this: Possible unsafe locking scenario: CPU0 ---- lock(_xmit_ETHER#2); lock(_xmit_ETHER#2); *** DEADLOCK *** May be due to missing lock nesting notation 11 locks held by syz-executor.0/8074: #0: 000000005582edcf (rcu_read_lock){....}, at: l3mdev_l3_out include/net/l3mdev.h:172 [inline] #0: 000000005582edcf (rcu_read_lock){....}, at: l3mdev_ip6_out include/net/l3mdev.h:193 [inline] #0: 000000005582edcf (rcu_read_lock){....}, at: rawv6_send_hdrinc net/ipv6/raw.c:682 [inline] #0: 000000005582edcf (rcu_read_lock){....}, at: rawv6_sendmsg+0x17de/0x32e0 net/ipv6/raw.c:947 #1: 00000000c41f73ec (rcu_read_lock_bh){....}, at: lwtunnel_xmit_redirect include/net/lwtunnel.h:92 [inline] #1: 00000000c41f73ec (rcu_read_lock_bh){....}, at: ip6_finish_output2+0x1cc/0x2080 net/ipv6/ip6_output.c:106 #2: 00000000c41f73ec (rcu_read_lock_bh){....}, at: __dev_queue_xmit+0x20b/0x2900 net/core/dev.c:3773 #3: 0000000005a2ccd1 (&(&sch->seqlock)->rlock){+...}, at: spin_trylock include/linux/spinlock.h:339 [inline] #3: 0000000005a2ccd1 (&(&sch->seqlock)->rlock){+...}, at: qdisc_run_begin include/net/sch_generic.h:130 [inline] #3: 0000000005a2ccd1 (&(&sch->seqlock)->rlock){+...}, at: qdisc_run include/net/pkt_sched.h:119 [inline] #3: 0000000005a2ccd1 (&(&sch->seqlock)->rlock){+...}, at: __dev_xmit_skb net/core/dev.c:3451 [inline] #3: 0000000005a2ccd1 (&(&sch->seqlock)->rlock){+...}, at: __dev_queue_xmit+0x2109/0x2900 net/core/dev.c:3807 #4: 00000000d6248060 (dev->qdisc_running_key ?: &qdisc_running_key){+...}, at: dev_queue_xmit+0xb/0x10 net/core/dev.c:3871 #5: 0000000016b6923d (_xmit_ETHER#2){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline] #5: 0000000016b6923d (_xmit_ETHER#2){+.-.}, at: __netif_tx_lock include/linux/netdevice.h:3828 [inline] #5: 0000000016b6923d (_xmit_ETHER#2){+.-.}, at: sch_direct_xmit+0x235/0x1790 net/sched/sch_generic.c:330 #6: 000000005582edcf (rcu_read_lock){....}, at: icmpv6_send+0x0/0x160 net/ipv6/ip6_icmp.c:31 #7: 000000007ca2f769 (k-slock-AF_INET6){+...}, at: spin_trylock include/linux/spinlock.h:339 [inline] #7: 000000007ca2f769 (k-slock-AF_INET6){+...}, at: icmpv6_xmit_lock net/ipv6/icmp.c:119 [inline] #7: 000000007ca2f769 (k-slock-AF_INET6){+...}, at: icmp6_send+0xc2a/0x1e20 net/ipv6/icmp.c:532 #8: 000000005582edcf (rcu_read_lock){....}, at: icmp6_send+0x1180/0x1e20 net/ipv6/icmp.c:565 #9: 00000000c41f73ec (rcu_read_lock_bh){....}, at: lwtunnel_xmit_redirect include/net/lwtunnel.h:92 [inline] #9: 00000000c41f73ec (rcu_read_lock_bh){....}, at: ip6_finish_output2+0x1cc/0x2080 net/ipv6/ip6_output.c:106 #10: 00000000c41f73ec (rcu_read_lock_bh){....}, at: __dev_queue_xmit+0x20b/0x2900 net/core/dev.c:3773 stack backtrace: CPU: 1 PID: 8074 Comm: syz-executor.0 Not tainted 4.19.148-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x17c/0x22a lib/dump_stack.c:118 print_deadlock_bug kernel/locking/lockdep.c:1763 [inline] check_deadlock kernel/locking/lockdep.c:1807 [inline] validate_chain kernel/locking/lockdep.c:2403 [inline] __lock_acquire.cold.40+0x359/0x740 kernel/locking/lockdep.c:3415 lock_acquire+0x180/0x3a0 kernel/locking/lockdep.c:3907 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline] _raw_spin_lock+0x2d/0x40 kernel/locking/spinlock.c:144 spin_lock include/linux/spinlock.h:329 [inline] __netif_tx_lock include/linux/netdevice.h:3828 [inline] __dev_queue_xmit+0x21bb/0x2900 net/core/dev.c:3834 dev_queue_xmit+0xb/0x10 net/core/dev.c:3871 neigh_resolve_output+0x4b5/0x830 net/core/neighbour.c:1374 neigh_output include/net/neighbour.h:501 [inline] ip6_finish_output2+0xf4e/0x2080 net/ipv6/ip6_output.c:120 ip6_finish_output+0x64e/0xa10 net/ipv6/ip6_output.c:154 NF_HOOK_COND include/linux/netfilter.h:278 [inline] ip6_output+0x1bb/0x610 net/ipv6/ip6_output.c:171 dst_output include/net/dst.h:455 [inline] ip6_local_out+0x74/0x110 net/ipv6/output_core.c:178 ip6_send_skb+0x92/0x2a0 net/ipv6/ip6_output.c:1699 ip6_push_pending_frames+0x94/0xb0 net/ipv6/ip6_output.c:1719 icmpv6_push_pending_frames+0x24a/0x480 net/ipv6/icmp.c:288 icmp6_send+0x17c6/0x1e20 net/ipv6/icmp.c:584 icmpv6_send+0x89/0x160 net/ipv6/ip6_icmp.c:43 ip6_link_failure+0x1d/0x360 net/ipv6/route.c:2297 dst_link_failure include/net/dst.h:438 [inline] ip_tunnel_xmit+0x539/0x30e0 net/ipv4/ip_tunnel.c:789 __gre_xmit+0x588/0x8e0 net/ipv4/ip_gre.c:447 erspan_xmit+0x95c/0x2f30 net/ipv4/ip_gre.c:756 __netdev_start_xmit include/linux/netdevice.h:4333 [inline] netdev_start_xmit include/linux/netdevice.h:4347 [inline] xmit_one net/core/dev.c:3256 [inline] dev_hard_start_xmit+0x160/0x6e0 net/core/dev.c:3272 sch_direct_xmit+0x2d3/0x1790 net/sched/sch_generic.c:332 qdisc_restart net/sched/sch_generic.c:395 [inline] __qdisc_run+0x3ec/0x15c0 net/sched/sch_generic.c:403 qdisc_run include/net/pkt_sched.h:120 [inline] __dev_xmit_skb net/core/dev.c:3451 [inline] __dev_queue_xmit+0x1aa2/0x2900 net/core/dev.c:3807 dev_queue_xmit+0xb/0x10 net/core/dev.c:3871 neigh_resolve_output+0x4b5/0x830 net/core/neighbour.c:1374 neigh_output include/net/neighbour.h:501 [inline] ip6_finish_output2+0xf4e/0x2080 net/ipv6/ip6_output.c:120 ip6_finish_output+0x64e/0xa10 net/ipv6/ip6_output.c:154 NF_HOOK_COND include/linux/netfilter.h:278 [inline] ip6_output+0x1bb/0x610 net/ipv6/ip6_output.c:171 dst_output include/net/dst.h:455 [inline] NF_HOOK include/linux/netfilter.h:289 [inline] rawv6_send_hdrinc net/ipv6/raw.c:692 [inline] rawv6_sendmsg+0x198b/0x32e0 net/ipv6/raw.c:947 inet_sendmsg+0x108/0x440 net/ipv4/af_inet.c:798 sock_sendmsg_nosec net/socket.c:622 [inline] sock_sendmsg+0xac/0xf0 net/socket.c:632 sock_write_iter+0x215/0x420 net/socket.c:901 call_write_iter include/linux/fs.h:1821 [inline] new_sync_write fs/read_write.c:474 [inline] __vfs_write+0x443/0x890 fs/read_write.c:487 vfs_write+0x150/0x4d0 fs/read_write.c:549 ksys_write+0x103/0x260 fs/read_write.c:599 __do_sys_write fs/read_write.c:611 [inline] __se_sys_write fs/read_write.c:608 [inline] __x64_sys_write+0x6e/0xb0 fs/read_write.c:608 do_syscall_64+0xd0/0x4e0 arch/x86/entry/common.c:293 entry_SYSCALL_64_after_hwframe+0x49/0xbe RIP: 0033:0x45cc79 Code: 2d b6 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 fb b5 fb ff c3 66 2e 0f 1f 84 00 00 00 00 RSP: 002b:00007fd7976e8c78 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 RAX: ffffffffffffffda RBX: 00000000000358c0 RCX: 000000000045cc79 RDX: 0000000000000028 RSI: 0000000020000140 RDI: 0000000000000005 RBP: 000000000078bf40 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 000000000078bf0c R13: 00007ffd02a7799f R14: 00007fd7976e99c0 R15: 000000000078bf0c syz-executor.0 (8074) used greatest stack depth: 22024 bytes left Bluetooth: hci0: command 0x0409 tx timeout Bluetooth: hci2: command 0x0409 tx timeout Bluetooth: hci1: command 0x0409 tx timeout Bluetooth: hci5: command 0x0409 tx timeout Bluetooth: hci3: command 0x0409 tx timeout Bluetooth: hci4: command 0x0409 tx timeout Bluetooth: hci0: command 0x041b tx timeout Bluetooth: hci1: command 0x041b tx timeout Bluetooth: hci2: command 0x041b tx timeout Bluetooth: hci4: command 0x041b tx timeout Bluetooth: hci3: command 0x041b tx timeout Bluetooth: hci5: command 0x041b tx timeout Bluetooth: hci0: command 0x040f tx timeout Bluetooth: hci2: command 0x040f tx timeout Bluetooth: hci1: command 0x040f tx timeout Bluetooth: hci5: command 0x040f tx timeout Bluetooth: hci3: command 0x040f tx timeout Bluetooth: hci4: command 0x040f tx timeout Bluetooth: hci0: command 0x0419 tx timeout Bluetooth: hci1: command 0x0419 tx timeout Bluetooth: hci2: command 0x0419 tx timeout Bluetooth: hci4: command 0x0419 tx timeout Bluetooth: hci3: command 0x0419 tx timeout Bluetooth: hci5: command 0x0419 tx timeout