syzbot


possible deadlock in sch_direct_xmit (2)

Status: upstream: reported on 2024/02/22 19:25
Reported-by: syzbot+3f44bf8b6f083aa47b0a@syzkaller.appspotmail.com
First crash: 69d, last: 7d19h
Similar bugs (11)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-6.1 possible deadlock in sch_direct_xmit (2) origin:lts-only C 10 1d04h 113d 0/3 upstream: reported C repro on 2024/01/09 18:28
android-44 possible deadlock in sch_direct_xmit C 240 1611d 1848d 0/2 public: reported C repro on 2019/04/11 08:44
upstream possible deadlock in sch_direct_xmit (2) net C done unreliable 109 289d 1464d 0/26 auto-obsoleted due to no activity on 2024/01/14 06:05
linux-4.19 possible deadlock in sch_direct_xmit (2) C error 15 432d 949d 0/1 upstream: reported C repro on 2021/09/26 01:30
upstream possible deadlock in sch_direct_xmit net C done done 1548 1618d 2297d 15/26 fixed on 2020/04/17 19:57
linux-4.14 possible deadlock in sch_direct_xmit 1 1795d 1795d 0/1 auto-closed as invalid on 2019/10/25 08:40
linux-4.14 possible deadlock in sch_direct_xmit (2) 1 1628d 1628d 0/1 auto-closed as invalid on 2020/03/15 19:58
linux-4.19 possible deadlock in sch_direct_xmit 1 1796d 1796d 0/1 auto-closed as invalid on 2019/10/25 08:50
linux-5.15 possible deadlock in sch_direct_xmit 1 356d 356d 0/3 auto-obsoleted due to no activity on 2023/08/23 09:09
linux-6.1 possible deadlock in sch_direct_xmit 2 364d 403d 0/3 auto-obsoleted due to no activity on 2023/08/23 09:10
upstream possible deadlock in sch_direct_xmit (3) net 1 80d 80d 26/26 fixed on 2024/04/10 16:40

Sample crash report:
============================================
WARNING: possible recursive locking detected
5.15.152-syzkaller #0 Not tainted
--------------------------------------------
syz-executor.1/4918 is trying to acquire lock:
ffff888167574398 (_xmit_ETHER#2){+.-.}-{2:2}, at: spin_lock include/linux/spinlock.h:363 [inline]
ffff888167574398 (_xmit_ETHER#2){+.-.}-{2:2}, at: __netif_tx_lock include/linux/netdevice.h:4429 [inline]
ffff888167574398 (_xmit_ETHER#2){+.-.}-{2:2}, at: sch_direct_xmit+0x1c0/0x5e0 net/sched/sch_generic.c:340

but task is already holding lock:
ffff88801a72f898 (_xmit_ETHER#2){+.-.}-{2:2}, at: spin_lock include/linux/spinlock.h:363 [inline]
ffff88801a72f898 (_xmit_ETHER#2){+.-.}-{2:2}, at: __netif_tx_lock include/linux/netdevice.h:4429 [inline]
ffff88801a72f898 (_xmit_ETHER#2){+.-.}-{2:2}, at: sch_direct_xmit+0x1c0/0x5e0 net/sched/sch_generic.c:340

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock(_xmit_ETHER#2);
  lock(_xmit_ETHER#2);

 *** DEADLOCK ***

 May be due to missing lock nesting notation

12 locks held by syz-executor.1/4918:
 #0: ffffffff8c91f720 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x5/0x30 include/linux/rcupdate.h:268
 #1: ffffffff8c91f780 (rcu_read_lock_bh){....}-{1:2}, at: rcu_lock_acquire+0x9/0x30 include/linux/rcupdate.h:269
 #2: ffffffff8c91f780 (rcu_read_lock_bh){....}-{1:2}, at: rcu_lock_acquire+0x9/0x30 include/linux/rcupdate.h:269
 #3: ffff888078a70258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: spin_trylock include/linux/spinlock.h:373 [inline]
 #3: ffff888078a70258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: qdisc_run_begin include/net/sch_generic.h:173 [inline]
 #3: ffff888078a70258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: __dev_xmit_skb net/core/dev.c:3831 [inline]
 #3: ffff888078a70258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: __dev_queue_xmit+0x11f2/0x3230 net/core/dev.c:4213
 #4: ffff88801a72f898 (_xmit_ETHER#2){+.-.}-{2:2}, at: spin_lock include/linux/spinlock.h:363 [inline]
 #4: ffff88801a72f898 (_xmit_ETHER#2){+.-.}-{2:2}, at: __netif_tx_lock include/linux/netdevice.h:4429 [inline]
 #4: ffff88801a72f898 (_xmit_ETHER#2){+.-.}-{2:2}, at: sch_direct_xmit+0x1c0/0x5e0 net/sched/sch_generic.c:340
 #5: ffff88807b4424e0 (k-slock-AF_INET6){+.-.}-{2:2}, at: spin_trylock include/linux/spinlock.h:373 [inline]
 #5: ffff88807b4424e0 (k-slock-AF_INET6){+.-.}-{2:2}, at: icmpv6_xmit_lock net/ipv6/icmp.c:118 [inline]
 #5: ffff88807b4424e0 (k-slock-AF_INET6){+.-.}-{2:2}, at: icmp6_send+0xcb5/0x21e0 net/ipv6/icmp.c:551
 #6: ffffffff8c91f720 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x5/0x30 include/linux/rcupdate.h:268
 #7: ffffffff8c91f780 (rcu_read_lock_bh){....}-{1:2}, at: rcu_lock_acquire+0x9/0x30 include/linux/rcupdate.h:269
 #8: ffffffff8c91f720 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x5/0x30 include/linux/rcupdate.h:268
 #9: ffffffff8c91f780 (rcu_read_lock_bh){....}-{1:2}, at: rcu_lock_acquire+0x9/0x30 include/linux/rcupdate.h:269
 #10: ffffffff8c91f780 (rcu_read_lock_bh){....}-{1:2}, at: rcu_lock_acquire+0x9/0x30 include/linux/rcupdate.h:269
 #11: ffff888024715258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: spin_trylock include/linux/spinlock.h:373 [inline]
 #11: ffff888024715258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: qdisc_run_begin include/net/sch_generic.h:173 [inline]
 #11: ffff888024715258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: __dev_xmit_skb net/core/dev.c:3831 [inline]
 #11: ffff888024715258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: __dev_queue_xmit+0x11f2/0x3230 net/core/dev.c:4213

stack backtrace:
CPU: 1 PID: 4918 Comm: syz-executor.1 Not tainted 5.15.152-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 02/29/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e3/0x2cb lib/dump_stack.c:106
 print_deadlock_bug kernel/locking/lockdep.c:2946 [inline]
 check_deadlock kernel/locking/lockdep.c:2989 [inline]
 validate_chain+0x46d2/0x5930 kernel/locking/lockdep.c:3775
 __lock_acquire+0x1295/0x1ff0 kernel/locking/lockdep.c:5012
 lock_acquire+0x1db/0x4f0 kernel/locking/lockdep.c:5623
 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
 _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:154
 spin_lock include/linux/spinlock.h:363 [inline]
 __netif_tx_lock include/linux/netdevice.h:4429 [inline]
 sch_direct_xmit+0x1c0/0x5e0 net/sched/sch_generic.c:340
 __dev_xmit_skb net/core/dev.c:3844 [inline]
 __dev_queue_xmit+0x18ee/0x3230 net/core/dev.c:4213
 neigh_output include/net/neighbour.h:509 [inline]
 ip6_finish_output2+0xee8/0x14f0 net/ipv6/ip6_output.c:126
 dst_output include/net/dst.h:443 [inline]
 NF_HOOK include/linux/netfilter.h:302 [inline]
 ndisc_send_skb+0xae0/0x13c0 net/ipv6/ndisc.c:509
 ndisc_send_ns+0x64d/0x830 net/ipv6/ndisc.c:651
 ndisc_solicit+0x3cd/0x570
 neigh_probe net/core/neighbour.c:1017 [inline]
 __neigh_event_send+0xe8e/0x1470 net/core/neighbour.c:1178
 neigh_event_send include/net/neighbour.h:438 [inline]
 neigh_resolve_output+0x1d3/0x6d0 net/core/neighbour.c:1488
 neigh_output include/net/neighbour.h:509 [inline]
 ip6_finish_output2+0xee8/0x14f0 net/ipv6/ip6_output.c:126
 ip6_send_skb+0x12b/0x240 net/ipv6/ip6_output.c:1943
 icmp6_send+0x1732/0x21e0 net/ipv6/icmp.c:630
 __icmpv6_send include/linux/icmpv6.h:28 [inline]
 icmpv6_send include/linux/icmpv6.h:49 [inline]
 ip6_link_failure+0x37/0x4a0 net/ipv6/route.c:2790
 ip_tunnel_xmit+0x1cee/0x27b0 net/ipv4/ip_tunnel.c:830
 __gre_xmit net/ipv4/ip_gre.c:469 [inline]
 erspan_xmit+0xa9c/0x1530 net/ipv4/ip_gre.c:718
 __netdev_start_xmit include/linux/netdevice.h:5019 [inline]
 netdev_start_xmit include/linux/netdevice.h:5033 [inline]
 xmit_one net/core/dev.c:3617 [inline]
 dev_hard_start_xmit+0x298/0x7a0 net/core/dev.c:3633
 sch_direct_xmit+0x2b2/0x5e0 net/sched/sch_generic.c:342
 __dev_xmit_skb net/core/dev.c:3844 [inline]
 __dev_queue_xmit+0x18ee/0x3230 net/core/dev.c:4213
 neigh_output include/net/neighbour.h:509 [inline]
 ip6_finish_output2+0xee8/0x14f0 net/ipv6/ip6_output.c:126
 dst_output include/net/dst.h:443 [inline]
 NF_HOOK include/linux/netfilter.h:302 [inline]
 rawv6_send_hdrinc+0xceb/0x1800 net/ipv6/raw.c:691
 rawv6_sendmsg+0x160e/0x2100 net/ipv6/raw.c:949
 sock_sendmsg_nosec net/socket.c:704 [inline]
 __sock_sendmsg net/socket.c:716 [inline]
 ____sys_sendmsg+0x59e/0x8f0 net/socket.c:2431
 ___sys_sendmsg+0x252/0x2e0 net/socket.c:2485
 __sys_sendmmsg+0x2bf/0x560 net/socket.c:2571
 __do_sys_sendmmsg net/socket.c:2600 [inline]
 __se_sys_sendmmsg net/socket.c:2597 [inline]
 __x64_sys_sendmmsg+0x9c/0xb0 net/socket.c:2597
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x61/0xcb
RIP: 0033:0x7fb0d9c2cda9
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 e1 20 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fb0d81ad0c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000133
RAX: ffffffffffffffda RBX: 00007fb0d9d5af80 RCX: 00007fb0d9c2cda9
RDX: 0000000000000002 RSI: 0000000020001e80 RDI: 0000000000000003
RBP: 00007fb0d9c7947a R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 000000000000000b R14: 00007fb0d9d5af80 R15: 00007ffc14e8be78
 </TASK>

Crashes (4):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/03/24 11:42 linux-5.15.y b95c01af2113 0ea90952 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in sch_direct_xmit
2024/04/24 13:33 linux-5.15.y c52b9710c83d 21339d7b .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in sch_direct_xmit
2024/03/27 21:07 linux-5.15.y 9465fef4ae35 120789fd .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in sch_direct_xmit
2024/02/22 19:24 linux-5.15.y 6139f2a02fe0 8d446f15 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in sch_direct_xmit
* Struck through repros no longer work on HEAD.