syzbot


possible deadlock in sch_direct_xmit (2)

Status: upstream: reported C repro on 2024/02/22 19:25
Bug presence: origin:lts-only
[Documentation on labels]
Reported-by: syzbot+3f44bf8b6f083aa47b0a@syzkaller.appspotmail.com
First crash: 284d, last: 5d10h
Bug presence (2)
Date Name Commit Repro Result
2024/05/18 linux-5.15.y (ToT) 83655231580b C [report] possible deadlock in sch_direct_xmit
2024/05/18 upstream (ToT) 4b377b4868ef C Didn't crash
Similar bugs (12)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-6.1 possible deadlock in sch_direct_xmit (2) origin:lts-only C done 19 14d 328d 0/3 upstream: reported C repro on 2024/01/09 18:28
android-44 possible deadlock in sch_direct_xmit C 240 1826d 2062d 0/2 public: reported C repro on 2019/04/11 08:44
upstream possible deadlock in sch_direct_xmit (2) net C done unreliable 109 504d 1679d 0/28 auto-obsoleted due to no activity on 2024/01/14 06:05
linux-4.19 possible deadlock in sch_direct_xmit (2) C error 15 647d 1164d 0/1 upstream: reported C repro on 2021/09/26 01:30
upstream possible deadlock in sch_direct_xmit net C done done 1548 1833d 2512d 15/28 fixed on 2020/04/17 19:57
linux-4.14 possible deadlock in sch_direct_xmit 1 2010d 2010d 0/1 auto-closed as invalid on 2019/10/25 08:40
upstream possible deadlock in sch_direct_xmit (4) net 1 219d 219d 25/28 fixed on 2024/06/05 13:52
linux-4.14 possible deadlock in sch_direct_xmit (2) 1 1843d 1843d 0/1 auto-closed as invalid on 2020/03/15 19:58
linux-4.19 possible deadlock in sch_direct_xmit 1 2011d 2011d 0/1 auto-closed as invalid on 2019/10/25 08:50
linux-5.15 possible deadlock in sch_direct_xmit 1 571d 571d 0/3 auto-obsoleted due to no activity on 2023/08/23 09:09
linux-6.1 possible deadlock in sch_direct_xmit 2 579d 617d 0/3 auto-obsoleted due to no activity on 2023/08/23 09:10
upstream possible deadlock in sch_direct_xmit (3) net 1 294d 294d 25/28 fixed on 2024/04/10 16:40
Last patch testing requests (1)
Created Duration User Patch Repo Result
2024/10/08 11:25 11m retest repro linux-5.15.y report log
Fix bisection attempts (4)
Created Duration User Patch Repo Result
2024/10/23 03:01 2m fix candidate upstream error job log
2024/09/17 14:38 1m fix candidate upstream error job log
2024/08/02 00:55 1m fix candidate upstream error job log
2024/05/25 13:09 1m fix candidate upstream error job log

Sample crash report:
============================================
WARNING: possible recursive locking detected
5.15.173-syzkaller #0 Not tainted
--------------------------------------------
syz-executor777/4418 is trying to acquire lock:
ffff0000cd212398 (_xmit_ETHER#2){+.-.}-{2:2}, at: spin_lock include/linux/spinlock.h:363 [inline]
ffff0000cd212398 (_xmit_ETHER#2){+.-.}-{2:2}, at: __netif_tx_lock include/linux/netdevice.h:4429 [inline]
ffff0000cd212398 (_xmit_ETHER#2){+.-.}-{2:2}, at: sch_direct_xmit+0x15c/0x484 net/sched/sch_generic.c:340

but task is already holding lock:
ffff0000da2a1498 (_xmit_ETHER#2){+.-.}-{2:2}, at: spin_lock include/linux/spinlock.h:363 [inline]
ffff0000da2a1498 (_xmit_ETHER#2){+.-.}-{2:2}, at: __netif_tx_lock include/linux/netdevice.h:4429 [inline]
ffff0000da2a1498 (_xmit_ETHER#2){+.-.}-{2:2}, at: sch_direct_xmit+0x15c/0x484 net/sched/sch_generic.c:340

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock(_xmit_ETHER#2);
  lock(_xmit_ETHER#2);

 *** DEADLOCK ***

 May be due to missing lock nesting notation

13 locks held by syz-executor777/4418:
 #0: ffff800014d322e0 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x10/0x4c include/linux/rcupdate.h:311
 #1: ffff800014d32340 (rcu_read_lock_bh){....}-{1:2}, at: rcu_lock_acquire+0x18/0x54 include/linux/rcupdate.h:311
 #2: ffff800014d32340 (rcu_read_lock_bh){....}-{1:2}, at: rcu_lock_acquire+0x18/0x54 include/linux/rcupdate.h:311
 #3: ffff0000cd74e258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: spin_trylock include/linux/spinlock.h:373 [inline]
 #3: ffff0000cd74e258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: qdisc_run_begin+0x130/0x2bc include/net/sch_generic.h:173
 #4: ffff0000da2a1498 (_xmit_ETHER#2){+.-.}-{2:2}, at: spin_lock include/linux/spinlock.h:363 [inline]
 #4: ffff0000da2a1498 (_xmit_ETHER#2){+.-.}-{2:2}, at: __netif_tx_lock include/linux/netdevice.h:4429 [inline]
 #4: ffff0000da2a1498 (_xmit_ETHER#2){+.-.}-{2:2}, at: sch_direct_xmit+0x15c/0x484 net/sched/sch_generic.c:340
 #5: ffff0000d3258f20 (k-slock-AF_INET6){+...}-{2:2}, at: spin_trylock include/linux/spinlock.h:373 [inline]
 #5: ffff0000d3258f20 (k-slock-AF_INET6){+...}-{2:2}, at: icmpv6_xmit_lock+0x100/0x188 net/ipv6/icmp.c:118
 #6: ffff800014d322e0 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x10/0x4c include/linux/rcupdate.h:311
 #7: ffff800014d322e0 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x18/0x54 include/linux/rcupdate.h:311
 #8: ffff800014d32340 (rcu_read_lock_bh){....}-{1:2}, at: rcu_lock_acquire+0x18/0x54 include/linux/rcupdate.h:311
 #9: ffff800014d322e0 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x10/0x4c include/linux/rcupdate.h:311
 #10: ffff800014d32340 (rcu_read_lock_bh){....}-{1:2}, at: rcu_lock_acquire+0x18/0x54 include/linux/rcupdate.h:311
 #11: ffff800014d32340 (rcu_read_lock_bh){....}-{1:2}, at: rcu_lock_acquire+0x18/0x54 include/linux/rcupdate.h:311
 #12: ffff0000ce5a9258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: spin_trylock include/linux/spinlock.h:373 [inline]
 #12: ffff0000ce5a9258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: qdisc_run_begin+0x130/0x2bc include/net/sch_generic.h:173

stack backtrace:
CPU: 0 PID: 4418 Comm: syz-executor777 Not tainted 5.15.173-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Call trace:
 dump_backtrace+0x0/0x530 arch/arm64/kernel/stacktrace.c:152
 show_stack+0x2c/0x3c arch/arm64/kernel/stacktrace.c:216
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x108/0x170 lib/dump_stack.c:106
 dump_stack+0x1c/0x58 lib/dump_stack.c:113
 __lock_acquire+0x62bc/0x7638 kernel/locking/lockdep.c:5012
 lock_acquire+0x240/0x77c kernel/locking/lockdep.c:5623
 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
 _raw_spin_lock+0xb0/0x10c kernel/locking/spinlock.c:154
 spin_lock include/linux/spinlock.h:363 [inline]
 __netif_tx_lock include/linux/netdevice.h:4429 [inline]
 sch_direct_xmit+0x15c/0x484 net/sched/sch_generic.c:340
 __dev_xmit_skb net/core/dev.c:3852 [inline]
 __dev_queue_xmit+0x1488/0x2ac8 net/core/dev.c:4221
 dev_queue_xmit+0x24/0x34 net/core/dev.c:4289
 neigh_hh_output include/net/neighbour.h:493 [inline]
 neigh_output include/net/neighbour.h:507 [inline]
 ip6_finish_output2+0x132c/0x1cec net/ipv6/ip6_output.c:130
 __ip6_finish_output+0x580/0x6ec net/ipv6/ip6_output.c:201
 ip6_finish_output+0x40/0x218 net/ipv6/ip6_output.c:211
 NF_HOOK_COND include/linux/netfilter.h:291 [inline]
 ip6_output+0x274/0x594 net/ipv6/ip6_output.c:234
 dst_output include/net/dst.h:443 [inline]
 NF_HOOK include/linux/netfilter.h:302 [inline]
 ndisc_send_skb+0xbf8/0x1788 net/ipv6/ndisc.c:511
 ndisc_send_ns+0x538/0x6ec net/ipv6/ndisc.c:653
 ndisc_solicit+0x2f4/0x47c
 neigh_probe+0xc4/0x138 net/core/neighbour.c:1017
 __neigh_event_send+0xca4/0x1338 net/core/neighbour.c:1178
 neigh_event_send include/net/neighbour.h:438 [inline]
 neigh_resolve_output+0x178/0x5dc net/core/neighbour.c:1488
 neigh_output include/net/neighbour.h:509 [inline]
 ip6_finish_output2+0x1360/0x1cec net/ipv6/ip6_output.c:130
 __ip6_finish_output+0x580/0x6ec net/ipv6/ip6_output.c:201
 ip6_finish_output+0x40/0x218 net/ipv6/ip6_output.c:211
 NF_HOOK_COND include/linux/netfilter.h:291 [inline]
 ip6_output+0x274/0x594 net/ipv6/ip6_output.c:234
 dst_output include/net/dst.h:443 [inline]
 ip6_local_out+0x120/0x160 net/ipv6/output_core.c:161
 ip6_send_skb+0x1a4/0x580 net/ipv6/ip6_output.c:1951
 ip6_push_pending_frames+0xd0/0x118 net/ipv6/ip6_output.c:1972
 icmpv6_push_pending_frames+0x244/0x398 net/ipv6/icmp.c:311
 icmp6_send+0x11a4/0x1b18 net/ipv6/icmp.c:630
 __icmpv6_send include/linux/icmpv6.h:28 [inline]
 icmpv6_send include/linux/icmpv6.h:49 [inline]
 ip6_link_failure+0x44/0x4a8 net/ipv6/route.c:2788
 dst_link_failure+0x11c/0x160 include/net/dst.h:422
 ip_tunnel_xmit+0x15f4/0x2184 net/ipv4/ip_tunnel.c:844
 __gre_xmit net/ipv4/ip_gre.c:474 [inline]
 erspan_xmit+0x9cc/0x14cc net/ipv4/ip_gre.c:723
 __netdev_start_xmit include/linux/netdevice.h:5019 [inline]
 netdev_start_xmit include/linux/netdevice.h:5033 [inline]
 xmit_one net/core/dev.c:3617 [inline]
 dev_hard_start_xmit+0x2bc/0x92c net/core/dev.c:3633
 sch_direct_xmit+0x2e0/0x484 net/sched/sch_generic.c:342
 __dev_xmit_skb net/core/dev.c:3852 [inline]
 __dev_queue_xmit+0x1488/0x2ac8 net/core/dev.c:4221
 dev_queue_xmit+0x24/0x34 net/core/dev.c:4289
 neigh_hh_output include/net/neighbour.h:493 [inline]
 neigh_output include/net/neighbour.h:507 [inline]
 ip6_finish_output2+0x132c/0x1cec net/ipv6/ip6_output.c:130
 __ip6_finish_output+0x580/0x6ec net/ipv6/ip6_output.c:201
 ip6_finish_output+0x40/0x218 net/ipv6/ip6_output.c:211
 NF_HOOK_COND include/linux/netfilter.h:291 [inline]
 ip6_output+0x274/0x594 net/ipv6/ip6_output.c:234
 dst_output include/net/dst.h:443 [inline]
 NF_HOOK include/linux/netfilter.h:302 [inline]
 rawv6_send_hdrinc+0xd64/0x1c14 net/ipv6/raw.c:691
 rawv6_sendmsg+0x1074/0x1bcc net/ipv6/raw.c:949
 inet_sendmsg+0x15c/0x290 net/ipv4/af_inet.c:836
 sock_sendmsg_nosec net/socket.c:704 [inline]
 __sock_sendmsg net/socket.c:716 [inline]
 sock_write_iter+0x2b0/0x3f8 net/socket.c:1079
 call_write_iter include/linux/fs.h:2174 [inline]
 new_sync_write fs/read_write.c:507 [inline]
 vfs_write+0x884/0xb44 fs/read_write.c:594
 ksys_write+0x15c/0x26c fs/read_write.c:647
 __do_sys_write fs/read_write.c:659 [inline]
 __se_sys_write fs/read_write.c:656 [inline]
 __arm64_sys_write+0x7c/0x90 fs/read_write.c:656
 __invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
 invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:52
 el0_svc_common+0x138/0x258 arch/arm64/kernel/syscall.c:142
 do_el0_svc+0x58/0x14c arch/arm64/kernel/syscall.c:181
 el0_svc+0x7c/0x1f0 arch/arm64/kernel/entry-common.c:608
 el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:626
 el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584

Crashes (14):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/11/27 16:20 linux-5.15.y 0a51d2d4527b 52b38cc1 .config console log report syz / log C [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in sch_direct_xmit
2024/05/18 16:43 linux-5.15.y 83655231580b c0f1611a .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in sch_direct_xmit
2024/07/14 08:13 linux-5.15.y f45bea23c39c eaeb5c15 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in sch_direct_xmit
2024/03/24 11:42 linux-5.15.y b95c01af2113 0ea90952 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in sch_direct_xmit
2024/10/27 13:40 linux-5.15.y 74cdd62cb470 65e8686b .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in sch_direct_xmit
2024/09/02 04:42 linux-5.15.y fa93fa65db6e 1eda0d14 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in sch_direct_xmit
2024/08/21 15:32 linux-5.15.y fa93fa65db6e db5852f9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in sch_direct_xmit
2024/08/17 06:10 linux-5.15.y 7e89efd3ae1c dbc93b08 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in sch_direct_xmit
2024/08/15 09:04 linux-5.15.y 7e89efd3ae1c e4bacdaf .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in sch_direct_xmit
2024/08/14 19:20 linux-5.15.y 7e89efd3ae1c e6b88e20 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in sch_direct_xmit
2024/05/18 14:37 linux-5.15.y 83655231580b c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in sch_direct_xmit
2024/04/24 13:33 linux-5.15.y c52b9710c83d 21339d7b .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in sch_direct_xmit
2024/03/27 21:07 linux-5.15.y 9465fef4ae35 120789fd .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in sch_direct_xmit
2024/02/22 19:24 linux-5.15.y 6139f2a02fe0 8d446f15 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in sch_direct_xmit
* Struck through repros no longer work on HEAD.