syzbot


possible deadlock in sch_direct_xmit (2)

Status: upstream: reported C repro on 2024/02/22 19:25
Bug presence: origin:lts-only
[Documentation on labels]
Reported-by: syzbot+3f44bf8b6f083aa47b0a@syzkaller.appspotmail.com
First crash: 114d, last: 28d
Bug presence (2)
Date Name Commit Repro Result
2024/05/18 linux-5.15.y (ToT) 83655231580b C [report] possible deadlock in sch_direct_xmit
2024/05/18 upstream (ToT) 4b377b4868ef C Didn't crash
Similar bugs (12)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-6.1 possible deadlock in sch_direct_xmit (2) origin:lts-only C done 11 34d 158d 0/3 upstream: reported C repro on 2024/01/09 18:28
android-44 possible deadlock in sch_direct_xmit C 240 1656d 1892d 0/2 public: reported C repro on 2019/04/11 08:44
upstream possible deadlock in sch_direct_xmit (2) net C done unreliable 109 334d 1509d 0/27 auto-obsoleted due to no activity on 2024/01/14 06:05
linux-4.19 possible deadlock in sch_direct_xmit (2) C error 15 477d 994d 0/1 upstream: reported C repro on 2021/09/26 01:30
upstream possible deadlock in sch_direct_xmit net C done done 1548 1663d 2342d 15/27 fixed on 2020/04/17 19:57
linux-4.14 possible deadlock in sch_direct_xmit 1 1840d 1840d 0/1 auto-closed as invalid on 2019/10/25 08:40
upstream possible deadlock in sch_direct_xmit (4) net 1 49d 49d 26/27 fixed on 2024/06/05 13:52
linux-4.14 possible deadlock in sch_direct_xmit (2) 1 1673d 1673d 0/1 auto-closed as invalid on 2020/03/15 19:58
linux-4.19 possible deadlock in sch_direct_xmit 1 1841d 1841d 0/1 auto-closed as invalid on 2019/10/25 08:50
linux-5.15 possible deadlock in sch_direct_xmit 1 401d 401d 0/3 auto-obsoleted due to no activity on 2023/08/23 09:09
linux-6.1 possible deadlock in sch_direct_xmit 2 409d 447d 0/3 auto-obsoleted due to no activity on 2023/08/23 09:10
upstream possible deadlock in sch_direct_xmit (3) net 1 124d 124d 26/27 fixed on 2024/04/10 16:40
Fix bisection attempts (1)
Created Duration User Patch Repo Result
2024/05/25 13:09 1m fix candidate upstream error job log (0)

Sample crash report:
============================================
WARNING: possible recursive locking detected
5.15.159-syzkaller #0 Not tainted
--------------------------------------------
syz-executor800/4872 is trying to acquire lock:
ffff0000d9b3e398 (_xmit_ETHER#2){+.-.}-{2:2}, at: spin_lock include/linux/spinlock.h:363 [inline]
ffff0000d9b3e398 (_xmit_ETHER#2){+.-.}-{2:2}, at: __netif_tx_lock include/linux/netdevice.h:4429 [inline]
ffff0000d9b3e398 (_xmit_ETHER#2){+.-.}-{2:2}, at: sch_direct_xmit+0x15c/0x484 net/sched/sch_generic.c:340

but task is already holding lock:
ffff0000d9a75c98 (_xmit_ETHER#2){+.-.}-{2:2}, at: spin_lock include/linux/spinlock.h:363 [inline]
ffff0000d9a75c98 (_xmit_ETHER#2){+.-.}-{2:2}, at: __netif_tx_lock include/linux/netdevice.h:4429 [inline]
ffff0000d9a75c98 (_xmit_ETHER#2){+.-.}-{2:2}, at: sch_direct_xmit+0x15c/0x484 net/sched/sch_generic.c:340

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock(_xmit_ETHER#2);
  lock(_xmit_ETHER#2);

 *** DEADLOCK ***

 May be due to missing lock nesting notation

12 locks held by syz-executor800/4872:
 #0: ffff800014b214a0 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x10/0x4c include/linux/rcupdate.h:311
 #1: ffff800014b21500 (rcu_read_lock_bh){....}-{1:2}, at: rcu_lock_acquire+0x18/0x54 include/linux/rcupdate.h:311
 #2: ffff800014b21500 (rcu_read_lock_bh){....}-{1:2}, at: rcu_lock_acquire+0x18/0x54 include/linux/rcupdate.h:311
 #3: ffff0000c8e6b258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: spin_trylock include/linux/spinlock.h:373 [inline]
 #3: ffff0000c8e6b258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: qdisc_run_begin+0x130/0x2bc include/net/sch_generic.h:173
 #4: ffff0000d9a75c98 (_xmit_ETHER#2){+.-.}-{2:2}, at: spin_lock include/linux/spinlock.h:363 [inline]
 #4: ffff0000d9a75c98 (_xmit_ETHER#2){+.-.}-{2:2}, at: __netif_tx_lock include/linux/netdevice.h:4429 [inline]
 #4: ffff0000d9a75c98 (_xmit_ETHER#2){+.-.}-{2:2}, at: sch_direct_xmit+0x15c/0x484 net/sched/sch_generic.c:340
 #5: ffff0000d2961660 (k-slock-AF_INET6){+...}-{2:2}, at: spin_trylock include/linux/spinlock.h:373 [inline]
 #5: ffff0000d2961660 (k-slock-AF_INET6){+...}-{2:2}, at: icmpv6_xmit_lock+0x100/0x188 net/ipv6/icmp.c:118
 #6: ffff800014b214a0 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x10/0x4c include/linux/rcupdate.h:311
 #7: ffff800014b21500 (rcu_read_lock_bh){....}-{1:2}, at: rcu_lock_acquire+0x18/0x54 include/linux/rcupdate.h:311
 #8: ffff800014b214a0 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x10/0x4c include/linux/rcupdate.h:311
 #9: ffff800014b21500 (rcu_read_lock_bh){....}-{1:2}, at: rcu_lock_acquire+0x18/0x54 include/linux/rcupdate.h:311
 #10: ffff800014b21500 (rcu_read_lock_bh){....}-{1:2}, at: rcu_lock_acquire+0x18/0x54 include/linux/rcupdate.h:311
 #11: ffff0000da36e258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: spin_trylock include/linux/spinlock.h:373 [inline]
 #11: ffff0000da36e258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: qdisc_run_begin+0x130/0x2bc include/net/sch_generic.h:173

stack backtrace:
CPU: 0 PID: 4872 Comm: syz-executor800 Not tainted 5.15.159-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/27/2024
Call trace:
 dump_backtrace+0x0/0x530 arch/arm64/kernel/stacktrace.c:152
 show_stack+0x2c/0x3c arch/arm64/kernel/stacktrace.c:216
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x108/0x170 lib/dump_stack.c:106
 dump_stack+0x1c/0x58 lib/dump_stack.c:113
 __lock_acquire+0x62bc/0x7638 kernel/locking/lockdep.c:5012
 lock_acquire+0x240/0x77c kernel/locking/lockdep.c:5623
 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
 _raw_spin_lock+0xb0/0x10c kernel/locking/spinlock.c:154
 spin_lock include/linux/spinlock.h:363 [inline]
 __netif_tx_lock include/linux/netdevice.h:4429 [inline]
 sch_direct_xmit+0x15c/0x484 net/sched/sch_generic.c:340
 __dev_xmit_skb net/core/dev.c:3844 [inline]
 __dev_queue_xmit+0x14b4/0x2a6c net/core/dev.c:4213
 dev_queue_xmit+0x24/0x34 net/core/dev.c:4281
 neigh_hh_output include/net/neighbour.h:493 [inline]
 neigh_output include/net/neighbour.h:507 [inline]
 ip6_finish_output2+0x1314/0x1c4c net/ipv6/ip6_output.c:126
 __ip6_finish_output+0x580/0x6ec net/ipv6/ip6_output.c:197
 ip6_finish_output+0x40/0x218 net/ipv6/ip6_output.c:207
 NF_HOOK_COND include/linux/netfilter.h:291 [inline]
 ip6_output+0x270/0x594 net/ipv6/ip6_output.c:230
 dst_output include/net/dst.h:443 [inline]
 NF_HOOK include/linux/netfilter.h:302 [inline]
 ndisc_send_skb+0xbf8/0x1788 net/ipv6/ndisc.c:509
 ndisc_send_ns+0x538/0x6ec net/ipv6/ndisc.c:651
 ndisc_solicit+0x2f4/0x47c
 neigh_probe+0xc4/0x138 net/core/neighbour.c:1017
 __neigh_event_send+0xca4/0x1338 net/core/neighbour.c:1178
 neigh_event_send include/net/neighbour.h:438 [inline]
 neigh_resolve_output+0x178/0x5dc net/core/neighbour.c:1488
 neigh_output include/net/neighbour.h:509 [inline]
 ip6_finish_output2+0x1348/0x1c4c net/ipv6/ip6_output.c:126
 __ip6_finish_output+0x580/0x6ec net/ipv6/ip6_output.c:197
 ip6_finish_output+0x40/0x218 net/ipv6/ip6_output.c:207
 NF_HOOK_COND include/linux/netfilter.h:291 [inline]
 ip6_output+0x270/0x594 net/ipv6/ip6_output.c:230
 dst_output include/net/dst.h:443 [inline]
 ip6_local_out+0x120/0x160 net/ipv6/output_core.c:161
 ip6_send_skb+0x118/0x428 net/ipv6/ip6_output.c:1943
 ip6_push_pending_frames+0xd0/0x118 net/ipv6/ip6_output.c:1963
 icmpv6_push_pending_frames+0x244/0x398 net/ipv6/icmp.c:311
 icmp6_send+0x11a4/0x1b18 net/ipv6/icmp.c:630
 __icmpv6_send include/linux/icmpv6.h:28 [inline]
 icmpv6_send include/linux/icmpv6.h:49 [inline]
 ip6_link_failure+0x44/0x4a8 net/ipv6/route.c:2790
 dst_link_failure+0x11c/0x160 include/net/dst.h:422
 ip_tunnel_xmit+0x16e0/0x2334 net/ipv4/ip_tunnel.c:843
 __gre_xmit net/ipv4/ip_gre.c:474 [inline]
 erspan_xmit+0x9cc/0x14cc net/ipv4/ip_gre.c:723
 __netdev_start_xmit include/linux/netdevice.h:5019 [inline]
 netdev_start_xmit include/linux/netdevice.h:5033 [inline]
 xmit_one net/core/dev.c:3617 [inline]
 dev_hard_start_xmit+0x2bc/0x92c net/core/dev.c:3633
 sch_direct_xmit+0x2e0/0x484 net/sched/sch_generic.c:342
 __dev_xmit_skb net/core/dev.c:3844 [inline]
 __dev_queue_xmit+0x14b4/0x2a6c net/core/dev.c:4213
 dev_queue_xmit+0x24/0x34 net/core/dev.c:4281
 neigh_hh_output include/net/neighbour.h:493 [inline]
 neigh_output include/net/neighbour.h:507 [inline]
 ip6_finish_output2+0x1314/0x1c4c net/ipv6/ip6_output.c:126
 __ip6_finish_output+0x580/0x6ec net/ipv6/ip6_output.c:197
 ip6_finish_output+0x40/0x218 net/ipv6/ip6_output.c:207
 NF_HOOK_COND include/linux/netfilter.h:291 [inline]
 ip6_output+0x270/0x594 net/ipv6/ip6_output.c:230
 dst_output include/net/dst.h:443 [inline]
 NF_HOOK include/linux/netfilter.h:302 [inline]
 rawv6_send_hdrinc+0xd64/0x1c14 net/ipv6/raw.c:691
 rawv6_sendmsg+0x1074/0x1bcc net/ipv6/raw.c:949
 inet_sendmsg+0x15c/0x290 net/ipv4/af_inet.c:828
 sock_sendmsg_nosec net/socket.c:704 [inline]
 __sock_sendmsg net/socket.c:716 [inline]
 ____sys_sendmsg+0x584/0x870 net/socket.c:2431
 ___sys_sendmsg+0x214/0x294 net/socket.c:2485
 __sys_sendmmsg+0x23c/0x648 net/socket.c:2571
 __do_sys_sendmmsg net/socket.c:2600 [inline]
 __se_sys_sendmmsg net/socket.c:2597 [inline]
 __arm64_sys_sendmmsg+0xa0/0xbc net/socket.c:2597
 __invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
 invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:52
 el0_svc_common+0x138/0x258 arch/arm64/kernel/syscall.c:142
 do_el0_svc+0x58/0x14c arch/arm64/kernel/syscall.c:181
 el0_svc+0x7c/0x1f0 arch/arm64/kernel/entry-common.c:608
 el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:626
 el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584

Crashes (6):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/05/18 16:43 linux-5.15.y 83655231580b c0f1611a .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in sch_direct_xmit
2024/03/24 11:42 linux-5.15.y b95c01af2113 0ea90952 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in sch_direct_xmit
2024/05/18 14:37 linux-5.15.y 83655231580b c0f1611a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in sch_direct_xmit
2024/04/24 13:33 linux-5.15.y c52b9710c83d 21339d7b .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in sch_direct_xmit
2024/03/27 21:07 linux-5.15.y 9465fef4ae35 120789fd .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in sch_direct_xmit
2024/02/22 19:24 linux-5.15.y 6139f2a02fe0 8d446f15 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 possible deadlock in sch_direct_xmit
* Struck through repros no longer work on HEAD.