syzbot


possible deadlock in sch_direct_xmit (2)

Status: upstream: reported C repro on 2021/09/26 01:30
Reported-by: syzbot+1d2fdcdd5c3164b28c58@syzkaller.appspotmail.com
First crash: 941d, last: 424d
Fix bisection: failed (error log, bisect log)
  
Similar bugs (11)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-6.1 possible deadlock in sch_direct_xmit (2) C 9 13d 105d 0/3 upstream: reported C repro on 2024/01/09 18:28
android-44 possible deadlock in sch_direct_xmit C 240 1603d 1839d 0/2 public: reported C repro on 2019/04/11 08:44
upstream possible deadlock in sch_direct_xmit (2) net C done unreliable 109 281d 1456d 0/26 auto-obsoleted due to no activity on 2024/01/14 06:05
upstream possible deadlock in sch_direct_xmit net C done done 1548 1610d 2289d 15/26 fixed on 2020/04/17 19:57
linux-5.15 possible deadlock in sch_direct_xmit (2) 3 27d 61d 0/3 upstream: reported on 2024/02/22 19:25
linux-4.14 possible deadlock in sch_direct_xmit 1 1787d 1787d 0/1 auto-closed as invalid on 2019/10/25 08:40
linux-4.14 possible deadlock in sch_direct_xmit (2) 1 1620d 1620d 0/1 auto-closed as invalid on 2020/03/15 19:58
linux-4.19 possible deadlock in sch_direct_xmit 1 1788d 1788d 0/1 auto-closed as invalid on 2019/10/25 08:50
linux-5.15 possible deadlock in sch_direct_xmit 1 348d 348d 0/3 auto-obsoleted due to no activity on 2023/08/23 09:09
linux-6.1 possible deadlock in sch_direct_xmit 2 356d 394d 0/3 auto-obsoleted due to no activity on 2023/08/23 09:10
upstream possible deadlock in sch_direct_xmit (3) net 1 71d 71d 26/26 fixed on 2024/04/10 16:40

Sample crash report:
IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready
wlan1: Created IBSS using preconfigured BSSID 50:50:50:50:50:50
wlan1: Creating new IBSS network, BSSID 50:50:50:50:50:50
IPv6: ADDRCONF(NETDEV_CHANGE): wlan1: link becomes ready
============================================
WARNING: possible recursive locking detected
4.19.211-syzkaller #0 Not tainted
--------------------------------------------
syz-executor970/8091 is trying to acquire lock:
00000000309389d1 (_xmit_ETHER#2){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline]
00000000309389d1 (_xmit_ETHER#2){+.-.}, at: __netif_tx_lock include/linux/netdevice.h:3842 [inline]
00000000309389d1 (_xmit_ETHER#2){+.-.}, at: sch_direct_xmit+0x254/0xf70 net/sched/sch_generic.c:330

but task is already holding lock:
00000000f72df3e7 (_xmit_ETHER#2){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline]
00000000f72df3e7 (_xmit_ETHER#2){+.-.}, at: __netif_tx_lock include/linux/netdevice.h:3842 [inline]
00000000f72df3e7 (_xmit_ETHER#2){+.-.}, at: sch_direct_xmit+0x254/0xf70 net/sched/sch_generic.c:330

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock(_xmit_ETHER#2);
  lock(_xmit_ETHER#2);

 *** DEADLOCK ***

 May be due to missing lock nesting notation

9 locks held by syz-executor970/8091:
 #0: 000000003f116d3c (rcu_read_lock_bh){....}, at: lwtunnel_xmit_redirect include/net/lwtunnel.h:92 [inline]
 #0: 000000003f116d3c (rcu_read_lock_bh){....}, at: ip_finish_output2+0x28d/0x15a0 net/ipv4/ip_output.c:214
 #1: 000000003f116d3c (rcu_read_lock_bh){....}, at: __dev_queue_xmit+0x1e2/0x2e00 net/core/dev.c:3773
 #2: 0000000088eceec2 (&(&sch->seqlock)->rlock){+...}, at: spin_trylock include/linux/spinlock.h:339 [inline]
 #2: 0000000088eceec2 (&(&sch->seqlock)->rlock){+...}, at: qdisc_run_begin include/net/sch_generic.h:130 [inline]
 #2: 0000000088eceec2 (&(&sch->seqlock)->rlock){+...}, at: qdisc_run include/net/pkt_sched.h:119 [inline]
 #2: 0000000088eceec2 (&(&sch->seqlock)->rlock){+...}, at: __dev_xmit_skb net/core/dev.c:3451 [inline]
 #2: 0000000088eceec2 (&(&sch->seqlock)->rlock){+...}, at: __dev_queue_xmit+0x278b/0x2e00 net/core/dev.c:3807
 #3: 00000000ac8de731 (dev->qdisc_running_key ?: &qdisc_running_key){+...}, at: neigh_resolve_output+0x55a/0x910 net/core/neighbour.c:1374
 #4: 00000000f72df3e7 (_xmit_ETHER#2){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline]
 #4: 00000000f72df3e7 (_xmit_ETHER#2){+.-.}, at: __netif_tx_lock include/linux/netdevice.h:3842 [inline]
 #4: 00000000f72df3e7 (_xmit_ETHER#2){+.-.}, at: sch_direct_xmit+0x254/0xf70 net/sched/sch_generic.c:330
 #5: 000000003f116d3c (rcu_read_lock_bh){....}, at: lwtunnel_xmit_redirect include/net/lwtunnel.h:92 [inline]
 #5: 000000003f116d3c (rcu_read_lock_bh){....}, at: ip_finish_output2+0x28d/0x15a0 net/ipv4/ip_output.c:214
 #6: 000000003f116d3c (rcu_read_lock_bh){....}, at: __dev_queue_xmit+0x1e2/0x2e00 net/core/dev.c:3773
 #7: 00000000b02d8be2 (&(&sch->seqlock)->rlock){+...}, at: spin_trylock include/linux/spinlock.h:339 [inline]
 #7: 00000000b02d8be2 (&(&sch->seqlock)->rlock){+...}, at: qdisc_run_begin include/net/sch_generic.h:130 [inline]
 #7: 00000000b02d8be2 (&(&sch->seqlock)->rlock){+...}, at: qdisc_run include/net/pkt_sched.h:119 [inline]
 #7: 00000000b02d8be2 (&(&sch->seqlock)->rlock){+...}, at: __dev_xmit_skb net/core/dev.c:3451 [inline]
 #7: 00000000b02d8be2 (&(&sch->seqlock)->rlock){+...}, at: __dev_queue_xmit+0x278b/0x2e00 net/core/dev.c:3807
 #8: 000000009ca40145 (dev->qdisc_running_key ?: &qdisc_running_key){+...}, at: neigh_resolve_output+0x55a/0x910 net/core/neighbour.c:1374

stack backtrace:
CPU: 1 PID: 8091 Comm: syz-executor970 Not tainted 4.19.211-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x1fc/0x2ef lib/dump_stack.c:118
 print_deadlock_bug kernel/locking/lockdep.c:1764 [inline]
 check_deadlock kernel/locking/lockdep.c:1808 [inline]
 validate_chain kernel/locking/lockdep.c:2404 [inline]
 __lock_acquire.cold+0x121/0x57e kernel/locking/lockdep.c:3416
 lock_acquire+0x170/0x3c0 kernel/locking/lockdep.c:3908
 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
 _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
 spin_lock include/linux/spinlock.h:329 [inline]
 __netif_tx_lock include/linux/netdevice.h:3842 [inline]
 sch_direct_xmit+0x254/0xf70 net/sched/sch_generic.c:330
 qdisc_restart net/sched/sch_generic.c:395 [inline]
 __qdisc_run+0x4d0/0x1640 net/sched/sch_generic.c:403
 qdisc_run include/net/pkt_sched.h:120 [inline]
 __dev_xmit_skb net/core/dev.c:3451 [inline]
 __dev_queue_xmit+0x2102/0x2e00 net/core/dev.c:3807
 neigh_resolve_output+0x55a/0x910 net/core/neighbour.c:1374
 neigh_output include/net/neighbour.h:501 [inline]
 ip_finish_output2+0xd76/0x15a0 net/ipv4/ip_output.c:230
 ip_finish_output+0xae9/0x10b0 net/ipv4/ip_output.c:318
 NF_HOOK_COND include/linux/netfilter.h:278 [inline]
 ip_output+0x203/0x5f0 net/ipv4/ip_output.c:406
 dst_output include/net/dst.h:455 [inline]
 ip_local_out+0xaf/0x170 net/ipv4/ip_output.c:125
 iptunnel_xmit+0x63b/0x9d0 net/ipv4/ip_tunnel_core.c:91
 ip_tunnel_xmit+0x13ad/0x3850 net/ipv4/ip_tunnel.c:790
 erspan_xmit+0xd6e/0x27e0 net/ipv4/ip_gre.c:759
 __netdev_start_xmit include/linux/netdevice.h:4349 [inline]
 netdev_start_xmit include/linux/netdevice.h:4363 [inline]
 xmit_one net/core/dev.c:3256 [inline]
 dev_hard_start_xmit+0x1a8/0x920 net/core/dev.c:3272
 sch_direct_xmit+0x2d6/0xf70 net/sched/sch_generic.c:332
 qdisc_restart net/sched/sch_generic.c:395 [inline]
 __qdisc_run+0x4d0/0x1640 net/sched/sch_generic.c:403
 qdisc_run include/net/pkt_sched.h:120 [inline]
 __dev_xmit_skb net/core/dev.c:3451 [inline]
 __dev_queue_xmit+0x2102/0x2e00 net/core/dev.c:3807
 neigh_resolve_output+0x55a/0x910 net/core/neighbour.c:1374
 neigh_output include/net/neighbour.h:501 [inline]
 ip_finish_output2+0xd76/0x15a0 net/ipv4/ip_output.c:230
 ip_finish_output+0xae9/0x10b0 net/ipv4/ip_output.c:318
 NF_HOOK_COND include/linux/netfilter.h:278 [inline]
 ip_output+0x203/0x5f0 net/ipv4/ip_output.c:406
 dst_output include/net/dst.h:455 [inline]
 ip_local_out+0xaf/0x170 net/ipv4/ip_output.c:125
 ip_send_skb+0x3e/0xe0 net/ipv4/ip_output.c:1452
 udp_send_skb+0x6a4/0x1170 net/ipv4/udp.c:848
 udp_sendmsg+0x1cb4/0x2550 net/ipv4/udp.c:1135
 udpv6_sendmsg+0x14b2/0x2ae0 net/ipv6/udp.c:1224
 inet_sendmsg+0x132/0x5a0 net/ipv4/af_inet.c:798
 sock_sendmsg_nosec net/socket.c:651 [inline]
 sock_sendmsg+0xc3/0x120 net/socket.c:661
 ___sys_sendmsg+0x3b3/0x8e0 net/socket.c:2227
 __sys_sendmmsg+0x195/0x470 net/socket.c:2322
 __do_sys_sendmmsg net/socket.c:2351 [inline]
 __se_sys_sendmmsg net/socket.c:2348 [inline]
 __x64_sys_sendmmsg+0x99/0x100 net/socket.c:2348
 do_syscall_64+0xf9/0x620 arch/x86/entry/common.c:293
 entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x7fd94fe4eaa9
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 41 15 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 c0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fff36cb4548 EFLAGS: 00000246 ORIG_RAX: 0000000000000133
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fd94fe4eaa9
RDX: 0000000000000001 RSI: 0000000020004d80 RDI: 0000

Crashes (15):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2021/10/28 23:48 linux-4.19.y 3f8a27f9e27b be531bb4 .config console log report syz C ci2-linux-4-19 possible deadlock in sch_direct_xmit
2023/02/25 01:44 linux-4.19.y 3f8a27f9e27b ee50e71c .config console log report info [disk image] [vmlinux] ci2-linux-4-19 possible deadlock in sch_direct_xmit
2023/02/16 02:44 linux-4.19.y 3f8a27f9e27b 6be0f1f5 .config console log report info [disk image] [vmlinux] ci2-linux-4-19 possible deadlock in sch_direct_xmit
2023/02/04 14:41 linux-4.19.y 3f8a27f9e27b be607b78 .config console log report info [disk image] [vmlinux] ci2-linux-4-19 possible deadlock in sch_direct_xmit
2023/01/12 22:49 linux-4.19.y 3f8a27f9e27b 96166539 .config console log report info [disk image] [vmlinux] ci2-linux-4-19 possible deadlock in sch_direct_xmit
2022/10/08 05:31 linux-4.19.y 3f8a27f9e27b aea5da89 .config console log report info [disk image] [vmlinux] ci2-linux-4-19 possible deadlock in sch_direct_xmit
2022/10/01 17:46 linux-4.19.y 3f8a27f9e27b feb56351 .config console log report info [disk image] [vmlinux] ci2-linux-4-19 possible deadlock in sch_direct_xmit
2022/09/28 00:31 linux-4.19.y 3f8a27f9e27b 75c78242 .config console log report info [disk image] [vmlinux] ci2-linux-4-19 possible deadlock in sch_direct_xmit
2022/09/17 04:19 linux-4.19.y 3f8a27f9e27b dd9a85ff .config console log report info [disk image] [vmlinux] ci2-linux-4-19 possible deadlock in sch_direct_xmit
2022/01/09 08:42 linux-4.19.y 3f8a27f9e27b 2ca0d385 .config console log report info ci2-linux-4-19 possible deadlock in sch_direct_xmit
2021/12/21 17:19 linux-4.19.y 3f8a27f9e27b a938f0b8 .config console log report info ci2-linux-4-19 possible deadlock in sch_direct_xmit
2021/12/13 10:37 linux-4.19.y 3f8a27f9e27b 49ca1f59 .config console log report info ci2-linux-4-19 possible deadlock in sch_direct_xmit
2021/10/28 22:09 linux-4.19.y 3f8a27f9e27b be531bb4 .config console log report info ci2-linux-4-19 possible deadlock in sch_direct_xmit
2021/10/28 22:09 linux-4.19.y 3f8a27f9e27b be531bb4 .config console log report info ci2-linux-4-19 possible deadlock in sch_direct_xmit
2021/09/26 01:29 linux-4.19.y 2950c9c5e0df 8cac236e .config console log report info ci2-linux-4-19 possible deadlock in sch_direct_xmit
* Struck through repros no longer work on HEAD.