syzbot


possible deadlock in sch_direct_xmit (2)
Status: upstream: reported C repro on 2020/04/29 00:59
Reported-by: syzbot+e18ac85757292b7baf96@syzkaller.appspotmail.com
First crash: 761d, last: 14d

Cause bisection: failed (bisect log)

Fix bisection: fixed by (bisect log) [merge commit]:
commit 616355cc818c6ddadc393fdfd4491f94458cb715
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date: Mon Mar 21 23:48:55 2022 +0000

  Merge tag 'for-5.18/block-2022-03-18' of git://git.kernel.dk/linux-block

similar bugs (6):
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
android-44 possible deadlock in sch_direct_xmit C 240 904d 1141d 0/2 public: reported C repro on 2019/04/11 08:44
linux-4.19 possible deadlock in sch_direct_xmit (2) C error 7 137d 242d 0/1 upstream: reported C repro on 2021/09/26 01:30
upstream possible deadlock in sch_direct_xmit C done done 1548 912d 1590d 17/22 fixed on 2020/04/17 19:57
linux-4.14 possible deadlock in sch_direct_xmit 1 1088d 1088d 0/1 auto-closed as invalid on 2019/10/25 08:40
linux-4.14 possible deadlock in sch_direct_xmit (2) 1 921d 921d 0/1 auto-closed as invalid on 2020/03/15 19:58
linux-4.19 possible deadlock in sch_direct_xmit 1 1089d 1089d 0/1 auto-closed as invalid on 2019/10/25 08:50

Sample crash report:
============================================
WARNING: possible recursive locking detected
5.18.0-rc6-syzkaller-00009-gfeb9c5e19e91 #0 Not tainted
--------------------------------------------
syz-executor179/3597 is trying to acquire lock:
ffff88807b96f458 (_xmit_ETHER#2){+.-.}-{2:2}, at: spin_lock include/linux/spinlock.h:349 [inline]
ffff88807b96f458 (_xmit_ETHER#2){+.-.}-{2:2}, at: __netif_tx_lock include/linux/netdevice.h:4223 [inline]
ffff88807b96f458 (_xmit_ETHER#2){+.-.}-{2:2}, at: sch_direct_xmit+0x1bd/0x5e0 net/sched/sch_generic.c:340

but task is already holding lock:
ffff88813fe0a4d8 (_xmit_ETHER#2){+.-.}-{2:2}, at: spin_lock include/linux/spinlock.h:349 [inline]
ffff88813fe0a4d8 (_xmit_ETHER#2){+.-.}-{2:2}, at: __netif_tx_lock include/linux/netdevice.h:4223 [inline]
ffff88813fe0a4d8 (_xmit_ETHER#2){+.-.}-{2:2}, at: sch_direct_xmit+0x1bd/0x5e0 net/sched/sch_generic.c:340

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock(_xmit_ETHER#2);
  lock(_xmit_ETHER#2);

 *** DEADLOCK ***

 May be due to missing lock nesting notation

7 locks held by syz-executor179/3597:
 #0: ffffffff8cb1b080 (rcu_read_lock_bh){....}-{1:2}, at: rcu_lock_acquire+0x9/0x30 include/linux/rcupdate.h:268
 #1: ffffffff8cb1b080 (rcu_read_lock_bh){....}-{1:2}, at: rcu_lock_acquire+0x9/0x30 include/linux/rcupdate.h:268
 #2: ffff8880253ac258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: spin_trylock include/linux/spinlock.h:359 [inline]
 #2: ffff8880253ac258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: qdisc_run_begin include/net/sch_generic.h:187 [inline]
 #2: ffff8880253ac258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: __dev_xmit_skb net/core/dev.c:3746 [inline]
 #2: ffff8880253ac258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: __dev_queue_xmit+0x12ce/0x3640 net/core/dev.c:4141
 #3: ffff88813fe0a4d8 (_xmit_ETHER#2){+.-.}-{2:2}, at: spin_lock include/linux/spinlock.h:349 [inline]
 #3: ffff88813fe0a4d8 (_xmit_ETHER#2){+.-.}-{2:2}, at: __netif_tx_lock include/linux/netdevice.h:4223 [inline]
 #3: ffff88813fe0a4d8 (_xmit_ETHER#2){+.-.}-{2:2}, at: sch_direct_xmit+0x1bd/0x5e0 net/sched/sch_generic.c:340
 #4: ffffffff8cb1b080 (rcu_read_lock_bh){....}-{1:2}, at: rcu_lock_acquire+0x9/0x30 include/linux/rcupdate.h:268
 #5: ffffffff8cb1b080 (rcu_read_lock_bh){....}-{1:2}, at: rcu_lock_acquire+0x9/0x30 include/linux/rcupdate.h:268
 #6: ffff88807d682258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: spin_trylock include/linux/spinlock.h:359 [inline]
 #6: ffff88807d682258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: qdisc_run_begin include/net/sch_generic.h:187 [inline]
 #6: ffff88807d682258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: __dev_xmit_skb net/core/dev.c:3746 [inline]
 #6: ffff88807d682258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: __dev_queue_xmit+0x12ce/0x3640 net/core/dev.c:4141

stack backtrace:
CPU: 1 PID: 3597 Comm: syz-executor179 Not tainted 5.18.0-rc6-syzkaller-00009-gfeb9c5e19e91 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e3/0x2cb lib/dump_stack.c:106
 print_deadlock_bug kernel/locking/lockdep.c:2958 [inline]
 check_deadlock kernel/locking/lockdep.c:3001 [inline]
 validate_chain+0x485d/0x65c0 kernel/locking/lockdep.c:3790
 __lock_acquire+0x129a/0x1f80 kernel/locking/lockdep.c:5029
 lock_acquire+0x1a7/0x400 kernel/locking/lockdep.c:5641
 __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
 _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:154
 spin_lock include/linux/spinlock.h:349 [inline]
 __netif_tx_lock include/linux/netdevice.h:4223 [inline]
 sch_direct_xmit+0x1bd/0x5e0 net/sched/sch_generic.c:340
 __dev_xmit_skb net/core/dev.c:3759 [inline]
 __dev_queue_xmit+0x1970/0x3640 net/core/dev.c:4141
 neigh_output include/net/neighbour.h:549 [inline]
 ip_finish_output2+0xdc8/0x11c0 net/ipv4/ip_output.c:228
 iptunnel_xmit+0x4cb/0x8d0 net/ipv4/ip_tunnel_core.c:82
 ip_tunnel_xmit+0x1c3c/0x24e0 net/ipv4/ip_tunnel.c:811
 __gre_xmit net/ipv4/ip_gre.c:469 [inline]
 erspan_xmit+0xbf6/0x18b0 net/ipv4/ip_gre.c:711
 __netdev_start_xmit include/linux/netdevice.h:4775 [inline]
 netdev_start_xmit include/linux/netdevice.h:4789 [inline]
 xmit_one net/core/dev.c:3532 [inline]
 dev_hard_start_xmit+0x20b/0x450 net/core/dev.c:3548
 sch_direct_xmit+0x2ae/0x5e0 net/sched/sch_generic.c:342
 __dev_xmit_skb net/core/dev.c:3759 [inline]
 __dev_queue_xmit+0x1970/0x3640 net/core/dev.c:4141
 neigh_output include/net/neighbour.h:549 [inline]
 ip_finish_output2+0xdc8/0x11c0 net/ipv4/ip_output.c:228
 dst_output include/net/dst.h:451 [inline]
 ip_local_out net/ipv4/ip_output.c:126 [inline]
 ip_send_skb+0x113/0x1a0 net/ipv4/ip_output.c:1571
 udp_send_skb+0x8f7/0x11e0 net/ipv4/udp.c:967
 udp_sendmsg+0x1d56/0x2b50 net/ipv4/udp.c:1254
 udpv6_sendmsg+0xc4a/0x3180
 sock_sendmsg_nosec net/socket.c:705 [inline]
 sock_sendmsg net/socket.c:725 [inline]
 ____sys_sendmsg+0x597/0x8e0 net/socket.c:2413
 ___sys_sendmsg net/socket.c:2467 [inline]
 __sys_sendmmsg+0x3d7/0x770 net/socket.c:2553
 __do_sys_sendmmsg net/socket.c:2582 [inline]
 __se_sys_sendmmsg net/socket.c:2579 [inline]
 __x64_sys_sendmmsg+0x9c/0xb0 net/socket.c:2579
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x2b/0x70 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7fe7b8cc7b29
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 41 15 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 c0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fffa336f958 EFLAGS: 00000246 ORIG_RAX: 0000000000000133
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe7b8cc7b29
RDX: 0000000000000001 RSI: 0000000020004d80 RDI: 0000000000000004
RBP: 0000000000000003 R08: 00007fffa336f987 R09: 00007fffa336f987
R10: 0000000004000000 R11: 0000000000000246 R12: 0000000000000031
R13: 00007fffa336f9e0 R14: 00007fe7b8d434a0 R15: 00007fffa336f9b2
 </TASK>

Crashes (90):
Manager Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Title
ci-upstream-kasan-gce-smack-root 2022/05/11 21:38 upstream feb9c5e19e91 beb0b407 .config log report syz C possible deadlock in sch_direct_xmit
ci-upstream-kasan-gce-root 2022/04/03 12:57 upstream be2d3ecedd99 79a2a8fc .config log report syz C possible deadlock in sch_direct_xmit
ci-upstream-kasan-gce 2021/12/09 12:55 upstream 2a987e65025e a4a2a501 .config log report syz C possible deadlock in sch_direct_xmit
ci-upstream-net-this-kasan-gce 2021/10/29 02:07 net 35392da51b1a be531bb4 .config log report syz C possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2021/10/29 02:14 net-next f2edaa4ad5d5 be531bb4 .config log report syz C possible deadlock in sch_direct_xmit
ci-upstream-linux-next-kasan-gce-root 2022/05/10 00:20 linux-next 38a288f5941e 8b277b8e .config log report syz C possible deadlock in sch_direct_xmit
ci-upstream-kasan-gce-root 2021/12/25 12:00 upstream b927dfc67d05 6caa12e4 .config log report info possible deadlock in sch_direct_xmit
ci-qemu-upstream 2021/12/18 05:51 upstream 9eaa88c7036e 44068e19 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-kasan-gce-smack-root 2021/11/07 02:17 upstream 512b7931ad05 4c1be0be .config log report info possible deadlock in sch_direct_xmit
ci-upstream-kasan-gce-smack-root 2021/08/27 23:37 upstream 77dd11439b86 d5a29e53 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-kasan-gce-smack-root 2021/07/27 22:05 upstream ff1176468d36 17d6ab15 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-kasan-gce-386 2021/09/07 13:47 upstream 4b93c544e90e 6ca60148 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-this-kasan-gce 2022/05/03 22:17 net 3a58f13a881e dc9e5259 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-this-kasan-gce 2022/05/01 23:11 net 79396934e289 2df221f6 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-this-kasan-gce 2022/02/16 05:01 net 8d2b1a1ec9f5 8b9ca619 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-this-kasan-gce 2022/01/01 18:10 net d6d86830705f e1768e9c .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-this-kasan-gce 2021/12/10 16:20 net 92816e262980 49ca1f59 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-this-kasan-gce 2021/12/08 09:28 net 2a62df369271 0230ba3e .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-this-kasan-gce 2021/11/10 07:56 net fceb07950a7a 55fa030c .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-this-kasan-gce 2021/11/08 07:11 net 08fcdfa6e3ae 4c1be0be .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-this-kasan-gce 2021/10/28 16:19 net 72f898ca0ab8 be531bb4 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-this-kasan-gce 2021/09/23 08:51 net 977d293e23b4 8cac236e .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-this-kasan-gce 2021/09/15 00:55 net 550ac9c1aaaa 07e953c1 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-this-kasan-gce 2021/09/02 13:35 net 57f780f1c433 15cea0a3 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-this-kasan-gce 2021/08/23 22:23 net 82a44ae113b7 b599f2fc .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-this-kasan-gce 2021/07/31 23:07 net c7d102232649 6c236867 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-this-kasan-gce 2021/06/28 04:35 net d6765985a42a 9d2ab5df .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-this-kasan-gce 2021/06/25 15:01 net be7f62eebaff ae6bf8dd .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2022/05/08 12:03 net-next eb60020411fc e60b1103 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2022/02/24 12:43 net-next 6a47cdc38143 6e821dbf .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2022/01/08 06:56 net-next 82192cb497f9 2ca0d385 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2021/12/09 20:37 net-next 9d922f5df538 b54aa474 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2021/12/02 10:38 net-next 10184da91666 61f86278 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2021/11/28 15:42 net-next d40ce48cb3a6 63eeac02 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2021/11/21 01:28 net-next 520fbdf7fb19 4eb20a4e .config log report info possible deadlock in sch_direct_xmit
ci-upstream-bpf-next-kasan-gce 2021/11/20 10:19 bpf-next 7615209f42a1 3a9d0024 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-bpf-next-kasan-gce 2021/11/17 15:21 bpf-next db813d7bd919 cafff8b6 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2021/10/26 18:57 net-next 36d935a0a67e d50eb50a .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2021/10/24 10:56 net-next 0998aee279c3 282f03fb .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2021/10/19 20:06 net-next cb3dc8901ba4 466b7db1 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2021/10/10 01:53 net-next 7932d53162dc 838e7e2c .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2021/09/27 05:33 net-next d59bdda85eb7 78494d16 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2021/09/26 19:46 net-next d59bdda85eb7 8cac236e .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2021/09/15 21:12 net-next 5706383b30cf 07e953c1 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2021/09/15 13:04 net-next 339133f6c318 07e953c1 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2021/09/07 06:44 net-next 27151f177827 6ca60148 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2021/09/06 23:03 net-next 27151f177827 6ca60148 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2021/08/10 16:13 net-next 4ef3960ea19c 6972b106 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2021/08/08 10:53 net-next f9be84db09d2 6972b106 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2021/07/14 20:03 net-next 5e437416ff66 94e0b707 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2021/05/28 08:51 net-next af9207adb6d9 858ea628 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2021/03/28 20:40 net-next a1281601f88e a8529b82 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-net-kasan-gce 2021/01/23 16:26 net-next 59a49d9617e2 52e37319 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-linux-next-kasan-gce-root 2022/02/17 13:01 linux-next ef6b35306dd8 2bea8a27 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-linux-next-kasan-gce-root 2021/11/26 19:05 linux-next f81e94e91878 63eeac02 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-linux-next-kasan-gce-root 2021/07/28 06:21 linux-next 42d0b5f52c9b 17d6ab15 .config log report info possible deadlock in sch_direct_xmit
ci-upstream-kasan-gce 2020/10/02 17:38 upstream 472e5b056f00 4969d6ca .config log report info
ci-upstream-net-this-kasan-gce 2020/09/27 14:33 net 059432495e20 5dd8aee8 .config log report info
ci-upstream-net-this-kasan-gce 2020/08/12 18:37 net 633f5b6bca9b bc15f7db .config log report
ci-upstream-net-kasan-gce 2020/12/26 05:45 net-next 3db1a3fa9880 821e0b09 .config log report info
ci-upstream-net-kasan-gce 2020/11/28 16:06 net-next 5c39f26e67c9 76b4dcc7 .config log report info
ci-upstream-bpf-next-kasan-gce 2020/07/07 16:17 bpf-next cb8e59cc8720 51095195 .config log report
ci-upstream-bpf-next-kasan-gce 2020/05/25 07:43 bpf-next a152b85984a0 11284182 .config log report
ci-upstream-bpf-next-kasan-gce 2020/05/23 10:13 bpf-next a5dfaa2ab940 9682898d .config log report
ci-upstream-bpf-next-kasan-gce 2020/05/22 12:37 bpf-next 29ae90d22175 5afa2ddd .config log report
ci-upstream-bpf-next-kasan-gce 2020/05/18 05:16 bpf-next 96586dd9268d 37bccd4e .config log report
ci-upstream-bpf-next-kasan-gce 2020/04/26 17:15 bpf-next 3f2eaebb5450 99b258dd .config log report
ci-upstream-bpf-next-kasan-gce 2020/04/26 15:50 bpf-next 3f2eaebb5450 99b258dd .config log report
ci-upstream-bpf-next-kasan-gce 2020/04/25 23:01 bpf-next 3f2eaebb5450 b8bb8e5f .config log report
ci-upstream-bpf-next-kasan-gce 2020/04/25 00:53 bpf-next 3f2eaebb5450 03d97a1b .config log report