syzbot


possible deadlock in __dev_queue_xmit (3)

Status: upstream: reported on 2024/10/25 05:17
Reported-by: syzbot+1f5d85a41ae645abffc0@syzkaller.appspotmail.com
First crash: 95d, last: 3d11h
Similar bugs (12)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
android-49 possible deadlock in __dev_queue_xmit 4 2047d 2116d 0/3 auto-closed as invalid on 2019/10/25 08:36
upstream possible deadlock in __dev_queue_xmit (3) net C done inconclusive 1015 3h44m 1883d 0/28 upstream: reported C repro on 2019/12/03 09:55
linux-4.19 possible deadlock in __dev_queue_xmit C error 5 1224d 1642d 0/1 upstream: reported C repro on 2020/07/31 07:05
android-414 possible deadlock in __dev_queue_xmit 3 2106d 2116d 0/1 auto-closed as invalid on 2019/10/21 21:31
android-44 possible deadlock in __dev_queue_xmit 14 1882d 1944d 0/2 auto-closed as invalid on 2020/04/02 07:14
linux-6.1 possible deadlock in __dev_queue_xmit 98 466d 580d 0/3 auto-obsoleted due to no activity on 2023/12/28 22:22
linux-4.14 possible deadlock in __dev_queue_xmit 7 1593d 2037d 0/1 auto-closed as invalid on 2021/01/16 21:33
upstream possible deadlock in __dev_queue_xmit net 1 2205d 2205d 0/28 closed as invalid on 2019/03/10 18:51
linux-5.15 possible deadlock in __dev_queue_xmit 113 531d 667d 0/3 auto-obsoleted due to no activity on 2023/10/25 04:56
linux-6.1 possible deadlock in __dev_queue_xmit (2) 7 265d 375d 0/3 auto-obsoleted due to no activity on 2024/08/16 05:44
linux-5.15 possible deadlock in __dev_queue_xmit (2) origin:lts-only C done 35 5h10m 410d 0/3 upstream: reported C repro on 2023/12/15 02:11
upstream possible deadlock in __dev_queue_xmit (2) kernel 2 2017d 2133d 0/28 auto-closed as invalid on 2019/11/19 09:01

Sample crash report:
============================================
WARNING: possible recursive locking detected
6.1.127-syzkaller #0 Not tainted
--------------------------------------------
kworker/u4:20/8576 is trying to acquire lock:
ffff888079a1e218 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: spin_lock include/linux/spinlock.h:351 [inline]
ffff888079a1e218 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: __dev_xmit_skb net/core/dev.c:3894 [inline]
ffff888079a1e218 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: __dev_queue_xmit+0x22db/0x3d50 net/core/dev.c:4269

but task is already holding lock:
ffff88801cafe258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: spin_trylock include/linux/spinlock.h:361 [inline]
ffff88801cafe258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: qdisc_run_begin include/net/sch_generic.h:195 [inline]
ffff88801cafe258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: __dev_xmit_skb net/core/dev.c:3851 [inline]
ffff88801cafe258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: __dev_queue_xmit+0x12c7/0x3d50 net/core/dev.c:4269

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock(dev->qdisc_tx_busylock ?: &qdisc_tx_busylock);
  lock(dev->qdisc_tx_busylock ?: &qdisc_tx_busylock);

 *** DEADLOCK ***

 May be due to missing lock nesting notation

8 locks held by kworker/u4:20/8576:
 #0: ffff88807a271938 ((wq_completion)bond1#2){+.+.}-{0:0}, at: process_one_work+0x7a9/0x11d0 kernel/workqueue.c:2267
 #1: ffffc90005b6fd20 ((work_completion)(&(&bond->alb_work)->work)){+.+.}-{0:0}, at: process_one_work+0x7a9/0x11d0 kernel/workqueue.c:2267
 #2: ffffffff8d32b0c0 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:350 [inline]
 #2: ffffffff8d32b0c0 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:791 [inline]
 #2: ffffffff8d32b0c0 (rcu_read_lock){....}-{1:2}, at: bond_alb_monitor+0x10c/0x1750 drivers/net/bonding/bond_alb.c:1547
 #3: ffffffff8d32b0c0 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:350 [inline]
 #3: ffffffff8d32b0c0 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:791 [inline]
 #3: ffffffff8d32b0c0 (rcu_read_lock){....}-{1:2}, at: alb_send_learning_packets+0x1a3/0x370 drivers/net/bonding/bond_alb.c:1017
 #4: ffffffff8d32b120 (rcu_read_lock_bh){....}-{1:2}, at: local_bh_disable include/linux/bottom_half.h:20 [inline]
 #4: ffffffff8d32b120 (rcu_read_lock_bh){....}-{1:2}, at: rcu_read_lock_bh include/linux/rcupdate.h:843 [inline]
 #4: ffffffff8d32b120 (rcu_read_lock_bh){....}-{1:2}, at: __dev_queue_xmit+0x2d6/0x3d50 net/core/dev.c:4226
 #5: ffff88801cafe258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: spin_trylock include/linux/spinlock.h:361 [inline]
 #5: ffff88801cafe258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: qdisc_run_begin include/net/sch_generic.h:195 [inline]
 #5: ffff88801cafe258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: __dev_xmit_skb net/core/dev.c:3851 [inline]
 #5: ffff88801cafe258 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{2:2}, at: __dev_queue_xmit+0x12c7/0x3d50 net/core/dev.c:4269
 #6: ffffffff8d32b0c0 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:350 [inline]
 #6: ffffffff8d32b0c0 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:791 [inline]
 #6: ffffffff8d32b0c0 (rcu_read_lock){....}-{1:2}, at: ip_finish_output2+0x402/0x1330 net/ipv4/ip_output.c:221
 #7: ffffffff8d32b120 (rcu_read_lock_bh){....}-{1:2}, at: local_bh_disable include/linux/bottom_half.h:20 [inline]
 #7: ffffffff8d32b120 (rcu_read_lock_bh){....}-{1:2}, at: rcu_read_lock_bh include/linux/rcupdate.h:843 [inline]
 #7: ffffffff8d32b120 (rcu_read_lock_bh){....}-{1:2}, at: __dev_queue_xmit+0x2d6/0x3d50 net/core/dev.c:4226

stack backtrace:
CPU: 1 PID: 8576 Comm: kworker/u4:20 Not tainted 6.1.127-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 12/27/2024
Workqueue: bond1 bond_alb_monitor
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e3/0x2cb lib/dump_stack.c:106
 print_deadlock_bug kernel/locking/lockdep.c:2983 [inline]
 check_deadlock kernel/locking/lockdep.c:3026 [inline]
 validate_chain+0x4711/0x5950 kernel/locking/lockdep.c:3812
 __lock_acquire+0x125b/0x1f80 kernel/locking/lockdep.c:5049
 lock_acquire+0x1f8/0x5a0 kernel/locking/lockdep.c:5662
 __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
 _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:154
 spin_lock include/linux/spinlock.h:351 [inline]
 __dev_xmit_skb net/core/dev.c:3894 [inline]
 __dev_queue_xmit+0x22db/0x3d50 net/core/dev.c:4269
 dev_queue_xmit include/linux/netdevice.h:3043 [inline]
 neigh_hh_output include/net/neighbour.h:528 [inline]
 neigh_output include/net/neighbour.h:542 [inline]
 ip_finish_output2+0xcfc/0x1330 net/ipv4/ip_output.c:228
 iptunnel_xmit+0x513/0x920 net/ipv4/ip_tunnel_core.c:82
 ip_tunnel_xmit+0x1f87/0x2690 net/ipv4/ip_tunnel.c:858
 __gre_xmit net/ipv4/ip_gre.c:474 [inline]
 gre_tap_xmit+0x4eb/0x6e0 net/ipv4/ip_gre.c:751
 __netdev_start_xmit include/linux/netdevice.h:4888 [inline]
 netdev_start_xmit include/linux/netdevice.h:4902 [inline]
 xmit_one net/core/dev.c:3627 [inline]
 dev_hard_start_xmit+0x261/0x8c0 net/core/dev.c:3643
 sch_direct_xmit+0x2b2/0x5e0 net/sched/sch_generic.c:342
 __dev_xmit_skb net/core/dev.c:3864 [inline]
 __dev_queue_xmit+0x1a7a/0x3d50 net/core/dev.c:4269
 alb_upper_dev_walk+0x219/0x3e0
 netdev_walk_all_upper_dev_rcu+0xf3/0x3c0 net/core/dev.c:7040
 alb_send_learning_packets+0x27b/0x370 drivers/net/bonding/bond_alb.c:1018
 bond_alb_monitor+0x3ef/0x1750 drivers/net/bonding/bond_alb.c:1564
 process_one_work+0x8a9/0x11d0 kernel/workqueue.c:2292
 worker_thread+0xa47/0x1200 kernel/workqueue.c:2439
 kthread+0x28d/0x320 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>

Crashes (10):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2025/01/25 10:44 linux-6.1.y 75cefdf153f5 9fbd772e .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in __dev_queue_xmit
2025/01/23 04:43 linux-6.1.y f4f677285b38 a44b0418 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in __dev_queue_xmit
2025/01/18 22:13 linux-6.1.y 60ceadf9247e f2cb035c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in __dev_queue_xmit
2025/01/16 23:53 linux-6.1.y c63962be84ef f9e07a6e .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in __dev_queue_xmit
2025/01/21 00:18 linux-6.1.y f4f677285b38 6e87cfa2 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 possible deadlock in __dev_queue_xmit
2025/01/15 00:03 linux-6.1.y c63962be84ef f310a27d .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 possible deadlock in __dev_queue_xmit
2025/01/10 15:02 linux-6.1.y c63962be84ef 67d7ec0a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 possible deadlock in __dev_queue_xmit
2025/01/07 11:15 linux-6.1.y 7dc732d24ff7 f3558dbf .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 possible deadlock in __dev_queue_xmit
2024/12/11 07:27 linux-6.1.y e4d90d63d385 cfc402b4 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 possible deadlock in __dev_queue_xmit
2024/10/25 05:16 linux-6.1.y 7ec6f9fa3d97 c79b8ca5 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 possible deadlock in __dev_queue_xmit
* Struck through repros no longer work on HEAD.