syzbot

 sign-in | mailing list | source | docs
🐞 Open [712]
🐞 Fixed [297]
🐞 Invalid [838]
≡ Crashes
Kernel Health Bugs/Month Bug Lifetimes Fuzzing
💬 Send us feedback

possible deadlock in __dev_queue_xmit

Status: upstream: reported C repro on 2020/07/31 07:05
Reported-by: syzbot+ea0ea1495c431e6d12a3@syzkaller.appspotmail.com
First crash: 1552d, last: 1134d
Fix bisection: failed (error log, bisect log)
  
▶ ▼ Similar bugs (12)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
android-49 possible deadlock in __dev_queue_xmit 4 1957d 2026d 0/3 auto-closed as invalid on 2019/10/25 08:36
upstream possible deadlock in __dev_queue_xmit (3) net C done inconclusive 1012 111d 1793d 0/28 upstream: reported C repro on 2019/12/03 09:55
android-414 possible deadlock in __dev_queue_xmit 3 2016d 2026d 0/1 auto-closed as invalid on 2019/10/21 21:31
linux-6.1 possible deadlock in __dev_queue_xmit (3) 1 5d16h 5d16h 0/3 upstream: reported on 2024/10/25 05:17
android-44 possible deadlock in __dev_queue_xmit 14 1792d 1854d 0/2 auto-closed as invalid on 2020/04/02 07:14
linux-6.1 possible deadlock in __dev_queue_xmit 98 376d 490d 0/3 auto-obsoleted due to no activity on 2023/12/28 22:22
linux-4.14 possible deadlock in __dev_queue_xmit 7 1502d 1947d 0/1 auto-closed as invalid on 2021/01/16 21:33
upstream possible deadlock in __dev_queue_xmit net 1 2115d 2115d 0/28 closed as invalid on 2019/03/10 18:51
linux-5.15 possible deadlock in __dev_queue_xmit 113 441d 577d 0/3 auto-obsoleted due to no activity on 2023/10/25 04:56
linux-6.1 possible deadlock in __dev_queue_xmit (2) 7 175d 285d 0/3 auto-obsoleted due to no activity on 2024/08/16 05:44
linux-5.15 possible deadlock in __dev_queue_xmit (2) origin:lts-only C done 14 19d 320d 0/3 upstream: reported C repro on 2023/12/15 02:11
upstream possible deadlock in __dev_queue_xmit (2) kernel 2 1927d 2043d 0/28 auto-closed as invalid on 2019/11/19 09:01
▶ ▼ Fix bisection attempts (15)
Created Duration User Patch Repo Result
2021/10/23 05:58 17m bisect fix linux-4.19.y error job log
2021/09/22 11:15 26m bisect fix linux-4.19.y OK (0) job log log
2021/08/21 22:24 30m bisect fix linux-4.19.y OK (0) job log log
2021/07/22 21:55 28m bisect fix linux-4.19.y OK (0) job log log
2021/06/22 15:30 32m bisect fix linux-4.19.y OK (0) job log log
2021/05/23 07:54 31m bisect fix linux-4.19.y OK (0) job log log
2021/04/23 05:31 23m bisect fix linux-4.19.y OK (0) job log log
2021/03/24 01:41 22m bisect fix linux-4.19.y OK (0) job log log
2021/02/22 00:53 23m bisect fix linux-4.19.y OK (0) job log log
2021/02/18 11:54 19m bisect fix linux-4.19.y error job log
2021/02/10 13:15 1m bisect fix linux-4.19.y error job log
2021/01/11 12:51 23m bisect fix linux-4.19.y OK (0) job log log
2020/11/14 20:36 24m bisect fix linux-4.19.y OK (0) job log log
2020/09/29 07:33 24m bisect fix linux-4.19.y OK (0) job log log
2020/08/30 07:05 28m bisect fix linux-4.19.y OK (0) job log log

Sample crash report:
batman_adv: batadv0: Interface activated: batadv_slave_1
IPv6: ADDRCONF(NETDEV_CHANGE): batadv_slave_1: link becomes ready
IPv6: ADDRCONF(NETDEV_CHANGE): veth1_to_batadv: link becomes ready
============================================
WARNING: possible recursive locking detected
4.19.135-syzkaller #0 Not tainted
--------------------------------------------
syz-executor916/6447 is trying to acquire lock:
00000000ec7871f0 (_xmit_ETHER#2){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline]
00000000ec7871f0 (_xmit_ETHER#2){+.-.}, at: __netif_tx_lock include/linux/netdevice.h:3828 [inline]
00000000ec7871f0 (_xmit_ETHER#2){+.-.}, at: __dev_queue_xmit+0x2592/0x2e00 net/core/dev.c:3834

but task is already holding lock:
000000002216d9e3 (_xmit_ETHER#2){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline]
000000002216d9e3 (_xmit_ETHER#2){+.-.}, at: __netif_tx_lock include/linux/netdevice.h:3828 [inline]
000000002216d9e3 (_xmit_ETHER#2){+.-.}, at: sch_direct_xmit+0x254/0xf50 net/sched/sch_generic.c:330

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock(_xmit_ETHER#2);
  lock(_xmit_ETHER#2);

 *** DEADLOCK ***

 May be due to missing lock nesting notation

11 locks held by syz-executor916/6447:
 #0: 0000000041be4ca5 (rcu_read_lock){....}, at: l3mdev_l3_out include/net/l3mdev.h:172 [inline]
 #0: 0000000041be4ca5 (rcu_read_lock){....}, at: l3mdev_ip6_out include/net/l3mdev.h:193 [inline]
 #0: 0000000041be4ca5 (rcu_read_lock){....}, at: rawv6_send_hdrinc net/ipv6/raw.c:682 [inline]
 #0: 0000000041be4ca5 (rcu_read_lock){....}, at: rawv6_sendmsg+0x1e3d/0x36a0 net/ipv6/raw.c:947
 #1: 00000000fac33b54 (rcu_read_lock_bh){....}, at: lwtunnel_xmit_redirect include/net/lwtunnel.h:92 [inline]
 #1: 00000000fac33b54 (rcu_read_lock_bh){....}, at: ip6_finish_output2+0x1f2/0x2290 net/ipv6/ip6_output.c:106
 #2: 00000000fac33b54 (rcu_read_lock_bh){....}, at: __dev_queue_xmit+0x1e2/0x2e00 net/core/dev.c:3773
 #3: 00000000cac4d69a (&(&sch->seqlock)->rlock){+...}, at: spin_trylock include/linux/spinlock.h:339 [inline]
 #3: 00000000cac4d69a (&(&sch->seqlock)->rlock){+...}, at: qdisc_run_begin include/net/sch_generic.h:130 [inline]
 #3: 00000000cac4d69a (&(&sch->seqlock)->rlock){+...}, at: qdisc_run include/net/pkt_sched.h:119 [inline]
 #3: 00000000cac4d69a (&(&sch->seqlock)->rlock){+...}, at: __dev_xmit_skb net/core/dev.c:3451 [inline]
 #3: 00000000cac4d69a (&(&sch->seqlock)->rlock){+...}, at: __dev_queue_xmit+0x278b/0x2e00 net/core/dev.c:3807
 #4: 00000000ce3642cd (dev->qdisc_running_key ?: &qdisc_running_key){+...}, at: neigh_resolve_output+0x55a/0x910 net/core/neighbour.c:1374
 #5: 000000002216d9e3 (_xmit_ETHER#2){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline]
 #5: 000000002216d9e3 (_xmit_ETHER#2){+.-.}, at: __netif_tx_lock include/linux/netdevice.h:3828 [inline]
 #5: 000000002216d9e3 (_xmit_ETHER#2){+.-.}, at: sch_direct_xmit+0x254/0xf50 net/sched/sch_generic.c:330
 #6: 0000000041be4ca5 (rcu_read_lock){....}, at: icmpv6_send+0x0/0x210 net/ipv6/ip6_icmp.c:31
 #7: 0000000094cb44d9 (k-slock-AF_INET6){+...}, at: spin_trylock include/linux/spinlock.h:339 [inline]
 #7: 0000000094cb44d9 (k-slock-AF_INET6){+...}, at: icmpv6_xmit_lock net/ipv6/icmp.c:119 [inline]
 #7: 0000000094cb44d9 (k-slock-AF_INET6){+...}, at: icmp6_send+0xfea/0x2230 net/ipv6/icmp.c:532
 #8: 0000000041be4ca5 (rcu_read_lock){....}, at: icmp6_send+0x1685/0x2230 net/ipv6/icmp.c:565
 #9: 00000000fac33b54 (rcu_read_lock_bh){....}, at: lwtunnel_xmit_redirect include/net/lwtunnel.h:92 [inline]
 #9: 00000000fac33b54 (rcu_read_lock_bh){....}, at: ip6_finish_output2+0x1f2/0x2290 net/ipv6/ip6_output.c:106
 #10: 00000000fac33b54 (rcu_read_lock_bh){....}, at: __dev_queue_xmit+0x1e2/0x2e00 net/core/dev.c:3773

stack backtrace:
CPU: 0 PID: 6447 Comm: syz-executor916 Not tainted 4.19.135-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x1fc/0x2fe lib/dump_stack.c:118
 print_deadlock_bug kernel/locking/lockdep.c:1763 [inline]
 check_deadlock kernel/locking/lockdep.c:1807 [inline]
 validate_chain kernel/locking/lockdep.c:2403 [inline]
 __lock_acquire.cold+0x121/0x57e kernel/locking/lockdep.c:3415
 lock_acquire+0x170/0x3c0 kernel/locking/lockdep.c:3907
 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
 _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
 spin_lock include/linux/spinlock.h:329 [inline]
 __netif_tx_lock include/linux/netdevice.h:3828 [inline]
 __dev_queue_xmit+0x2592/0x2e00 net/core/dev.c:3834
 neigh_resolve_output+0x55a/0x910 net/core/neighbour.c:1374
 neigh_output include/net/neighbour.h:501 [inline]
 ip6_finish_output2+0x113d/0x2290 net/ipv6/ip6_output.c:120
 ip6_finish_output+0x7eb/0xc10 net/ipv6/ip6_output.c:154
 NF_HOOK_COND include/linux/netfilter.h:278 [inline]
 ip6_output+0x205/0x770 net/ipv6/ip6_output.c:171
 dst_output include/net/dst.h:455 [inline]
 ip6_local_out+0xaf/0x170 net/ipv6/output_core.c:178
 ip6_send_skb+0xb3/0x300 net/ipv6/ip6_output.c:1699
 ip6_push_pending_frames+0xbd/0xe0 net/ipv6/ip6_output.c:1719
 icmpv6_push_pending_frames+0x294/0x470 net/ipv6/icmp.c:288
 icmp6_send+0x1b5f/0x2230 net/ipv6/icmp.c:584
 icmpv6_send+0xde/0x210 net/ipv6/ip6_icmp.c:43
 ip6_link_failure+0x26/0x4e0 net/ipv6/route.c:2297
 dst_link_failure include/net/dst.h:438 [inline]
 ip_tunnel_xmit+0x195f/0x37f0 net/ipv4/ip_tunnel.c:789
 erspan_xmit+0x1837/0x2790 net/ipv4/ip_gre.c:756
 __netdev_start_xmit include/linux/netdevice.h:4333 [inline]
 netdev_start_xmit include/linux/netdevice.h:4347 [inline]
 xmit_one net/core/dev.c:3256 [inline]
 dev_hard_start_xmit+0x1a8/0x920 net/core/dev.c:3272
 sch_direct_xmit+0x2d6/0xf50 net/sched/sch_generic.c:332
 qdisc_restart net/sched/sch_generic.c:395 [inline]
 __qdisc_run+0x4d0/0x1630 net/sched/sch_generic.c:403
 qdisc_run include/net/pkt_sched.h:120 [inline]
 __dev_xmit_skb net/core/dev.c:3451 [inline]
 __dev_queue_xmit+0x2102/0x2e00 net/core/dev.c:3807
 neigh_resolve_output+0x55a/0x910 net/core/neighbour.c:1374
 neigh_output include/net/neighbour.h:501 [inline]
 ip6_finish_output2+0x113d/0x2290 net/ipv6/ip6_output.c:120
 ip6_finish_output+0x7eb/0xc10 net/ipv6/ip6_output.c:154
 NF_HOOK_COND include/linux/netfilter.h:278 [inline]
 ip6_output+0x205/0x770 net/ipv6/ip6_output.c:171
 dst_output include/net/dst.h:455 [inline]
 NF_HOOK include/linux/netfilter.h:289 [inline]
 rawv6_send_hdrinc net/ipv6/raw.c:692 [inline]
 rawv6_sendmsg+0x202c/0x36a0 net/ipv6/raw.c:947
 inet_sendmsg+0x132/0x5a0 net/ipv4/af_inet.c:798
 sock_sendmsg_nosec net/socket.c:622 [inline]
 sock_sendmsg+0xc3/0x120 net/socket.c:632
 sock_write_iter+0x287/0x3c0 net/socket.c:901
 call_write_iter include/linux/fs.h:1821 [inline]
 new_sync_write fs/read_write.c:474 [inline]
 __vfs_write+0x51b/0x770 fs/read_write.c:487
 vfs_write+0x1f3/0x540 fs/read_write.c:549
 ksys_write+0x12b/0x2a0 fs/read_write.c:599
 do_syscall_64+0xf9/0x620 arch/x86/entry/common.c:293
 entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x449ee9
Code: 18 89 d0 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 2b 0e fc ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007fffc4bee918 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 0000000000000004 RCX: 0000000000449ee9
RDX: 0000000000000028 RSI: 0000000020000140 RDI: 0000000000000005
RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007fffc4bee990
R13: 0000000000000003 R14: 0000000000959914 R15: 0000000000000013

Crashes (5):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2020/07/31 07:05 linux-4.19.y 205a42ce2861 8df85ed9 .config console log report syz C ci2-linux-4-19
2021/08/23 10:52 linux-4.19.y 59456c9cc40c b599f2fc .config console log report info ci2-linux-4-19 possible deadlock in __dev_queue_xmit
2020/12/12 12:51 linux-4.19.y 13d2ce42de8c bca53db9 .config console log report info ci2-linux-4-19
2020/11/24 12:34 linux-4.19.y 76bda503e640 1ab681a4 .config console log report info ci2-linux-4-19
2020/10/15 20:35 linux-4.19.y a1b977b49b66 6e262c73 .config console log report info ci2-linux-4-19
* Struck through repros no longer work on HEAD.