syzbot


INFO: rcu detected stall in sys_exit_group (7)

Status: upstream: reported C repro on 2021/10/15 18:41
Labels: kasan mm net (incorrect?)
Reported-by: syzbot+1c1c0d391f04584c1611@syzkaller.appspotmail.com
First crash: 623d, last: 3h08m

Cause bisection: introduced by (bisect log) [no-op commit]:
commit fcd29ad17c6ff885dfae58f557e9323941e63ba2
Author: Feras Daoud <ferasda@mellanox.com>
Date: Thu Aug 9 06:55:21 2018 +0000

  net/mlx5: Add Fast teardown support

Crash: general protection fault in batadv_iv_ogm_queue_add (log)
Repro: C syz .config

Fix bisection: the fix commit could be any of (bisect log):
  64570fbc14f8 Linux 5.15-rc5
  3bc1bc0b59d0 Merge tag '5.20-rc-smb3-client-fixes-part1' of git://git.samba.org/sfrench/cifs-2.6
Discussions (1)
Title Replies (including bot) Last reply
[syzbot] INFO: rcu detected stall in sys_exit_group (7) 0 (1) 2021/10/15 18:41
Similar bugs (7)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in sys_exit_group (6) mm C 4 974d 981d 21/24 fixed on 2021/03/10 01:48
linux-6.1 INFO: rcu detected stall in sys_exit_group 1 22d 22d 0/3 upstream: reported on 2023/05/10 22:25
upstream INFO: rcu detected stall in sys_exit_group (2) 56 1276d 1277d 0/24 closed as invalid on 2019/12/04 14:14
upstream INFO: rcu detected stall in sys_exit_group (3) 8 1241d 1241d 0/24 closed as invalid on 2020/01/08 05:23
upstream INFO: rcu detected stall in sys_exit_group (4) 13 1241d 1241d 0/24 closed as invalid on 2020/01/09 08:13
upstream INFO: rcu detected stall in sys_exit_group (5) 1 1162d 1162d 0/24 auto-closed as invalid on 2020/06/25 07:58
upstream INFO: rcu detected stall in sys_exit_group kernel C done 1 1357d 1353d 14/24 fixed on 2019/10/09 10:54
Last patch testing requests (2)
Created Duration User Patch Repo Result
2023/05/21 21:41 17m retest repro upstream report log
2022/12/23 02:31 16m retest repro upstream report log

Sample crash report:
rcu: INFO: rcu_preempt self-detected stall on CPU
rcu: 	0-...!: (1 GPs behind) idle=fb54/1/0x4000000000000000 softirq=9029/9034 fqs=0
rcu: 	(t=10501 jiffies g=8601 q=80 ncpus=2)
rcu: rcu_preempt kthread timer wakeup didn't happen for 10501 jiffies! g8601 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402
rcu: 	Possible timer handling issue on cpu=1 timer-softirq=2887
rcu: rcu_preempt kthread starved for 10504 jiffies! g8601 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:I stack:28808 pid:16    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5343 [inline]
 __schedule+0xc9a/0x5880 kernel/sched/core.c:6669
 schedule+0xde/0x1a0 kernel/sched/core.c:6745
 schedule_timeout+0x14e/0x2b0 kernel/time/timer.c:2167
 rcu_gp_fqs_loop+0x190/0x910 kernel/rcu/tree.c:1609
 rcu_gp_kthread+0x23a/0x360 kernel/rcu/tree.c:1808
 kthread+0x344/0x440 kernel/kthread.c:379
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 5685 Comm: syz-executor319 Not tainted 6.4.0-rc4-syzkaller-00099-g1874a42a7d74 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/25/2023
RIP: 0010:kvm_wait+0xa8/0x110 arch/x86/kernel/kvm.c:1064
Code: fa 83 e2 07 38 d0 7f 04 84 c0 75 66 0f b6 07 40 38 c6 74 1b 48 83 c4 10 c3 c3 e8 33 9d 52 00 66 90 0f 00 2d 0a 47 11 09 fb f4 <48> 83 c4 10 c3 66 90 0f 00 2d fa 46 11 09 f4 48 83 c4 10 c3 89 74
RSP: 0018:ffffc900056cf110 EFLAGS: 00000242
RAX: 000000000000a6b6 RBX: 0000000000000000 RCX: 1ffffffff22ae37e
RDX: 0000000000000000 RSI: 0000000000000201 RDI: 0000000000000000
RBP: ffff888021ad48f0 R08: 0000000000000001 R09: ffffffff91528dc7
R10: 0000000000000001 R11: 0000000000094001 R12: 0000000000000000
R13: ffffed100435a91e R14: 0000000000000001 R15: ffff8880b993d440
FS:  0000555555d20300(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000020000080 CR3: 000000002a9ef000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <TASK>
 pv_wait arch/x86/include/asm/paravirt.h:598 [inline]
 pv_wait_head_or_lock kernel/locking/qspinlock_paravirt.h:470 [inline]
 __pv_queued_spin_lock_slowpath+0x8cb/0xb50 kernel/locking/qspinlock.c:511
 pv_queued_spin_lock_slowpath arch/x86/include/asm/paravirt.h:586 [inline]
 queued_spin_lock_slowpath arch/x86/include/asm/qspinlock.h:51 [inline]
 queued_spin_lock include/asm-generic/qspinlock.h:114 [inline]
 do_raw_spin_lock+0x204/0x2b0 kernel/locking/spinlock_debug.c:115
 spin_lock_bh include/linux/spinlock.h:355 [inline]
 sch_tree_lock include/net/sch_generic.h:569 [inline]
 sch_tree_lock include/net/sch_generic.h:564 [inline]
 fq_pie_change+0x1ed/0xfd0 net/sched/sch_fq_pie.c:290
 fq_pie_init+0x4b9/0x8e0 net/sched/sch_fq_pie.c:411
 qdisc_create+0x4d1/0x1040 net/sched/sch_api.c:1297
 tc_modify_qdisc+0x488/0x1aa0 net/sched/sch_api.c:1682
 rtnetlink_rcv_msg+0x43d/0xd50 net/core/rtnetlink.c:6395
 netlink_rcv_skb+0x165/0x440 net/netlink/af_netlink.c:2546
 netlink_unicast_kernel net/netlink/af_netlink.c:1339 [inline]
 netlink_unicast+0x547/0x7f0 net/netlink/af_netlink.c:1365
 netlink_sendmsg+0x925/0xe30 net/netlink/af_netlink.c:1913
 sock_sendmsg_nosec net/socket.c:724 [inline]
 sock_sendmsg+0xde/0x190 net/socket.c:747
 ____sys_sendmsg+0x71c/0x900 net/socket.c:2503
 ___sys_sendmsg+0x110/0x1b0 net/socket.c:2557
 __sys_sendmsg+0xf7/0x1c0 net/socket.c:2586
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7f9e19dd6c89
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 41 15 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 c0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffc5181d978 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007f9e19dd6c89
RDX: 0000000000000000 RSI: 00000000200007c0 RDI: 0000000000000003
RBP: 0000000000000000 R08: 000000000000000d R09: 000000000000000d
R10: 000000000000000d R11: 0000000000000246 R12: 00007ffc5181d990
R13: 00000000000f4240 R14: 0000000000027f98 R15: 00007ffc5181d984
 </TASK>
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 2.911 msecs
CPU: 0 PID: 5684 Comm: syz-executor319 Not tainted 6.4.0-rc4-syzkaller-00099-g1874a42a7d74 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/25/2023
RIP: 0010:check_kcov_mode kernel/kcov.c:173 [inline]
RIP: 0010:__sanitizer_cov_trace_pc+0xb/0x70 kernel/kcov.c:207
Code: 0f 1e fa 48 8b be a8 01 00 00 e8 b0 ff ff ff 31 c0 c3 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 f3 0f 1e fa 65 8b 05 9d 77 7f 7e <89> c1 48 8b 34 24 81 e1 00 01 00 00 65 48 8b 14 25 c0 bb 03 00 a9
RSP: 0018:ffffc90000007618 EFLAGS: 00000246
RAX: 0000000000000302 RBX: ffff888073231ee0 RCX: 0000000000000100
RDX: ffff888022cc1dc0 RSI: ffffffff883ef518 RDI: ffff888073231ee8
RBP: ffff888021ad4800 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000005 R12: dffffc0000000000
R13: ffff888021ad4ae0 R14: ffff888021ad4af0 R15: 0000000000000001
FS:  0000000000000000(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9e19e53370 CR3: 000000000c571000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 list_empty include/linux/list.h:292 [inline]
 fq_pie_qdisc_dequeue+0x53/0x9c0 net/sched/sch_fq_pie.c:238
 dequeue_skb net/sched/sch_generic.c:292 [inline]
 qdisc_restart net/sched/sch_generic.c:397 [inline]
 __qdisc_run+0x1b2/0x1780 net/sched/sch_generic.c:415
 __dev_xmit_skb net/core/dev.c:3868 [inline]
 __dev_queue_xmit+0x2215/0x3b10 net/core/dev.c:4210
 dev_queue_xmit include/linux/netdevice.h:3085 [inline]
 neigh_connected_output+0x3c2/0x550 net/core/neighbour.c:1581
 neigh_output include/net/neighbour.h:544 [inline]
 ip6_finish_output2+0x55a/0x1560 net/ipv6/ip6_output.c:134
 __ip6_finish_output net/ipv6/ip6_output.c:195 [inline]
 ip6_finish_output+0x69a/0x1170 net/ipv6/ip6_output.c:206
 NF_HOOK_COND include/linux/netfilter.h:292 [inline]
 ip6_output+0x1f1/0x540 net/ipv6/ip6_output.c:227
 dst_output include/net/dst.h:458 [inline]
 NF_HOOK include/linux/netfilter.h:303 [inline]
 ndisc_send_skb+0xa63/0x1850 net/ipv6/ndisc.c:508
 ndisc_send_rs+0x132/0x6f0 net/ipv6/ndisc.c:718
 addrconf_rs_timer+0x3f1/0x870 net/ipv6/addrconf.c:3936
 call_timer_fn+0x1a0/0x580 kernel/time/timer.c:1700
 expire_timers+0x29b/0x4b0 kernel/time/timer.c:1751
 __run_timers kernel/time/timer.c:2022 [inline]
 __run_timers kernel/time/timer.c:1995 [inline]
 run_timer_softirq+0x326/0x910 kernel/time/timer.c:2035
 __do_softirq+0x1d4/0x905 kernel/softirq.c:571
 invoke_softirq kernel/softirq.c:445 [inline]
 __irq_exit_rcu+0x114/0x190 kernel/softirq.c:650
 irq_exit_rcu+0x9/0x20 kernel/softirq.c:662
 sysvec_apic_timer_interrupt+0x97/0xc0 arch/x86/kernel/apic/apic.c:1106
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:645
RIP: 0010:lock_acquire+0x1f5/0x520 kernel/locking/lockdep.c:5673
Code: 99 9c 7e 83 f8 01 0f 85 b9 02 00 00 9c 58 f6 c4 02 0f 85 a4 02 00 00 48 83 7c 24 08 00 74 01 fb 48 b8 00 00 00 00 00 fc ff df <48> 01 c3 48 c7 03 00 00 00 00 48 c7 43 08 00 00 00 00 48 8b 84 24
RSP: 0018:ffffc9000553fcf8 EFLAGS: 00000206
RAX: dffffc0000000000 RBX: 1ffff92000aa7fa1 RCX: 000000000000426a
RDX: 1ffff11004598510 RSI: 0000000000000001 RDI: 0000000000000000
RBP: 0000000000000001 R08: 0000000000000000 R09: ffffffff91528d17
R10: fffffbfff22a51a2 R11: 0000000000094001 R12: 0000000000000000
R13: 0000000000000000 R14: ffff888022cc2780 R15: 0000000000000000
 __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
 _raw_spin_lock+0x2e/0x40 kernel/locking/spinlock.c:154
 spin_lock include/linux/spinlock.h:350 [inline]
 task_lock include/linux/sched/task.h:184 [inline]
 mpol_put_task_policy+0x1f/0x80 mm/mempolicy.c:2647
 do_exit+0x1592/0x2960 kernel/exit.c:893
 do_group_exit+0xd4/0x2a0 kernel/exit.c:1021
 __do_sys_exit_group kernel/exit.c:1032 [inline]
 __se_sys_exit_group kernel/exit.c:1030 [inline]
 __x64_sys_exit_group+0x3e/0x50 kernel/exit.c:1030
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7f9e19dd5839
Code: Unable to access opcode bytes at 0x7f9e19dd580f.
RSP: 002b:00007ffc5181d928 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
RAX: ffffffffffffffda RBX: 00007f9e19e50410 RCX: 00007f9e19dd5839
RDX: 000000000000003c RSI: 00000000000000e7 RDI: 0000000000000000
RBP: 0000000000000000 R08: ffffffffffffffc0 R09: 000000000000000d
R10: 000000000000000d R11: 0000000000000246 R12: 00007f9e19e50410
R13: 0000000000000001 R14: 0000000000000000 R15: 0000000000000001
 </TASK>

Crashes (20):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets Manager Title
2023/06/02 11:07 upstream 1874a42a7d74 a4ae4f42 .config console log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: rcu detected stall in sys_exit_group
2021/10/11 19:51 upstream 64570fbc14f8 838e7e2c .config console log report syz C ci-upstream-kasan-gce-root INFO: rcu detected stall in sys_exit_group
2023/06/01 22:51 net be7f8012a513 a4ae4f42 .config console log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: rcu detected stall in sys_exit_group
2023/05/29 07:34 upstream e338142b39cf cf184559 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_exit_group
2023/03/12 20:55 upstream 134231664868 5205ef30 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_exit_group
2023/02/09 09:24 upstream 0983f6bf2bfc 14a312c8 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in sys_exit_group
2022/12/31 06:25 upstream c8451c141e07 ab32d508 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_exit_group
2022/09/14 00:53 upstream d1221cea11fc b884348d .config console log report info [disk image] [vmlinux] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in sys_exit_group
2022/07/08 18:35 upstream e8a4e1c1bb69 b5765a15 .config console log report info ci-upstream-kasan-gce-root INFO: rcu detected stall in sys_exit_group
2022/07/03 00:45 upstream 34074da5424c 1434eec0 .config console log report info ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in sys_exit_group
2022/05/09 11:34 upstream c5eb0a61238d 8b277b8e .config console log report info ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_exit_group
2022/03/14 11:54 upstream 09688c0166e7 9e8eaa75 .config console log report info ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_exit_group
2022/02/24 00:33 upstream 23d04328444a 6e821dbf .config console log report info ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in sys_exit_group
2022/02/08 13:51 upstream 555f3d7be91a 0b33604d .config console log report info ci-upstream-kasan-gce-root INFO: rcu detected stall in sys_exit_group
2022/02/03 14:07 upstream 88808fbbead4 4ebb2798 .config console log report info ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in sys_exit_group
2021/11/08 14:41 upstream 6b75d88fa81b d29682f1 .config console log report info ci-upstream-kasan-gce INFO: rcu detected stall in sys_exit_group
2021/09/16 16:23 upstream ff1ffd71d5f0 aae492f2 .config console log report info ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in sys_exit_group
2023/02/12 07:52 https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb.git usb-testing f87b564686ee 93e26d60 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-usb INFO: rcu detected stall in sys_exit_group
2023/01/27 17:56 https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb.git usb-testing c52c9acc415e 9dfcf09c .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-usb INFO: rcu detected stall in sys_exit_group
2022/02/21 23:13 linux-next ef6b35306dd8 6e821dbf .config console log report info ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in sys_exit_group
* Struck through repros no longer work on HEAD.