syzbot


INFO: rcu detected stall in sctp_generate_heartbeat_event (4)

Status: closed as invalid on 2022/02/08 10:10
Reported-by: syzbot+@syzkaller.appspotmail.com
First crash: 304d, last: 304d
similar bugs (6):
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in sctp_generate_heartbeat_event (3) 4 627d 752d 0/24 auto-closed as invalid on 2021/04/15 02:19
linux-4.14 INFO: rcu detected stall in sctp_generate_heartbeat_event 29 769d 1012d 0/1 auto-closed as invalid on 2020/12/24 11:25
upstream INFO: rcu detected stall in sctp_generate_heartbeat_event 2 1600d 1610d 9/24 fixed on 2018/07/09 18:05
upstream INFO: rcu detected stall in sctp_generate_heartbeat_event (2) 1 1099d 1097d 0/24 auto-closed as invalid on 2019/12/30 14:27
linux-4.14 BUG: soft lockup in sctp_generate_heartbeat_event (2) 1 559d 559d 0/1 auto-closed as invalid on 2021/07/22 12:40
linux-4.19 BUG: soft lockup in sctp_generate_heartbeat_event C error 13 91d 516d 0/1 upstream: reported C repro on 2021/05/06 15:10

Sample crash report:
rcu: INFO: rcu_preempt self-detected stall on CPU
rcu: 	1-...!: (10499 ticks this GP) idle=60d/1/0x4000000000000000 softirq=398969/398969 fqs=0 
	(t=10500 jiffies g=483925 q=429)
rcu: rcu_preempt kthread starved for 10500 jiffies! g483925 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:28688 pid:   14 ppid:     2 flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:4972 [inline]
 __schedule+0xa9a/0x4940 kernel/sched/core.c:6253
 schedule+0xd2/0x260 kernel/sched/core.c:6326
 schedule_timeout+0x14a/0x2a0 kernel/time/timer.c:1881
 rcu_gp_fqs_loop+0x186/0x810 kernel/rcu/tree.c:1955
 rcu_gp_kthread+0x1de/0x320 kernel/rcu/tree.c:2128
 kthread+0x405/0x4f0 kernel/kthread.c:327
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 30957 Comm: syz-executor.5 Not tainted 5.16.0-rc3-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
RIP: 0010:__run_hrtimer kernel/time/hrtimer.c:1713 [inline]
RIP: 0010:__hrtimer_run_queues+0x2d6/0xe50 kernel/time/hrtimer.c:1749
Code: ea 03 0f b6 04 02 84 c0 74 08 3c 03 0f 8e 65 0a 00 00 41 83 45 10 01 48 b8 00 00 00 00 00 fc ff df 48 8b 14 24 41 83 45 10 01 <48> c1 ea 03 80 3c 02 00 0f 85 31 0a 00 00 48 89 e8 48 c1 e0 07 49
RSP: 0018:ffffc900000078f8 EFLAGS: 00000002
RAX: dffffc0000000000 RBX: ffff888049281308 RCX: 0000000000000100
RDX: ffff8880b9c2a4c8 RSI: ffffffff8167123b RDI: ffff888049281308
RBP: 0000000000000000 R08: 000003552882dedf R09: 0000000000000001
R10: ffffffff8400059f R11: 0000000000000000 R12: 0000000000000000
R13: ffff8880b9c2a480 R14: ffff8880b9c2a400 R15: 0000000000000001
FS:  0000000000000000(0000) GS:ffff8880b9c00000(0063) knlGS:00000000f5f3eb40
CS:  0010 DS: 002b ES: 002b CR0: 0000000080050033
CR2: 000000002c920000 CR3: 000000003b2e2000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 hrtimer_interrupt+0x31c/0x790 kernel/time/hrtimer.c:1811
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1086 [inline]
 __sysvec_apic_timer_interrupt+0x146/0x530 arch/x86/kernel/apic/apic.c:1103
 sysvec_apic_timer_interrupt+0x40/0xc0 arch/x86/kernel/apic/apic.c:1097
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:pv_wait_node kernel/locking/qspinlock_paravirt.h:301 [inline]
RIP: 0010:__pv_queued_spin_lock_slowpath+0x5c4/0xb40 kernel/locking/qspinlock.c:473
Code: 32 84 c0 75 23 41 0f b6 14 24 4c 89 e9 83 e1 07 38 ca 7f 08 84 d2 0f 85 ed 03 00 00 0f b6 53 14 84 d2 0f 85 9e 00 00 00 f3 90 <83> e8 01 0f 84 93 00 00 00 0f b6 16 84 d2 74 09 80 fa 03 0f 8e 96
RSP: 0018:ffffc90000007b70 EFLAGS: 00000202
RAX: 000000000000343d RBX: ffff8880b9d3a880 RCX: 0000000000000004
RDX: 0000000000000000 RSI: ffffed1017387511 RDI: ffff8880b9c3a894
RBP: ffff8880b9c3a894 R08: 0000000000000001 R09: ffff8880b9c3a894
R10: ffffed1017387512 R11: 0000000000000000 R12: ffffed10173a7512
R13: ffff8880b9d3a894 R14: dffffc0000000000 R15: ffff8880b9c3a880
 pv_queued_spin_lock_slowpath arch/x86/include/asm/paravirt.h:591 [inline]
 queued_spin_lock_slowpath arch/x86/include/asm/qspinlock.h:51 [inline]
 queued_spin_lock include/asm-generic/qspinlock.h:85 [inline]
 do_raw_spin_lock+0x200/0x2b0 kernel/locking/spinlock_debug.c:115
 spin_lock include/linux/spinlock.h:349 [inline]
 sctp_generate_heartbeat_event+0xa1/0x490 net/sctp/sm_sideeffect.c:371
 call_timer_fn+0x1a5/0x6b0 kernel/time/timer.c:1421
 expire_timers kernel/time/timer.c:1466 [inline]
 __run_timers.part.0+0x675/0xa20 kernel/time/timer.c:1734
 __run_timers kernel/time/timer.c:1715 [inline]
 run_timer_softirq+0xb3/0x1d0 kernel/time/timer.c:1747
 __do_softirq+0x29b/0x9c2 kernel/softirq.c:558
 invoke_softirq kernel/softirq.c:432 [inline]
 __irq_exit_rcu+0x123/0x180 kernel/softirq.c:636
 irq_exit_rcu+0x5/0x20 kernel/softirq.c:648
 sysvec_apic_timer_interrupt+0x93/0xc0 arch/x86/kernel/apic/apic.c:1097
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:__sanitizer_cov_trace_switch+0xb/0xf0 kernel/kcov.c:300
Code: 65 48 8b 04 25 40 70 02 00 48 8b 80 98 15 00 00 c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 41 55 49 89 fb 49 89 f0 41 54 55 <53> 48 8b 46 08 48 83 f8 20 0f 84 c4 00 00 00 0f 87 9e 00 00 00 48
RSP: 0018:ffffc90004d57b20 EFLAGS: 00000246
RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffc90002b94000
RDX: 0000000000040000 RSI: ffffffff89bb26a0 RDI: 000000000000001b
RBP: ffffc90004d57cf0 R08: ffffffff89bb26a0 R09: 0000000000000000
R10: ffffffff81e3822f R11: 000000000000001b R12: 000000000000001b
R13: 000000000000001b R14: ffff888082c44938 R15: dffffc0000000000
 io_req_prep fs/io_uring.c:6417 [inline]
 io_init_req fs/io_uring.c:7187 [inline]
 io_submit_sqe fs/io_uring.c:7197 [inline]
 io_submit_sqes+0xaa5/0x8a20 fs/io_uring.c:7369
 __do_sys_io_uring_enter+0xf6e/0x1f50 fs/io_uring.c:10070
 do_syscall_32_irqs_on arch/x86/entry/common.c:112 [inline]
 __do_fast_syscall_32+0x65/0xf0 arch/x86/entry/common.c:178
 do_fast_syscall_32+0x2f/0x70 arch/x86/entry/common.c:203
 entry_SYSENTER_compat_after_hwframe+0x4d/0x5c
RIP: 0023:0xf6f44549
Code: 03 74 c0 01 10 05 03 74 b8 01 10 06 03 74 b4 01 10 07 03 74 b0 01 10 08 03 74 d8 01 00 00 00 00 00 51 52 55 89 e5 0f 34 cd 80 <5d> 5a 59 c3 90 90 90 90 8d b4 26 00 00 00 00 8d b4 26 00 00 00 00
RSP: 002b:00000000f5f3e5fc EFLAGS: 00000296 ORIG_RAX: 00000000000001aa
RAX: ffffffffffffffda RBX: 0000000000000004 RCX: 0000000000002a71
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
 </TASK>
NMI backtrace for cpu 1
CPU: 1 PID: 30954 Comm: syz-executor.0 Not tainted 5.16.0-rc3-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:106
 nmi_cpu_backtrace.cold+0x47/0x144 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x1b3/0x230 lib/nmi_backtrace.c:62
 trigger_single_cpu_backtrace include/linux/nmi.h:164 [inline]
 rcu_dump_cpu_stacks+0x25e/0x3f0 kernel/rcu/tree_stall.h:343
 print_cpu_stall kernel/rcu/tree_stall.h:627 [inline]
 check_cpu_stall kernel/rcu/tree_stall.h:711 [inline]
 rcu_pending kernel/rcu/tree.c:3878 [inline]
 rcu_sched_clock_irq.cold+0x9d/0x746 kernel/rcu/tree.c:2597
 update_process_times+0x16d/0x200 kernel/time/timer.c:1785
 tick_sched_handle+0x9b/0x180 kernel/time/tick-sched.c:226
 tick_sched_timer+0x1b0/0x2d0 kernel/time/tick-sched.c:1421
 __run_hrtimer kernel/time/hrtimer.c:1685 [inline]
 __hrtimer_run_queues+0x1c0/0xe50 kernel/time/hrtimer.c:1749
 hrtimer_interrupt+0x31c/0x790 kernel/time/hrtimer.c:1811
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1086 [inline]
 __sysvec_apic_timer_interrupt+0x146/0x530 arch/x86/kernel/apic/apic.c:1103
 sysvec_apic_timer_interrupt+0x40/0xc0 arch/x86/kernel/apic/apic.c:1097
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:pv_wait_head_or_lock kernel/locking/qspinlock_paravirt.h:434 [inline]
RIP: 0010:__pv_queued_spin_lock_slowpath+0x3ba/0xb40 kernel/locking/qspinlock.c:508
Code: eb c6 45 01 01 41 bc 00 80 00 00 48 c1 e9 03 83 e3 07 41 be 01 00 00 00 48 b8 00 00 00 00 00 fc ff df 4c 8d 2c 01 eb 0c f3 90 <41> 83 ec 01 0f 84 72 04 00 00 41 0f b6 45 00 38 d8 7f 08 84 c0 0f
RSP: 0018:ffffc90000dc0b70 EFLAGS: 00000202
RAX: 0000000000000001 RBX: 0000000000000000 RCX: 1ffff110083add31
RDX: 0000000000000001 RSI: dffffc0000000000 RDI: ffffffff8b5628a0
RBP: ffff888041d6e988 R08: 0000000000000001 R09: ffff888041d6e98b
R10: ffffed10083add31 R11: 0000000000000000 R12: 0000000000006d70
R13: ffffed10083add31 R14: 0000000000000001 R15: ffff8880b9d3a880
 pv_queued_spin_lock_slowpath arch/x86/include/asm/paravirt.h:591 [inline]
 queued_spin_lock_slowpath arch/x86/include/asm/qspinlock.h:51 [inline]
 queued_spin_lock include/asm-generic/qspinlock.h:85 [inline]
 do_raw_spin_lock+0x200/0x2b0 kernel/locking/spinlock_debug.c:115
 spin_lock include/linux/spinlock.h:349 [inline]
 sctp_generate_heartbeat_event+0xa1/0x490 net/sctp/sm_sideeffect.c:371
 call_timer_fn+0x1a5/0x6b0 kernel/time/timer.c:1421
 expire_timers kernel/time/timer.c:1466 [inline]
 __run_timers.part.0+0x675/0xa20 kernel/time/timer.c:1734
 __run_timers kernel/time/timer.c:1715 [inline]
 run_timer_softirq+0xb3/0x1d0 kernel/time/timer.c:1747
 __do_softirq+0x29b/0x9c2 kernel/softirq.c:558
 invoke_softirq kernel/softirq.c:432 [inline]
 __irq_exit_rcu+0x123/0x180 kernel/softirq.c:636
 irq_exit_rcu+0x5/0x20 kernel/softirq.c:648
 sysvec_apic_timer_interrupt+0x93/0xc0 arch/x86/kernel/apic/apic.c:1097
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:preempt_schedule_irq+0x49/0x90 kernel/sched/core.c:6668
Code: 55 53 65 48 8b 1c 25 40 70 02 00 48 89 dd 48 c1 ed 03 48 01 c5 bf 01 00 00 00 e8 b2 2d 0a f8 e8 ed 0e 38 f8 fb bf 01 00 00 00 <e8> 12 a9 ff ff 9c 58 fa f6 c4 02 75 27 bf 01 00 00 00 e8 e0 1c 0a
RSP: 0018:ffffc90005846fa8 EFLAGS: 00000206
RAX: 000000000000227b RBX: ffff888022459d00 RCX: 1ffffffff1ff7cee
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000001
RBP: ffffed100448b3a0 R08: 0000000000000001 R09: ffffffff8ff72acf
R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
 irqentry_exit+0x31/0x80 kernel/entry/common.c:425
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:check_kcov_mode+0x7/0x40 kernel/kcov.c:166
Code: 00 e9 59 fe ff ff 48 8b 7c 24 08 e8 33 c6 46 00 e9 61 fd ff ff cc cc cc cc cc cc cc cc cc cc cc cc cc cc 65 8b 05 e9 c9 8a 7e <89> c2 81 e2 00 01 00 00 a9 00 01 ff 00 74 10 31 c0 85 d2 74 15 8b
RSP: 0018:ffffc90005847070 EFLAGS: 00000202
RAX: 0000000080000000 RBX: 0000000000000005 RCX: 0000000000000007
RDX: 0000000000000000 RSI: ffff888022459d00 RDI: 0000000000000003
RBP: 0000000000000007 R08: ffffffff8abd27c0 R09: ffffffff87e46538
R10: 000000000000000b R11: 0000000000000000 R12: ffff888022459d00
R13: ffffc900058471d0 R14: ffff888075054ec0 R15: ffffc90005847190
 write_comp_data kernel/kcov.c:221 [inline]
 __sanitizer_cov_trace_switch+0x63/0xf0 kernel/kcov.c:323
 ipv6_get_saddr_eval net/ipv6/addrconf.c:1537 [inline]
 ipv6_get_saddr_eval+0x68/0x1030 net/ipv6/addrconf.c:1516
 __ipv6_dev_get_saddr+0x1f4/0x5c0 net/ipv6/addrconf.c:1691
 ipv6_dev_get_saddr+0x824/0xbc0 net/ipv6/addrconf.c:1826
 ip6_route_get_saddr include/net/ip6_route.h:145 [inline]
 ip6_dst_lookup_tail+0xa6e/0x1620 net/ipv6/ip6_output.c:1075
 ip6_dst_lookup_flow+0x8c/0x1d0 net/ipv6/ip6_output.c:1200
 sctp_v6_get_dst+0x66c/0x1d60 net/sctp/ipv6.c:327
 sctp_transport_route+0x125/0x350 net/sctp/transport.c:458
 sctp_packet_config+0xa01/0xe50 net/sctp/output.c:103
 sctp_outq_select_transport+0x1e4/0x740 net/sctp/outqueue.c:863
 sctp_outq_flush_ctrl.constprop.0+0x234/0xab0 net/sctp/outqueue.c:897
 sctp_outq_flush net/sctp/outqueue.c:1202 [inline]
 sctp_outq_uncork+0x10b/0x200 net/sctp/outqueue.c:758
 sctp_cmd_interpreter net/sctp/sm_sideeffect.c:1812 [inline]
 sctp_side_effects net/sctp/sm_sideeffect.c:1195 [inline]
 sctp_do_sm+0x745/0x4ed0 net/sctp/sm_sideeffect.c:1166
 sctp_primitive_ASSOCIATE+0x98/0xc0 net/sctp/primitive.c:73
 __sctp_connect+0x9e2/0xc30 net/sctp/socket.c:1231
 __sctp_setsockopt_connectx+0x10d/0x180 net/sctp/socket.c:1333
 sctp_setsockopt_connectx_old net/sctp/socket.c:1344 [inline]
 sctp_setsockopt+0x3e4d/0xa890 net/sctp/socket.c:4606
 __sys_setsockopt+0x2db/0x610 net/socket.c:2176
 __do_sys_setsockopt net/socket.c:2187 [inline]
 __se_sys_setsockopt net/socket.c:2184 [inline]
 __ia32_sys_setsockopt+0xb9/0x150 net/socket.c:2184
 do_syscall_32_irqs_on arch/x86/entry/common.c:112 [inline]
 __do_fast_syscall_32+0x65/0xf0 arch/x86/entry/common.c:178
 do_fast_syscall_32+0x2f/0x70 arch/x86/entry/common.c:203
 entry_SYSENTER_compat_after_hwframe+0x4d/0x5c
RIP: 0023:0xf6f4a549
Code: 03 74 c0 01 10 05 03 74 b8 01 10 06 03 74 b4 01 10 07 03 74 b0 01 10 08 03 74 d8 01 00 00 00 00 00 51 52 55 89 e5 0f 34 cd 80 <5d> 5a 59 c3 90 90 90 90 8d b4 26 00 00 00 00 8d b4 26 00 00 00 00
RSP: 002b:00000000f5f445fc EFLAGS: 00000296 ORIG_RAX: 000000000000016e
RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 0000000000000084
RDX: 000000000000006b RSI: 000000002055bfe4 RDI: 000000000000001c
RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
 </TASK>
----------------
Code disassembly (best guess), 1 bytes skipped:
   0:	03 0f                	add    (%rdi),%ecx
   2:	b6 04                	mov    $0x4,%dh
   4:	02 84 c0 74 08 3c 03 	add    0x33c0874(%rax,%rax,8),%al
   b:	0f 8e 65 0a 00 00    	jle    0xa76
  11:	41 83 45 10 01       	addl   $0x1,0x10(%r13)
  16:	48 b8 00 00 00 00 00 	movabs $0xdffffc0000000000,%rax
  1d:	fc ff df
  20:	48 8b 14 24          	mov    (%rsp),%rdx
  24:	41 83 45 10 01       	addl   $0x1,0x10(%r13)
* 29:	48 c1 ea 03          	shr    $0x3,%rdx <-- trapping instruction
  2d:	80 3c 02 00          	cmpb   $0x0,(%rdx,%rax,1)
  31:	0f 85 31 0a 00 00    	jne    0xa68
  37:	48 89 e8             	mov    %rbp,%rax
  3a:	48 c1 e0 07          	shl    $0x7,%rax
  3e:	49                   	rex.WB

Crashes (1):
Manager Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Title
ci-upstream-kasan-gce-386 2021/12/04 10:05 upstream 12119cfa1052 a617004c .config log report info INFO: rcu detected stall in sctp_generate_heartbeat_event
* Struck through repros no longer work on HEAD.