bisecting fixing commit since f40ddce88593482919761f74910f42f4b84c004b
building syzkaller on 98682e5e2aefc9aad61354f4f3ac93be96002a2a
testing commit f40ddce88593482919761f74910f42f4b84c004b
compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.1
kernel signature: a20fdda6284e1de4805076b3f724e927513bee772a53c4f27dc22d1ef7831f86
run #0: crashed: INFO: rcu detected stall in mac80211_hwsim_beacon
run #1: crashed: INFO: rcu detected stall in mac80211_hwsim_beacon
run #2: crashed: INFO: rcu detected stall in mac80211_hwsim_beacon
run #3: crashed: INFO: rcu detected stall in mac80211_hwsim_beacon
run #4: crashed: INFO: rcu detected stall in mac80211_hwsim_beacon
run #5: crashed: INFO: rcu detected stall in mac80211_hwsim_beacon
run #6: crashed: INFO: rcu detected stall in mac80211_hwsim_beacon
run #7: crashed: INFO: rcu detected stall in mac80211_hwsim_beacon
run #8: crashed: INFO: rcu detected stall in mac80211_hwsim_beacon
run #9: crashed: INFO: rcu detected stall in mac80211_hwsim_beacon
run #10: crashed: INFO: rcu detected stall in sys_fcntl
run #11: crashed: BUG: soft lockup in ieee80211_tasklet_handler
run #12: crashed: INFO: rcu detected stall in corrupted
run #13: crashed: INFO: rcu detected stall in mac80211_hwsim_beacon
run #14: OK
run #15: crashed: WARNING in corrupted
run #16: crashed: INFO: rcu detected stall in mac80211_hwsim_beacon
run #17: crashed: INFO: rcu detected stall in mac80211_hwsim_beacon
run #18: crashed: INFO: rcu detected stall in mac80211_hwsim_beacon
run #19: OK
testing current HEAD f0fddcec6b6254b4b3611388786bbafb703ad257
testing commit f0fddcec6b6254b4b3611388786bbafb703ad257
compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.1
kernel signature: 96fb63afc4ff26dc0766334bade2fce9c520e50a3476f1a532e6dd16bd58521d
run #0: crashed: INFO: rcu detected stall in mac80211_hwsim_beacon
run #1: OK
run #2: OK
run #3: crashed: INFO: rcu detected stall in mac80211_hwsim_beacon
run #4: crashed: INFO: rcu detected stall in perf_fasync
run #5: crashed: INFO: rcu detected stall in ieee80211_tasklet_handler
run #6: crashed: INFO: rcu detected stall in mac80211_hwsim_beacon
run #7: crashed: INFO: rcu detected stall in mac80211_hwsim_beacon
run #8: crashed: INFO: rcu detected stall in mac80211_hwsim_beacon
run #9: OK
revisions tested: 2, total time: 36m31.279151877s (build: 13m5.485394963s, test: 22m48.548248991s)
the crash still happens on HEAD
commit msg: Merge tag 'for-5.14-rc2-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux
crash: INFO: rcu detected stall in mac80211_hwsim_beacon
rcu: INFO: rcu_preempt self-detected stall on CPU
rcu: 0-...!: (7958 ticks this GP) idle=9f2/1/0x4000000000000000 softirq=87398/87399 fqs=27
(t=10502 jiffies g=60325 q=4704)
rcu: rcu_preempt kthread starved for 10426 jiffies! g60325 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt state:R running task stack:29304 pid: 13 ppid: 2 flags:0x00004000
Call Trace:
context_switch kernel/sched/core.c:4683 [inline]
__schedule+0x902/0x26a0 kernel/sched/core.c:5940
schedule+0xd3/0x270 kernel/sched/core.c:6019
schedule_timeout+0x11d/0x250 kernel/time/timer.c:1878
rcu_gp_fqs_loop kernel/rcu/tree.c:1996 [inline]
rcu_gp_kthread+0xd34/0x1980 kernel/rcu/tree.c:2169
kthread+0x38b/0x460 kernel/kthread.c:319
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
rcu: Stack dump where RCU GP kthread last ran:
NMI backtrace for cpu 0
CPU: 0 PID: 27946 Comm: syz-executor502 Not tainted 5.14.0-rc2-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x45/0x59 lib/dump_stack.c:105
nmi_cpu_backtrace.cold+0x30/0x99 lib/nmi_backtrace.c:105
nmi_trigger_cpumask_backtrace+0x11f/0x170 lib/nmi_backtrace.c:62
trigger_single_cpu_backtrace include/linux/nmi.h:164 [inline]
rcu_check_gp_kthread_starvation.cold+0x1d1/0x1d6 kernel/rcu/tree_stall.h:479
print_cpu_stall kernel/rcu/tree_stall.h:623 [inline]
check_cpu_stall kernel/rcu/tree_stall.h:700 [inline]
rcu_pending kernel/rcu/tree.c:3922 [inline]
rcu_sched_clock_irq.cold+0x9a/0x747 kernel/rcu/tree.c:2641
update_process_times+0x13b/0x1c0 kernel/time/timer.c:1782
tick_sched_handle+0x6f/0x130 kernel/time/tick-sched.c:226
tick_sched_timer+0x132/0x210 kernel/time/tick-sched.c:1421
__run_hrtimer kernel/time/hrtimer.c:1537 [inline]
__hrtimer_run_queues+0x18a/0xb00 kernel/time/hrtimer.c:1601
hrtimer_interrupt+0x2f9/0x920 kernel/time/hrtimer.c:1663
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1089 [inline]
__sysvec_apic_timer_interrupt+0x146/0x530 arch/x86/kernel/apic/apic.c:1106
sysvec_apic_timer_interrupt+0x40/0xc0 arch/x86/kernel/apic/apic.c:1100
asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:rate_control_get_rate+0x5c/0x5c0 net/mac80211/rate.c:900
Code: 00 0f 85 b6 04 00 00 49 8b 9d 48 06 00 00 48 b8 00 00 00 00 00 fc ff df 48 8d bb 58 12 00 00 48 89 fa 48 c1 ea 03 80 3c 02 00 <0f> 85 83 04 00 00 48 8b 83 58 12 00 00 48 89 04 24 48 8d 45 18 48
RSP: 0018:ffffc90000007b38 EFLAGS: 00000246
RAX: dffffc0000000000 RBX: ffff888041b38d20 RCX: 1ffff110083670e5
RDX: 1ffff110083673ef RSI: 0000000000000000 RDI: ffff888041b39f78
RBP: ffffc90000007bf0 R08: 0000000000000001 R09: ffffc90000007bf0
R10: fffff52000000f85 R11: ffff888041aa8848 R12: 0000000000000000
R13: ffff888017e88c00 R14: ffff888017e89248 R15: ffff888041aa8780
__ieee80211_beacon_get+0x915/0x1840 net/mac80211/tx.c:5141
ieee80211_beacon_get_tim+0x7e/0x6e0 net/mac80211/tx.c:5168
ieee80211_beacon_get include/net/mac80211.h:4962 [inline]
mac80211_hwsim_beacon_tx+0xc2/0x7b0 drivers/net/wireless/mac80211_hwsim.c:1808
__iterate_interfaces+0x103/0x360 net/mac80211/util.c:793
ieee80211_iterate_active_interfaces_atomic+0x53/0xf0 net/mac80211/util.c:829
mac80211_hwsim_beacon+0xb7/0x160 drivers/net/wireless/mac80211_hwsim.c:1861
__run_hrtimer kernel/time/hrtimer.c:1537 [inline]
__hrtimer_run_queues+0x4d7/0xb00 kernel/time/hrtimer.c:1601
hrtimer_run_softirq+0x176/0x340 kernel/time/hrtimer.c:1618
__do_softirq+0x29b/0x9c2 kernel/softirq.c:558
invoke_softirq kernel/softirq.c:432 [inline]
__irq_exit_rcu+0x16e/0x1c0 kernel/softirq.c:636
irq_exit_rcu+0x5/0x20 kernel/softirq.c:648
sysvec_apic_timer_interrupt+0x93/0xc0 arch/x86/kernel/apic/apic.c:1100
asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:preempt_schedule_irq+0x49/0x90 kernel/sched/core.c:6328
Code: 55 53 65 48 8b 1c 25 00 f0 01 00 48 89 dd 48 c1 ed 03 48 01 c5 bf 01 00 00 00 e8 72 bd ee f8 e8 5d 7e 16 f9 fb bf 01 00 00 00 32 ce ff ff 9c 58 fa f6 c4 02 75 27 bf 01 00 00 00 e8 80 ac ee
RSP: 0018:ffffc900028efb58 EFLAGS: 00000206
RAX: 0000000000007a33 RBX: ffff888019e68000 RCX: 1ffffffff18b57e9
RDX: 0000000000000000 RSI: ffffffff88ab21a0 RDI: 0000000000000001
RBP: ffffed10033cd000 R08: 0000000000000001 R09: 0000000000000001
R10: ffffffff8dd75017 R11: 0000000000000001 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
irqentry_exit+0x31/0x80 kernel/entry/common.c:427
asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:stackdepot_memcmp lib/stackdepot.c:195 [inline]
RIP: 0010:find_stack lib/stackdepot.c:211 [inline]
RIP: 0010:stack_depot_save+0x13e/0x4e0 lib/stackdepot.c:281
Code: 37 4d 85 f6 75 11 e9 94 00 00 00 4d 8b 36 4d 85 f6 0f 84 88 00 00 00 41 39 5e 08 75 ee 45 3b 66 0c 75 e8 31 c0 49 8b 4c c6 18 <48> 39 4c c5 00 75 da 48 83 c0 01 49 39 c5 75 eb 48 8b 7c 24 28 48
RSP: 0018:ffffc900028efc20 EFLAGS: 00000246
RAX: 0000000000000000 RBX: 00000000680793d2 RCX: ffffffff81a601db
RDX: 00000000000793d2 RSI: 0000000000000800 RDI: 00000000bff834c0
RBP: ffffc900028efc90 R08: 00000000e50d1ab5 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000078088 R12: 0000000000000008
R13: 0000000000000008 R14: ffff8880228a23f0 R15: ffff88823b3c9e90
kasan_save_stack+0x32/0x40 mm/kasan/common.c:40
kasan_record_aux_stack+0xc5/0xf0 mm/kasan/generic.c:348
task_work_add+0x36/0x130 kernel/task_work.c:38
fput_many.part.0+0x87/0x110 fs/file_table.c:341
filp_close+0xed/0x120 fs/open.c:1318
__do_sys_close fs/open.c:1331 [inline]
__se_sys_close fs/open.c:1329 [inline]
__x64_sys_close+0x28/0x60 fs/open.c:1329
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x403343
Code: c7 c2 c0 ff ff ff f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 64 8b 04 25 18 00 00 00 85 c0 75 14 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 45 c3 0f 1f 40 00 48 83 ec 18 89 7c 24 0c e8
RSP: 002b:00007ffe2e0dedb8 EFLAGS: 00000246 ORIG_RAX: 0000000000000003
RAX: ffffffffffffffda RBX: 0000000000000004 RCX: 0000000000403343
RDX: 0000000000042000 RSI: 0000000000000004 RDI: 0000000000000003
RBP: 0000000000000000 R08: 0000000000000031 R09: 0000000000000031
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000068ad9
R13: 00007ffe2e0dee10 R14: 00007ffe2e0dee00 R15: 00007ffe2e0dedd4
NMI backtrace for cpu 0
CPU: 0 PID: 27946 Comm: syz-executor502 Not tainted 5.14.0-rc2-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x45/0x59 lib/dump_stack.c:105
nmi_cpu_backtrace.cold+0x30/0x99 lib/nmi_backtrace.c:105
nmi_trigger_cpumask_backtrace+0x11f/0x170 lib/nmi_backtrace.c:62
trigger_single_cpu_backtrace include/linux/nmi.h:164 [inline]
rcu_dump_cpu_stacks+0x25e/0x3f0 kernel/rcu/tree_stall.h:342
print_cpu_stall kernel/rcu/tree_stall.h:625 [inline]
check_cpu_stall kernel/rcu/tree_stall.h:700 [inline]
rcu_pending kernel/rcu/tree.c:3922 [inline]
rcu_sched_clock_irq.cold+0x9f/0x747 kernel/rcu/tree.c:2641
update_process_times+0x13b/0x1c0 kernel/time/timer.c:1782
tick_sched_handle+0x6f/0x130 kernel/time/tick-sched.c:226
tick_sched_timer+0x132/0x210 kernel/time/tick-sched.c:1421
__run_hrtimer kernel/time/hrtimer.c:1537 [inline]
__hrtimer_run_queues+0x18a/0xb00 kernel/time/hrtimer.c:1601
hrtimer_interrupt+0x2f9/0x920 kernel/time/hrtimer.c:1663
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1089 [inline]
__sysvec_apic_timer_interrupt+0x146/0x530 arch/x86/kernel/apic/apic.c:1106
sysvec_apic_timer_interrupt+0x40/0xc0 arch/x86/kernel/apic/apic.c:1100
asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:rate_control_get_rate+0x5c/0x5c0 net/mac80211/rate.c:900
Code: 00 0f 85 b6 04 00 00 49 8b 9d 48 06 00 00 48 b8 00 00 00 00 00 fc ff df 48 8d bb 58 12 00 00 48 89 fa 48 c1 ea 03 80 3c 02 00 <0f> 85 83 04 00 00 48 8b 83 58 12 00 00 48 89 04 24 48 8d 45 18 48
RSP: 0018:ffffc90000007b38 EFLAGS: 00000246
RAX: dffffc0000000000 RBX: ffff888041b38d20 RCX: 1ffff110083670e5
RDX: 1ffff110083673ef RSI: 0000000000000000 RDI: ffff888041b39f78
RBP: ffffc90000007bf0 R08: 0000000000000001 R09: ffffc90000007bf0
R10: fffff52000000f85 R11: ffff888041aa8848 R12: 0000000000000000
R13: ffff888017e88c00 R14: ffff888017e89248 R15: ffff888041aa8780
__ieee80211_beacon_get+0x915/0x1840 net/mac80211/tx.c:5141
ieee80211_beacon_get_tim+0x7e/0x6e0 net/mac80211/tx.c:5168
ieee80211_beacon_get include/net/mac80211.h:4962 [inline]
mac80211_hwsim_beacon_tx+0xc2/0x7b0 drivers/net/wireless/mac80211_hwsim.c:1808
__iterate_interfaces+0x103/0x360 net/mac80211/util.c:793
ieee80211_iterate_active_interfaces_atomic+0x53/0xf0 net/mac80211/util.c:829
mac80211_hwsim_beacon+0xb7/0x160 drivers/net/wireless/mac80211_hwsim.c:1861
__run_hrtimer kernel/time/hrtimer.c:1537 [inline]
__hrtimer_run_queues+0x4d7/0xb00 kernel/time/hrtimer.c:1601
hrtimer_run_softirq+0x176/0x340 kernel/time/hrtimer.c:1618
__do_softirq+0x29b/0x9c2 kernel/softirq.c:558
invoke_softirq kernel/softirq.c:432 [inline]
__irq_exit_rcu+0x16e/0x1c0 kernel/softirq.c:636
irq_exit_rcu+0x5/0x20 kernel/softirq.c:648
sysvec_apic_timer_interrupt+0x93/0xc0 arch/x86/kernel/apic/apic.c:1100
asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:preempt_schedule_irq+0x49/0x90 kernel/sched/core.c:6328
Code: 55 53 65 48 8b 1c 25 00 f0 01 00 48 89 dd 48 c1 ed 03 48 01 c5 bf 01 00 00 00 e8 72 bd ee f8 e8 5d 7e 16 f9 fb bf 01 00 00 00 32 ce ff ff 9c 58 fa f6 c4 02 75 27 bf 01 00 00 00 e8 80 ac ee
RSP: 0018:ffffc900028efb58 EFLAGS: 00000206
RAX: 0000000000007a33 RBX: ffff888019e68000 RCX: 1ffffffff18b57e9
RDX: 0000000000000000 RSI: ffffffff88ab21a0 RDI: 0000000000000001
RBP: ffffed10033cd000 R08: 0000000000000001 R09: 0000000000000001
R10: ffffffff8dd75017 R11: 0000000000000001 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
irqentry_exit+0x31/0x80 kernel/entry/common.c:427
asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:stackdepot_memcmp lib/stackdepot.c:195 [inline]
RIP: 0010:find_stack lib/stackdepot.c:211 [inline]
RIP: 0010:stack_depot_save+0x13e/0x4e0 lib/stackdepot.c:281
Code: 37 4d 85 f6 75 11 e9 94 00 00 00 4d 8b 36 4d 85 f6 0f 84 88 00 00 00 41 39 5e 08 75 ee 45 3b 66 0c 75 e8 31 c0 49 8b 4c c6 18 <48> 39 4c c5 00 75 da 48 83 c0 01 49 39 c5 75 eb 48 8b 7c 24 28 48
RSP: 0018:ffffc900028efc20 EFLAGS: 00000246
RAX: 0000000000000000 RBX: 00000000680793d2 RCX: ffffffff81a601db
RDX: 00000000000793d2 RSI: 0000000000000800 RDI: 00000000bff834c0
RBP: ffffc900028efc90 R08: 00000000e50d1ab5 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000078088 R12: 0000000000000008
R13: 0000000000000008 R14: ffff8880228a23f0 R15: ffff88823b3c9e90
kasan_save_stack+0x32/0x40 mm/kasan/common.c:40
kasan_record_aux_stack+0xc5/0xf0 mm/kasan/generic.c:348
task_work_add+0x36/0x130 kernel/task_work.c:38
fput_many.part.0+0x87/0x110 fs/file_table.c:341
filp_close+0xed/0x120 fs/open.c:1318
__do_sys_close fs/open.c:1331 [inline]
__se_sys_close fs/open.c:1329 [inline]
__x64_sys_close+0x28/0x60 fs/open.c:1329
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x403343
Code: c7 c2 c0 ff ff ff f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 64 8b 04 25 18 00 00 00 85 c0 75 14 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 45 c3 0f 1f 40 00 48 83 ec 18 89 7c 24 0c e8
RSP: 002b:00007ffe2e0dedb8 EFLAGS: 00000246 ORIG_RAX: 0000000000000003
RAX: ffffffffffffffda RBX: 0000000000000004 RCX: 0000000000403343
RDX: 0000000000042000 RSI: 0000000000000004 RDI: 0000000000000003
RBP: 0000000000000000 R08: 0000000000000031 R09: 0000000000000031
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000068ad9
R13: 00007ffe2e0dee10 R14: 00007ffe2e0dee00 R15: 00007ffe2e0dedd4