INFO: task hung in perf_event_free_task

Status: upstream: reported C repro on 2023/09/12 07:14
Bug presence: origin:upstream
Labels: missing-backport
[Documentation on labels]
Reported-by: syzbot+037dd0ce48b026949f99@syzkaller.appspotmail.com
First crash: 1073d, last: 969d
Fix bisection: failed (error log, bisect log)
  
▶ ▼ Bug presence (3)
Date Name Commit Repro Result
2024/06/08 linux-6.1.y (ToT) 88690811da69 C [report] INFO: rcu detected stall in corrupted
2023/09/12 upstream (ToT) 0bb80ecc33a8 C [report] INFO: task hung in perf_event_free_task
2024/06/08 upstream (ToT) 061d1af7b030 C Didn't crash
▶ ▼ Similar bugs (8)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.19 INFO: task hung in perf_event_free_task (3) 1 3 1570d 1716d 0/1 auto-obsoleted due to no activity on 2022/08/31 14:29
linux-5.15 INFO: task hung in perf_event_free_task 1 syz, C error 1 1023d 1023d 0/3 auto-obsoleted due to no activity on 2024/02/09 23:23
linux-4.19 INFO: task hung in perf_event_free_task (2) 1 2 1926d 1998d 0/1 auto-closed as invalid on 2021/09/09 09:13
linux-4.19 INFO: task hung in perf_event_free_task (4) 1 4 1269d 1438d 0/1 upstream: reported on 2022/09/12 12:14
upstream INFO: task hung in perf_event_free_task perf 1 syz, C done error 40 1075d 2584d 0/29 auto-obsoleted due to no activity on 2023/12/19 10:14
linux-5.15 INFO: task hung in perf_event_free_task (2) 1 1 909d 909d 0/3 auto-obsoleted due to no activity on 2024/06/02 07:34
linux-4.19 INFO: task hung in perf_event_free_task 1 5 2141d 2536d 0/1 auto-closed as invalid on 2021/02/06 21:15
upstream INFO: task can't die in perf_event_free_task perf 1 syz unreliable 6 1400d 2100d 0/29 auto-obsoleted due to no activity on 2023/04/21 23:56
▶ ▼ Fix bisection attempts (3)
Created Duration User Patch Repo Result
2024/06/18 13:16 9h28m fix candidate upstream OK (1) job log
2023/12/04 07:30 18m bisect fix linux-6.1.y error job log
2023/10/17 01:20 2h29m bisect fix linux-6.1.y OK (0) job log log

Sample crash report:
INFO: task syz-executor318:3667 blocked for more than 143 seconds.
      Not tainted 6.1.60-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor318 state:D stack:26056 pid:3667  ppid:3581   flags:0x00004006
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5241 [inline]
 __schedule+0x142d/0x4550 kernel/sched/core.c:6554
 schedule+0xbf/0x180 kernel/sched/core.c:6630
 perf_event_free_task+0x553/0x680 kernel/events/core.c:13100
 copy_process+0x2c7d/0x4060 kernel/fork.c:2552
 kernel_clone+0x222/0x920 kernel/fork.c:2682
 __do_sys_clone kernel/fork.c:2823 [inline]
 __se_sys_clone kernel/fork.c:2807 [inline]
 __x64_sys_clone+0x231/0x280 kernel/fork.c:2807
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7f2390a2dcb9
RSP: 002b:00007f23909e0148 EFLAGS: 00000246 ORIG_RAX: 0000000000000038
RAX: ffffffffffffffda RBX: 00007f2390ab61a8 RCX: 00007f2390a2dcb9
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 00000000d816e480
RBP: 00007f2390ab61a0 R08: 0000000000000000 R09: 00007f2390ab61a8
R10: 0000000000000000 R11: 0000000000000246 R12: 00007f2390ab61ac
R13: 0000000000000016 R14: 00007ffd1b1da6b0 R15: 00007ffd1b1da798
 </TASK>

Showing all locks held in the system:
1 lock held by rcu_tasks_kthre/12:
 #0: ffffffff8d12a230 (rcu_tasks.tasks_gp_mutex){+.+.}-{3:3}, at: rcu_tasks_one_gp+0x29/0xe30 kernel/rcu/tasks.h:516
1 lock held by rcu_tasks_trace/13:
 #0: ffffffff8d12aa30 (rcu_tasks_trace.tasks_gp_mutex){+.+.}-{3:3}, at: rcu_tasks_one_gp+0x29/0xe30 kernel/rcu/tasks.h:516
1 lock held by khungtaskd/28:
 #0: ffffffff8d12a060 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:306 [inline]
 #0: ffffffff8d12a060 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:747 [inline]
 #0: ffffffff8d12a060 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x51/0x290 kernel/locking/lockdep.c:6493
2 locks held by getty/3304:
 #0: ffff888028a3a098 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x21/0x70 drivers/tty/tty_ldisc.c:244
 #1: ffffc900031262f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0x6a7/0x1db0 drivers/tty/n_tty.c:2188

=============================================

NMI backtrace for cpu 1
CPU: 1 PID: 28 Comm: khungtaskd Not tainted 6.1.60-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/09/2023
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e3/0x2cb lib/dump_stack.c:106
 nmi_cpu_backtrace+0x4e1/0x560 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x1b0/0x3f0 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:148 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:220 [inline]
 watchdog+0xf88/0xfd0 kernel/hung_task.c:377
 kthread+0x28d/0x320 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306
 </TASK>
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 33 Comm: kworker/u4:2 Not tainted 6.1.60-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/09/2023
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:preempt_count arch/x86/include/asm/preempt.h:27 [inline]
RIP: 0010:preempt_count_sub+0x37/0x160 kernel/sched/core.c:5731
Code: c1 e8 03 49 be 00 00 00 00 00 fc ff df 42 0f b6 04 30 84 c0 0f 85 c0 00 00 00 83 3d b2 14 6a 10 00 75 22 65 8b 05 91 42 a4 7e <89> c1 81 e1 ff ff ff 7f 39 d9 7c 23 81 fb fe 00 00 00 77 07 0f b6
RSP: 0018:ffffc90000aa77b8 EFLAGS: 00000246
RAX: 0000000080000001 RBX: 0000000000000001 RCX: ffffffff91c84103
RDX: 0000000000000000 RSI: 0000000000000004 RDI: 0000000000000001
RBP: ffffc90000aa79b0 R08: dffffc0000000000 R09: ffffed100248cc0d
R10: 0000000000000000 R11: dffffc0000000001 R12: dffffc0000000000
R13: 1ffff92000154f0c R14: dffffc0000000000 R15: 0000000000000246
FS:  0000000000000000(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005580ed25f600 CR3: 000000000ce8e000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <TASK>
 __raw_spin_unlock include/linux/spinlock_api_smp.h:143 [inline]
 _raw_spin_unlock+0x24/0x40 kernel/locking/spinlock.c:186
 spin_unlock include/linux/spinlock.h:390 [inline]
 __text_poke+0x81a/0x9a0 arch/x86/kernel/alternative.c:1211
 text_poke arch/x86/kernel/alternative.c:1235 [inline]
 text_poke_bp_batch+0x153/0x940 arch/x86/kernel/alternative.c:1526
 text_poke_flush arch/x86/kernel/alternative.c:1720 [inline]
 text_poke_finish+0x16/0x30 arch/x86/kernel/alternative.c:1727
 arch_jump_label_transform_apply+0x13/0x20 arch/x86/kernel/jump_label.c:146
 static_key_disable_cpuslocked+0xca/0x1b0 kernel/jump_label.c:207
 static_key_disable+0x16/0x20 kernel/jump_label.c:215
 toggle_allocation_gate+0x3e0/0x480 mm/kfence/core.c:818
 process_one_work+0x8a9/0x11d0 kernel/workqueue.c:2292
 worker_thread+0xa47/0x1200 kernel/workqueue.c:2439
 kthread+0x28d/0x320 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306
 </TASK>

Crashes (3):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2023/11/02 02:20 linux-6.1.y 32c9cdbe383c 69904c9f .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: task hung in perf_event_free_task
2023/09/12 07:14 linux-6.1.y 59b13c2b647e 59da8366 .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: task hung in perf_event_free_task
2023/12/25 23:36 linux-6.1.y 4aa6747d9352 fb427a07 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-perf INFO: task hung in perf_event_free_task
* Struck through repros no longer work on HEAD.