syzbot


BUG: soft lockup in smp_call_function

Status: upstream: reported C repro on 2023/03/19 12:10
Bug presence: origin:upstream
Labels: missing-backport
[Documentation on labels]
Reported-by: syzbot+f61e8f4a149eefab5677@syzkaller.appspotmail.com
First crash: 573d, last: 5d07h
Bug presence (3)
Date Name Commit Repro Result
2023/08/06 linux-5.15.y (ToT) 38d4ca22a528 C [report] no output from test machine
2023/06/22 upstream (ToT) dad9774deaf1 C [report] BUG: workqueue lockup
2023/08/06 upstream (ToT) f0ab9f34e59e C Didn't crash
Similar bugs (10)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
android-6-1 BUG: soft lockup in smp_call_function missing-backport origin:upstream C inconclusive 74 4d13h 194d 0/2 upstream: reported C repro on 2024/04/02 06:05
linux-4.19 BUG: soft lockup in smp_call_function C error 2 775d 904d 0/1 upstream: reported C repro on 2022/04/23 05:26
upstream BUG: soft lockup in smp_call_function kernel C done 1229 1d08h 1553d 0/28 upstream: reported C repro on 2020/07/12 23:02
linux-6.1 BUG: soft lockup in smp_call_function 77 34d 575d 0/3 upstream: reported on 2023/03/18 00:36
android-54 BUG: soft lockup in smp_call_function 1 1072d 1072d 0/2 auto-closed as invalid on 2022/03/05 18:04
linux-4.14 BUG: soft lockup in smp_call_function C 1 601d 904d 0/1 upstream: reported C repro on 2022/04/23 04:43
android-5-15 BUG: soft lockup in smp_call_function origin:upstream C error inconclusive 937 16d 587d 0/2 upstream: reported C repro on 2023/03/05 21:20
android-54 BUG: soft lockup in smp_call_function (2) C 23 31d 259d 0/2 upstream: reported C repro on 2024/01/28 03:58
android-5-10 BUG: soft lockup in smp_call_function C error 79 21d 448d 0/2 upstream: reported C repro on 2023/07/22 22:59
upstream INFO: rcu detected stall in smp_call_function perf C 107 1281d 1537d 20/28 fixed on 2021/04/13 06:26
Fix bisection attempts (5)
Created Duration User Patch Repo Result
2024/05/16 21:05 1m fix candidate upstream error job log
2024/03/26 04:25 3m fix candidate upstream error job log
2024/01/21 18:14 0m fix candidate upstream error job log
2023/12/17 05:33 0m fix candidate upstream error job log
2023/11/12 14:10 1m fix candidate upstream error job log

Sample crash report:
watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [syz-executor247:4047]
Modules linked in:
irq event stamp: 9390
hardirqs last  enabled at (9389): [<ffff800011952c58>] __exit_to_kernel_mode arch/arm64/kernel/entry-common.c:81 [inline]
hardirqs last  enabled at (9389): [<ffff800011952c58>] exit_to_kernel_mode+0x100/0x178 arch/arm64/kernel/entry-common.c:91
hardirqs last disabled at (9390): [<ffff800011952ea0>] enter_el1_irq_or_nmi+0x10/0x1c arch/arm64/kernel/entry-common.c:227
softirqs last  enabled at (4938): [<ffff800008020ccc>] softirq_handle_end kernel/softirq.c:401 [inline]
softirqs last  enabled at (4938): [<ffff800008020ccc>] __do_softirq+0xb5c/0xe20 kernel/softirq.c:587
softirqs last disabled at (4753): [<ffff8000081b573c>] do_softirq_own_stack include/asm-generic/softirq_stack.h:10 [inline]
softirqs last disabled at (4753): [<ffff8000081b573c>] invoke_softirq kernel/softirq.c:439 [inline]
softirqs last disabled at (4753): [<ffff8000081b573c>] __irq_exit_rcu+0x28c/0x534 kernel/softirq.c:636
CPU: 1 PID: 4047 Comm: syz-executor247 Not tainted 5.15.118-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/25/2023
pstate: 80400005 (Nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : csd_lock_wait kernel/smp.c:440 [inline]
pc : smp_call_function_many_cond+0xa50/0x1088 kernel/smp.c:969
lr : csd_lock_wait kernel/smp.c:440 [inline]
lr : smp_call_function_many_cond+0xa68/0x1088 kernel/smp.c:969
sp : ffff80001d047900
x29: ffff80001d047950 x28: ffff0001b482cd48 x27: dfff800000000000
x26: 0000000000000011 x25: ffff0001b482cd40 x24: ffff8000148f0ff8
x23: 1fffe000369059a8 x22: ffff0001b48131c8 x21: ffff800014adda40
x20: 0000000000000000 x19: ffff0001b482cd48 x18: ffff80001d047620
x17: 0000000000000000 x16: ffff8000119547c4 x15: 0000000000011af3
x14: 1ffff0000291e06a x13: ffffffffffffffff x12: 0000000000000000
x11: ff8080000840ef98 x10: 0000000000000000 x9 : 0000000000000000
x8 : 0000000000000011 x7 : 0000000000000000 x6 : 0000000000000000
x5 : 0000000000000080 x4 : 0000000000000000 x3 : ffff80000840ef8c
x2 : 0000000000000000 x1 : 0000000000000004 x0 : 0000000000000001
Call trace:
 __cmpwait_case_32 arch/arm64/include/asm/cmpxchg.h:252 [inline]
 __cmpwait arch/arm64/include/asm/cmpxchg.h:278 [inline]
 csd_lock_wait kernel/smp.c:440 [inline]
 smp_call_function_many_cond+0xa50/0x1088 kernel/smp.c:969
 smp_call_function_many+0x40/0x54 kernel/smp.c:993
 clock_was_set+0x69c/0x784 kernel/time/hrtimer.c:974
 do_settimeofday64+0x6e0/0x8f0 kernel/time/timekeeping.c:1328
 do_sys_settimeofday64+0x1b8/0x23c kernel/time/time.c:195
 posix_clock_realtime_set+0x24/0x34 kernel/time/posix-timers.c:185
 __do_sys_clock_settime kernel/time/posix-timers.c:1092 [inline]
 __se_sys_clock_settime kernel/time/posix-timers.c:1080 [inline]
 __arm64_sys_clock_settime+0x1fc/0x248 kernel/time/posix-timers.c:1080
 __invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
 invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:52
 el0_svc_common+0x138/0x258 arch/arm64/kernel/syscall.c:142
 do_el0_svc+0x58/0x14c arch/arm64/kernel/syscall.c:181
 el0_svc+0x7c/0x1f0 arch/arm64/kernel/entry-common.c:596
 el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:614
 el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584

Crashes (167):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2023/06/22 05:03 linux-5.15.y f67653019430 79782afc .config console log report syz C [disk image] [vmlinux] [kernel image] [mounted in repro] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/01/14 06:19 linux-5.15.y 26c690eff0a5 551587c1 .config console log report syz C [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in smp_call_function
2024/01/13 23:54 linux-5.15.y 26c690eff0a5 551587c1 .config console log report syz [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in smp_call_function
2024/10/08 00:49 linux-5.15.y 3a5928702e71 d7906eff .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/10/06 17:04 linux-5.15.y 3a5928702e71 d7906eff .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/10/06 13:35 linux-5.15.y 3a5928702e71 d7906eff .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/10/05 22:17 linux-5.15.y 3a5928702e71 d7906eff .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/10/05 14:59 linux-5.15.y 3a5928702e71 d7906eff .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/10/02 15:51 linux-5.15.y 3a5928702e71 02f9582a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/09/30 20:52 linux-5.15.y 3a5928702e71 bbd4e0a4 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/09/29 14:46 linux-5.15.y 3a5928702e71 ba29ff75 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/09/27 22:42 linux-5.15.y 3a5928702e71 440b26ec .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/09/27 07:24 linux-5.15.y 3a5928702e71 9314348a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/09/23 19:09 linux-5.15.y 3a5928702e71 89298aad .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/09/23 15:00 linux-5.15.y 3a5928702e71 89298aad .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/09/22 02:37 linux-5.15.y 3a5928702e71 6f888b75 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/09/20 10:56 linux-5.15.y 3a5928702e71 6f888b75 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/09/17 12:59 linux-5.15.y 3a5928702e71 c673ca06 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/09/15 14:06 linux-5.15.y 3a5928702e71 08d8a733 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/09/09 03:24 linux-5.15.y 14e468424d3e 9750182a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/09/05 06:32 linux-5.15.y 14e468424d3e dfbe2ed4 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/31 00:53 linux-5.15.y fa93fa65db6e 1eda0d14 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/30 23:31 linux-5.15.y fa93fa65db6e 1eda0d14 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/30 11:45 linux-5.15.y fa93fa65db6e ee2602b8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/28 00:16 linux-5.15.y fa93fa65db6e 6c853ff9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/27 18:21 linux-5.15.y fa93fa65db6e 6c853ff9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/25 23:03 linux-5.15.y fa93fa65db6e d7d32352 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/20 09:01 linux-5.15.y fa93fa65db6e 9f0ab3fb .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/18 18:18 linux-5.15.y 7e89efd3ae1c dbc93b08 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/13 13:01 linux-5.15.y 7e89efd3ae1c 7b0f4b46 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/13 00:53 linux-5.15.y 7e89efd3ae1c 7b0f4b46 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/11 01:37 linux-5.15.y 7e89efd3ae1c 6f4edef4 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/09 13:57 linux-5.15.y 7e89efd3ae1c a83d9288 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/08 21:07 linux-5.15.y 7e89efd3ae1c 61405512 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/06 20:06 linux-5.15.y 7e89efd3ae1c 1ef9fe42 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/06 16:55 linux-5.15.y 7e89efd3ae1c 1ef9fe42 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/06 15:51 linux-5.15.y 7e89efd3ae1c 1ef9fe42 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/05 15:19 linux-5.15.y 7e89efd3ae1c e35c337f .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/05 09:55 linux-5.15.y 7e89efd3ae1c e35c337f .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/04 13:16 linux-5.15.y 7e89efd3ae1c 1786a2a8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/04 08:12 linux-5.15.y 7e89efd3ae1c 1786a2a8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/03 11:03 linux-5.15.y 7e89efd3ae1c 1786a2a8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/03 03:20 linux-5.15.y 7e89efd3ae1c 1786a2a8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/08/02 11:22 linux-5.15.y 7e89efd3ae1c 1e9c4cf3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/07/30 03:46 linux-5.15.y 7e89efd3ae1c 5187fc86 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/07/29 19:51 linux-5.15.y 7e89efd3ae1c 5187fc86 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/07/28 14:30 linux-5.15.y 7e89efd3ae1c 46eb10b7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/07/28 10:08 linux-5.15.y 7e89efd3ae1c 46eb10b7 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/07/24 14:09 linux-5.15.y 7c6d66f0266f 57b2edb1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2024/07/24 01:17 linux-5.15.y 7c6d66f0266f e50e8da5 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: soft lockup in smp_call_function
2023/03/19 12:10 linux-5.15.y 8020ae3c051d 7939252e .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan BUG: soft lockup in smp_call_function
2024/09/02 22:34 linux-5.15.y fa93fa65db6e 8045124c .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in smp_call_function
2024/09/02 18:50 linux-5.15.y fa93fa65db6e 1eda0d14 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in smp_call_function
* Struck through repros no longer work on HEAD.