syzbot


INFO: rcu detected stall in rtnl_newlink (4)

Status: internal: reported C repro on 2024/02/29 13:10
Subsystems: fs batman
[Documentation on labels]
Fix commit: b1f532a3b1e6 batman-adv: Avoid infinite loop trying to resize local TT
Patched on: [ci-qemu-upstream ci-qemu-upstream-386 ci-qemu2-arm32 ci-qemu2-arm64 ci-qemu2-arm64-compat ci-qemu2-arm64-mte ci-upstream-bpf-kasan-gce ci-upstream-bpf-next-kasan-gce ci-upstream-gce-arm64 ci-upstream-gce-leak ci-upstream-kasan-badwrites-root ci-upstream-kasan-gce ci-upstream-kasan-gce-386 ci-upstream-kasan-gce-root ci-upstream-kasan-gce-selinux-root ci-upstream-kasan-gce-smack-root ci-upstream-kmsan-gce-386-root ci-upstream-kmsan-gce-root ci-upstream-linux-next-kasan-gce-root ci-upstream-net-kasan-gce ci-upstream-net-this-kasan-gce ci2-upstream-fs ci2-upstream-kcsan-gce ci2-upstream-net-next-test-gce ci2-upstream-usb], missing on: [ci-qemu2-riscv64]
First crash: 64d, last: 4d12h
Cause bisection: failed (error log, bisect log)
  
Similar bugs (9)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-6.1 INFO: rcu detected stall in rtnl_newlink (2) 2 27d 51d 0/3 upstream: reported on 2024/03/13 06:43
upstream INFO: rcu detected stall in rtnl_newlink batman C done inconclusive 201 577d 1379d 0/26 auto-obsoleted due to no activity on 2023/02/01 16:38
upstream INFO: rcu detected stall in rtnl_newlink (3) batman C error done 3 252d 302d 0/26 auto-obsoleted due to no activity on 2023/12/03 18:15
linux-6.1 INFO: rcu detected stall in rtnl_newlink 1 206d 206d 0/3 auto-obsoleted due to no activity on 2024/01/17 21:04
linux-5.15 BUG: soft lockup in rtnl_newlink origin:lts-only C inconclusive 61 19d 420d 0/3 upstream: reported C repro on 2023/03/09 19:16
linux-4.19 INFO: rcu detected stall in rtnl_newlink C error 7 507d 1379d 0/1 upstream: reported C repro on 2020/07/24 01:06
upstream INFO: rcu detected stall in rtnl_newlink (2) net 1 446d 446d 0/26 auto-obsoleted due to no activity on 2023/05/13 18:46
android-5-15 BUG: soft lockup in rtnl_newlink 2 15d 15d 0/2 premoderation: reported on 2024/04/18 11:40
linux-4.14 BUG: soft lockup in rtnl_newlink 1 828d 828d 0/1 auto-closed as invalid on 2022/05/26 06:33
Last patch testing requests (1)
Created Duration User Patch Repo Result
2024/04/10 08:38 22m edumazet@google.com net-next error OK

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	0-...!: (1 GPs behind) idle=b074/1/0x4000000000000000 softirq=6367/6368 fqs=3
rcu: 	(detected by 1, t=10502 jiffies, g=7601, q=267 ncpus=2)
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 5097 Comm: syz-executor408 Not tainted 6.8.0-rc5-syzkaller-01654-g4ac828960a60 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/25/2024
RIP: 0010:rb_next+0x6/0xf0 lib/rbtree.c:493
Code: 3c 38 00 74 e3 48 89 df e8 17 cc 92 f6 eb d9 0f 1f 44 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 41 57 <41> 56 41 54 53 49 89 fc 49 bf 00 00 00 00 00 fc ff df 48 89 f8 48
RSP: 0018:ffffc90000007d18 EFLAGS: 00000046
RAX: ffffffff8b639651 RBX: ffff8880b942bad8 RCX: ffff8880244b3b80
RDX: 0000000000010001 RSI: ffff88802f1bf340 RDI: ffff88802f1bf340
RBP: dffffc0000000000 R08: ffffffff8b639618 R09: 1ffffffff1f0bcb5
R10: dffffc0000000000 R11: fffffbfff1f0bcb6 R12: 1ffff11005e37e68
R13: ffff88802f1bf340 R14: ffff8880b942bad0 R15: 1ffff1101728575b
FS:  000055555604a380(0000) GS:ffff8880b9400000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd235db62d0 CR3: 0000000024518000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 rb_erase_cached include/linux/rbtree.h:124 [inline]
 timerqueue_del+0x89/0x100 lib/timerqueue.c:57
 __remove_hrtimer kernel/time/hrtimer.c:1120 [inline]
 __run_hrtimer kernel/time/hrtimer.c:1669 [inline]
 __hrtimer_run_queues+0x3da/0xd00 kernel/time/hrtimer.c:1753
 hrtimer_interrupt+0x396/0x990 kernel/time/hrtimer.c:1815
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1065 [inline]
 __sysvec_apic_timer_interrupt+0x107/0x3a0 arch/x86/kernel/apic/apic.c:1082
 sysvec_apic_timer_interrupt+0x92/0xb0 arch/x86/kernel/apic/apic.c:1076
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:649
RIP: 0010:fib_sync_up+0x74b/0x7d0
Code: d9 80 e1 07 80 c1 03 38 c1 0f 8c 79 ff ff ff 48 89 df e8 88 49 0e f8 e9 6c ff ff ff e8 3e d0 ab f7 31 c0 eb 09 e8 35 d0 ab f7 <8b> 44 24 14 48 83 c4 60 5b 41 5c 41 5d 41 5e 41 5f 5d c3 cc cc cc
RSP: 0018:ffffc90003d1eaa8 EFLAGS: 00000293
RAX: ffffffff89e79d3b RBX: 0000000000000000 RCX: ffff8880244b3b80
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffff88801e801000 R08: ffffffff89e797cc R09: 1ffff1100ec7782c
R10: dffffc0000000000 R11: ffffed100ec7782d R12: ffff88801e8010f8
R13: dffffc0000000000 R14: ffff88801e801068 R15: ffff88801e8010f8
 fib_netdev_event+0x438/0x490 net/ipv4/fib_frontend.c:1494
 notifier_call_chain+0x18f/0x3b0 kernel/notifier.c:93
 __dev_notify_flags+0x207/0x400
 dev_change_flags+0xf0/0x1a0 net/core/dev.c:8771
 do_setlink+0xccd/0x41f0 net/core/rtnetlink.c:2887
 __rtnl_newlink net/core/rtnetlink.c:3683 [inline]
 rtnl_newlink+0x180b/0x20a0 net/core/rtnetlink.c:3730
 rtnetlink_rcv_msg+0x89b/0x10d0 net/core/rtnetlink.c:6599
 netlink_rcv_skb+0x1e3/0x430 net/netlink/af_netlink.c:2547
 netlink_unicast_kernel net/netlink/af_netlink.c:1335 [inline]
 netlink_unicast+0x7ea/0x980 net/netlink/af_netlink.c:1361
 netlink_sendmsg+0x8e0/0xcb0 net/netlink/af_netlink.c:1902
 sock_sendmsg_nosec net/socket.c:730 [inline]
 __sock_sendmsg+0x221/0x270 net/socket.c:745
 __sys_sendto+0x3a4/0x4f0 net/socket.c:2191
 __do_sys_sendto net/socket.c:2203 [inline]
 __se_sys_sendto net/socket.c:2199 [inline]
 __x64_sys_sendto+0xde/0x100 net/socket.c:2199
 do_syscall_64+0xf9/0x240
 entry_SYSCALL_64_after_hwframe+0x6f/0x77
RIP: 0033:0x7fd235d37263
Code: 64 89 02 48 c7 c0 ff ff ff ff eb b7 66 2e 0f 1f 84 00 00 00 00 00 90 80 3d 41 ae 07 00 00 41 89 ca 74 14 b8 2c 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 75 c3 0f 1f 40 00 55 48 83 ec 30 44 89 4c 24
RSP: 002b:00007ffea4d8d368 EFLAGS: 00000202 ORIG_RAX: 000000000000002c
RAX: ffffffffffffffda RBX: 00007fd235db4420 RCX: 00007fd235d37263
RDX: 000000000000002c RSI: 00007fd235db4470 RDI: 0000000000000003
RBP: 0000000000000003 R08: 00007ffea4d8d384 R09: 000000000000000c
R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000001
R13: 0000000000000000 R14: 00007fd235db4470 R15: 0000000000000000
 </TASK>
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 3.590 msecs
rcu: rcu_preempt kthread starved for 10496 jiffies! g7601 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:24656 pid:17    tgid:17    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5400 [inline]
 __schedule+0x17d1/0x49f0 kernel/sched/core.c:6727
 __schedule_loop kernel/sched/core.c:6802 [inline]
 schedule+0x149/0x260 kernel/sched/core.c:6817
 schedule_timeout+0x1bd/0x310 kernel/time/timer.c:2183
 rcu_gp_fqs_loop+0x2df/0x1330 kernel/rcu/tree.c:1663
 rcu_gp_kthread+0xa7/0x3b0 kernel/rcu/tree.c:1862
 kthread+0x2ef/0x390 kernel/kthread.c:388
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1b/0x30 arch/x86/entry/entry_64.S:242
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
CPU: 1 PID: 2415 Comm: kworker/u4:7 Not tainted 6.8.0-rc5-syzkaller-01654-g4ac828960a60 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/25/2024
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:csd_lock_wait kernel/smp.c:311 [inline]
RIP: 0010:smp_call_function_many_cond+0x1850/0x2960 kernel/smp.c:855
Code: 45 8b 65 00 44 89 e6 83 e6 01 31 ff e8 99 d4 0b 00 41 83 e4 01 49 bc 00 00 00 00 00 fc ff df 75 07 e8 44 d0 0b 00 eb 38 f3 90 <42> 0f b6 04 23 84 c0 75 11 41 f7 45 00 01 00 00 00 74 1e e8 28 d0
RSP: 0018:ffffc9000a0bf720 EFLAGS: 00000293
RAX: ffffffff81879d48 RBX: 1ffff110172888a1 RCX: ffff88802a375940
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffffc9000a0bf920 R08: ffffffff81879d17 R09: 1ffffffff2594284
R10: dffffc0000000000 R11: fffffbfff2594285 R12: dffffc0000000000
R13: ffff8880b9444508 R14: ffff8880b953da80 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff8880b9500000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055555604aca8 CR3: 000000000df32000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 </IRQ>
 <TASK>
 on_each_cpu_cond_mask+0x3f/0x80 kernel/smp.c:1023
 on_each_cpu include/linux/smp.h:71 [inline]
 text_poke_sync arch/x86/kernel/alternative.c:2087 [inline]
 text_poke_bp_batch+0x352/0xb30 arch/x86/kernel/alternative.c:2297
 text_poke_flush arch/x86/kernel/alternative.c:2488 [inline]
 text_poke_finish+0x30/0x50 arch/x86/kernel/alternative.c:2495
 arch_jump_label_transform_apply+0x1c/0x30 arch/x86/kernel/jump_label.c:146
 static_key_enable_cpuslocked+0x136/0x260 kernel/jump_label.c:205
 static_key_enable+0x1a/0x20 kernel/jump_label.c:218
 toggle_allocation_gate+0xb5/0x250 mm/kfence/core.c:826
 process_one_work kernel/workqueue.c:2633 [inline]
 process_scheduled_works+0x913/0x1420 kernel/workqueue.c:2706
 worker_thread+0xa5f/0x1000 kernel/workqueue.c:2787
 kthread+0x2ef/0x390 kernel/kthread.c:388
 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1b/0x30 arch/x86/entry/entry_64.S:242
 </TASK>

Crashes (6):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/02/29 13:09 net-next 4ac828960a60 352ab904 .config console log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: rcu detected stall in rtnl_newlink
2024/04/29 06:21 upstream e67572cd2204 07b455f9 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: rcu detected stall in rtnl_newlink
2024/04/21 07:28 upstream 977b1ef51866 af24b050 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: rcu detected stall in rtnl_newlink
2024/04/13 19:17 net f99c5f563c17 c8349e48 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: rcu detected stall in rtnl_newlink
2024/03/05 17:25 net-next 885c36e59f46 f39a7eed .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: rcu detected stall in rtnl_newlink
2024/03/19 03:37 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 707081b61156 baa80228 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 BUG: soft lockup in rtnl_newlink
* Struck through repros no longer work on HEAD.