syzbot


INFO: rcu detected stall in addrconf_dad_work (5)

Status: upstream: reported C repro on 2020/09/07 15:59
Subsystems: net
[Documentation on labels]
Reported-by: syzbot+251463bfa779ca087ad1@syzkaller.appspotmail.com
First crash: 1539d, last: 18d
Cause bisection: introduced by (bisect log) :
commit 5a781ccbd19e4664babcbe4b4ead7aa2b9283d22
Author: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Date: Sat Sep 29 00:59:43 2018 +0000

  tc: Add support for configuring the taprio scheduler

Crash: no output from test machine (log)
Repro: C syz .config
  
Fix bisection the fix commit could be any of (bisect log):
  fc3abb53250a Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/hid/hid
  9e9fb7655ed5 Merge tag 'net-next-5.15' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next
  
Discussions (1)
Title Replies (including bot) Last reply
INFO: rcu detected stall in addrconf_dad_work (5) 0 (1) 2020/09/07 15:59
Similar bugs (12)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in addrconf_dad_work (4) cgroups mm 8 1778d 1779d 0/28 closed as invalid on 2020/01/09 08:13
upstream INFO: rcu detected stall in addrconf_dad_work (3) kernel 6 1779d 1779d 0/28 closed as invalid on 2020/01/08 05:23
linux-4.14 INFO: rcu detected stall in addrconf_dad_work C done 18 1895d 1902d 1/1 fixed on 2019/12/06 10:33
upstream INFO: rcu detected stall in addrconf_dad_work (2) kernel 15 1813d 1815d 0/28 closed as invalid on 2019/12/04 14:14
upstream INFO: rcu detected stall in addrconf_dad_work C done 126 1893d 1898d 13/28 fixed on 2019/10/09 10:54
linux-4.19 INFO: rcu detected stall in addrconf_dad_work (2) C done 1 1798d 1798d 1/1 fixed on 2020/01/19 15:05
linux-5.15 INFO: rcu detected stall in addrconf_dad_work origin:lts-only syz done 1 11d 81d 0/3 upstream: reported syz repro on 2024/09/01 00:22
linux-4.19 INFO: rcu detected stall in addrconf_dad_work C done 19 1890d 1901d 1/1 fixed on 2019/12/07 19:18
linux-5.15 BUG: soft lockup in addrconf_dad_work 1 492d 492d 0/3 auto-obsoleted due to no activity on 2023/10/25 16:01
linux-4.19 BUG: soft lockup in addrconf_dad_work C error 55 670d 1011d 0/1 upstream: reported C repro on 2022/02/13 10:05
android-5-15 BUG: soft lockup in addrconf_dad_work 3 152d 207d 0/2 auto-obsoleted due to no activity on 2024/09/19 17:30
upstream BUG: soft lockup in addrconf_dad_work net C done 1 1897d 1897d 13/28 fixed on 2019/10/09 10:54
Last patch testing requests (10)
Created Duration User Patch Repo Result
2024/10/10 16:35 21m retest repro git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci report log
2024/09/19 19:54 17m retest repro linux-next report log
2024/08/30 00:39 33m retest repro git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci OK log
2024/08/01 11:56 29m retest repro git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci report log
2024/07/11 12:55 22m retest repro upstream OK log
2024/07/11 08:49 22m retest repro upstream OK log
2024/07/11 08:49 23m retest repro upstream OK log
2024/07/11 08:17 7m retest repro linux-next error
2024/06/20 20:06 3h44m retest repro git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci report log
2024/05/23 08:57 1h55m retest repro git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci report log
Fix bisection attempts (12)
Created Duration User Patch Repo Result
2021/09/01 13:04 16m bisect fix upstream OK (2) job log
2021/08/02 04:03 22m bisect fix upstream OK (0) job log log
2021/07/01 06:56 22m bisect fix upstream OK (0) job log log
2021/06/01 06:34 22m bisect fix upstream OK (0) job log log
2021/05/02 06:00 25m bisect fix upstream OK (0) job log log
2021/04/01 23:43 23m bisect fix upstream OK (0) job log log
2021/03/01 07:32 24m bisect fix upstream OK (0) job log log
2021/02/06 16:31 0m bisect fix upstream error job log
2021/01/07 16:04 26m bisect fix upstream OK (0) job log log
2020/12/07 15:34 24m bisect fix upstream OK (0) job log log
2020/11/07 13:38 25m bisect fix upstream OK (0) job log log
2020/10/08 08:53 26m bisect fix upstream OK (0) job log log

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	1-...!: (1 GPs behind) idle=daac/1/0x4000000000000000 softirq=9124/9125 fqs=1
rcu: 	(detected by 0, t=10506 jiffies, g=9577, q=103 ncpus=2)
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 2522 Comm: kworker/1:2 Not tainted 6.5.0-rc7-next-20230825-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/26/2023
Workqueue: ipv6_addrconf addrconf_dad_work
RIP: 0010:trace_lock_acquire include/trace/events/lock.h:24 [inline]
RIP: 0010:lock_acquire+0x464/0x510 kernel/locking/lockdep.c:5724
Code: 44 24 14 8b 54 24 10 e8 ba 15 fe ff 5e 5f 65 ff 0d 69 20 9b 7e 0f 85 37 fc ff ff e8 b6 e5 97 ff e9 2d fc ff ff e8 6c c4 07 00 <84> c0 0f 85 59 fc ff ff 0f 0b e9 52 fc ff ff 0f 0b e9 22 fc ff ff
RSP: 0018:ffffc900001f0d40 EFLAGS: 00000082
RAX: 0000000000000001 RBX: 1ffff9200003e1aa RCX: ffffffff816898c7
RDX: 0000000000000000 RSI: ffffffff8ae92820 RDI: ffffffff8c5cea48
RBP: 0000000000000001 R08: 0000000000000000 R09: fffffbfff1d9c252
R10: ffffffff8ece1297 R11: ffffc900001f0ff8 R12: 0000000000000001
R13: 0000000000000000 R14: ffff8880b992b958 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000020000600 CR3: 0000000073c1e000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 __raw_spin_lock_irq include/linux/spinlock_api_smp.h:119 [inline]
 _raw_spin_lock_irq+0x36/0x50 kernel/locking/spinlock.c:170
 __run_hrtimer kernel/time/hrtimer.c:1692 [inline]
 __hrtimer_run_queues+0x2bd/0xc10 kernel/time/hrtimer.c:1752
 hrtimer_interrupt+0x31b/0x800 kernel/time/hrtimer.c:1814
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1063 [inline]
 __sysvec_apic_timer_interrupt+0x105/0x3f0 arch/x86/kernel/apic/apic.c:1080
 sysvec_apic_timer_interrupt+0x8e/0xc0 arch/x86/kernel/apic/apic.c:1074
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:645
RIP: 0010:check_kcov_mode kernel/kcov.c:173 [inline]
RIP: 0010:__sanitizer_cov_trace_pc+0x11/0x70 kernel/kcov.c:207
Code: b0 01 00 00 e8 b0 ff ff ff 31 c0 c3 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 f3 0f 1e fa 65 8b 05 4d ea 7b 7e 89 c1 48 8b 34 24 <81> e1 00 01 00 00 65 48 8b 14 25 c0 bc 03 00 a9 00 01 ff 00 74 0e
RSP: 0018:ffffc9000a78ef90 EFLAGS: 00000202
RAX: 0000000080000201 RBX: ffffc9000a78f060 RCX: 0000000080000201
RDX: ffff8880258abb80 RSI: ffffffff813a46c5 RDI: 0000000000000005
RBP: ffffc9000a78fec8 R08: 0000000000000005 R09: 0000000000000000
R10: 0000000000000001 R11: 3b0000ff01000000 R12: ffffc9000a78f068
R13: ffffc9000a78f070 R14: ffffc9000a788000 R15: 0000000000000001
 on_stack arch/x86/include/asm/stacktrace.h:59 [inline]
 stack_access_ok+0x1c5/0x270 arch/x86/kernel/unwind_orc.c:393
 deref_stack_reg arch/x86/kernel/unwind_orc.c:403 [inline]
 unwind_next_frame+0x1a98/0x2390 arch/x86/kernel/unwind_orc.c:648
 arch_stack_walk+0xfa/0x170 arch/x86/kernel/stacktrace.c:25
 stack_trace_save+0x96/0xd0 kernel/stacktrace.c:122
 kasan_save_stack+0x33/0x50 mm/kasan/common.c:45
 __kasan_record_aux_stack+0xbc/0xd0 mm/kasan/generic.c:492
 __call_rcu_common.constprop.0+0x9a/0x790 kernel/rcu/tree.c:2653
 call_rcu_hurry include/linux/rcupdate.h:117 [inline]
 dst_release net/core/dst.c:167 [inline]
 dst_release+0x1b5/0x1e0 net/core/dst.c:164
 refdst_drop include/net/dst.h:270 [inline]
 skb_dst_drop include/net/dst.h:282 [inline]
 __dev_queue_xmit+0x2043/0x3d60 net/core/dev.c:4292
 dev_queue_xmit include/linux/netdevice.h:3082 [inline]
 neigh_resolve_output net/core/neighbour.c:1552 [inline]
 neigh_resolve_output+0x58c/0x900 net/core/neighbour.c:1532
 neigh_output include/net/neighbour.h:542 [inline]
 ip6_finish_output2+0x610/0x1b20 net/ipv6/ip6_output.c:135
 __ip6_finish_output net/ipv6/ip6_output.c:196 [inline]
 ip6_finish_output+0x485/0x1250 net/ipv6/ip6_output.c:207
 NF_HOOK_COND include/linux/netfilter.h:293 [inline]
 ip6_output+0x23a/0x880 net/ipv6/ip6_output.c:228
 dst_output include/net/dst.h:458 [inline]
 NF_HOOK.constprop.0+0xfd/0x540 include/linux/netfilter.h:304
 ndisc_send_skb+0x9f1/0x1430 net/ipv6/ndisc.c:509
 ndisc_send_ns+0xc7/0x140 net/ipv6/ndisc.c:667
 addrconf_dad_work+0xc7f/0x13e0 net/ipv6/addrconf.c:4213
 process_one_work+0x887/0x15d0 kernel/workqueue.c:2630
 process_scheduled_works kernel/workqueue.c:2703 [inline]
 worker_thread+0x8bb/0x1290 kernel/workqueue.c:2784
 kthread+0x33a/0x430 kernel/kthread.c:388
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:304
 </TASK>
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 5.074 msecs
rcu: rcu_preempt kthread starved for 10500 jiffies! g9577 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:28848 pid:17    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5382 [inline]
 __schedule+0xee1/0x59f0 kernel/sched/core.c:6695
 schedule+0xe7/0x1b0 kernel/sched/core.c:6771
 schedule_timeout+0x157/0x2c0 kernel/time/timer.c:2167
 rcu_gp_fqs_loop+0x1ec/0xa50 kernel/rcu/tree.c:1613
 rcu_gp_kthread+0x249/0x380 kernel/rcu/tree.c:1812
 kthread+0x33a/0x430 kernel/kthread.c:388
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:304
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
CPU: 0 PID: 12 Comm: kworker/u4:1 Not tainted 6.5.0-rc7-next-20230825-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/26/2023
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:csd_lock_wait kernel/smp.c:300 [inline]
RIP: 0010:smp_call_function_many_cond+0x4d6/0x1570 kernel/smp.c:844
Code: 0b 00 85 ed 74 4d 48 b8 00 00 00 00 00 fc ff df 4d 89 fc 4c 89 fd 49 c1 ec 03 83 e5 07 49 01 c4 83 c5 03 e8 dc 8c 0b 00 f3 90 <41> 0f b6 04 24 40 38 c5 7c 08 84 c0 0f 85 3c 0e 00 00 8b 43 08 31
RSP: 0018:ffffc90000117928 EFLAGS: 00000293
RAX: 0000000000000000 RBX: ffff8880b9941a60 RCX: 0000000000000000
RDX: ffff88801626d940 RSI: ffffffff817c4594 RDI: 0000000000000005
RBP: 0000000000000003 R08: 0000000000000005 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000000 R12: ffffed101732834d
R13: 0000000000000001 R14: ffff8880b983d8c0 R15: ffff8880b9941a68
FS:  0000000000000000(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000559a182f8000 CR3: 000000000c976000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 </IRQ>
 <TASK>
 on_each_cpu_cond_mask+0x40/0x90 kernel/smp.c:1012
 on_each_cpu include/linux/smp.h:71 [inline]
 text_poke_sync arch/x86/kernel/alternative.c:1998 [inline]
 text_poke_bp_batch+0x2ce/0x960 arch/x86/kernel/alternative.c:2208
 text_poke_flush arch/x86/kernel/alternative.c:2399 [inline]
 text_poke_flush arch/x86/kernel/alternative.c:2396 [inline]
 text_poke_finish+0x30/0x40 arch/x86/kernel/alternative.c:2406
 arch_jump_label_transform_apply+0x1c/0x30 arch/x86/kernel/jump_label.c:146
 jump_label_update+0x32e/0x410 kernel/jump_label.c:829
 static_key_enable_cpuslocked+0x1b5/0x270 kernel/jump_label.c:205
 static_key_enable+0x1a/0x20 kernel/jump_label.c:218
 toggle_allocation_gate mm/kfence/core.c:830 [inline]
 toggle_allocation_gate+0xf4/0x250 mm/kfence/core.c:822
 process_one_work+0x887/0x15d0 kernel/workqueue.c:2630
 process_scheduled_works kernel/workqueue.c:2703 [inline]
 worker_thread+0x8bb/0x1290 kernel/workqueue.c:2784
 kthread+0x33a/0x430 kernel/kthread.c:388
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:304
 </TASK>

Crashes (13):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2023/08/25 18:28 linux-next 626932085009 03d9c195 .config console log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in addrconf_dad_work
2024/02/29 02:34 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 381f163531d8 352ab904 .config console log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 BUG: soft lockup in addrconf_dad_work
2024/04/15 12:53 upstream fe46a7dd189e c8349e48 .config console log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in addrconf_dad_work
2023/09/04 01:04 upstream 6e32dfcccfcc 696ea0d2 .config console log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in addrconf_dad_work
2022/02/13 10:24 upstream b81b1829e7e3 8b9ca619 .config console log report syz C ci-upstream-kasan-gce INFO: rcu detected stall in addrconf_dad_work
2022/02/13 10:23 net-next-old 5a8fb33e5305 8b9ca619 .config console log report syz C ci-upstream-net-kasan-gce INFO: rcu detected stall in addrconf_dad_work
2020/09/03 15:50 upstream fc3abb53250a abf9ba4f .config console log report syz C ci-upstream-kasan-gce-root
2024/05/30 03:06 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci fda5695d692c 34889ee3 .config console log report syz / log C [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 BUG: soft lockup in addrconf_dad_work
2023/07/26 07:54 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci e40939bbfc68 6756545c .config console log report syz [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 BUG: soft lockup in addrconf_dad_work
2023/07/19 17:17 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci e40939bbfc68 022df2bb .config console log report syz [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 BUG: soft lockup in addrconf_dad_work
2024/11/03 00:59 upstream 11066801dd4b f00eed24 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in addrconf_dad_work
2024/06/21 09:51 upstream 50736169ecc8 dac2aa43 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in addrconf_dad_work
2024/06/01 05:42 upstream d8ec19857b09 3113787f .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in addrconf_dad_work
* Struck through repros no longer work on HEAD.