syzbot

 sign-in | mailing list | source | docs
🐞 Open [909] ≡ Subsystems 🐞 Fixed [4781] 🐞 Invalid [11289] ⬇ Missing Backports [32] 📈 Kernel Health 📈 Bug Lifetimes 📈 Fuzzing 📈 Crashes 💬 Send us feedback

INFO: rcu detected stall in addrconf_dad_work (5)

Status: upstream: reported C repro on 2020/09/07 15:59
Subsystems: net
[Documentation on labels]
Reported-by: syzbot+251463bfa779ca087ad1@syzkaller.appspotmail.com
First crash: 1123d, last: 28d
Cause bisection: introduced by (bisect log) :
commit 5a781ccbd19e4664babcbe4b4ead7aa2b9283d22
Author: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Date: Sat Sep 29 00:59:43 2018 +0000

  tc: Add support for configuring the taprio scheduler

Crash: no output from test machine (log)
Repro: C syz .config
  
Fix bisection the fix commit could be any of (bisect log):
  fc3abb53250a Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/hid/hid
  9e9fb7655ed5 Merge tag 'net-next-5.15' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next
   
▶ ▼ Discussions (1)
Title Replies (including bot) Last reply
INFO: rcu detected stall in addrconf_dad_work (5) 0 (1) 2020/09/07 15:59
▶ ▼ Similar bugs (10)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in addrconf_dad_work (4) cgroups mm 8 1362d 1363d 0/25 closed as invalid on 2020/01/09 08:13
upstream INFO: rcu detected stall in addrconf_dad_work (3) kernel 6 1363d 1363d 0/25 closed as invalid on 2020/01/08 05:23
linux-4.14 INFO: rcu detected stall in addrconf_dad_work C done 18 1479d 1486d 1/1 fixed on 2019/12/06 10:33
upstream INFO: rcu detected stall in addrconf_dad_work (2) kernel 15 1398d 1399d 0/25 closed as invalid on 2019/12/04 14:14
upstream INFO: rcu detected stall in addrconf_dad_work C done 126 1477d 1482d 14/25 fixed on 2019/10/09 10:54
linux-4.19 INFO: rcu detected stall in addrconf_dad_work (2) C done 1 1382d 1382d 1/1 fixed on 2020/01/19 15:05
linux-4.19 INFO: rcu detected stall in addrconf_dad_work C done 19 1474d 1485d 1/1 fixed on 2019/12/07 19:18
linux-5.15 BUG: soft lockup in addrconf_dad_work 1 76d 76d 0/3 upstream: reported on 2023/07/17 16:03
linux-4.19 BUG: soft lockup in addrconf_dad_work C error 55 254d 596d 0/1 upstream: reported C repro on 2022/02/13 10:05
upstream BUG: soft lockup in addrconf_dad_work net C done 1 1481d 1481d 14/25 fixed on 2019/10/09 10:54
▶ ▼ Last patch testing requests (10)
Created Duration User Patch Repo Result
2023/09/18 03:17 15m retest repro linux-next report log
2023/09/18 02:57 16m retest repro upstream report log
2023/09/18 02:09 19m retest repro upstream report log
2023/09/18 01:06 23m retest repro git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci OK log
2023/09/18 01:06 29m retest repro git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci OK log
2023/07/01 15:22 14m retest repro upstream report log
2023/04/22 15:03 14m retest repro upstream report log
2023/01/12 13:32 17m retest repro upstream report log
2022/10/03 23:30 7m retest repro upstream error OK
2022/09/21 02:29 18m retest repro upstream OK log
▶ ▼ Fix bisection attempts (12)
Created Duration User Patch Repo Result
2021/09/01 13:04 16m bisect fix upstream job log (2)
2021/08/02 04:03 22m bisect fix upstream job log (0) log
2021/07/01 06:56 22m bisect fix upstream job log (0) log
2021/06/01 06:34 22m bisect fix upstream job log (0) log
2021/05/02 06:00 25m bisect fix upstream job log (0) log
2021/04/01 23:43 23m bisect fix upstream job log (0) log
2021/03/01 07:32 24m bisect fix upstream job log (0) log
2021/02/06 16:31 0m bisect fix upstream error job log (0)
2021/01/07 16:04 26m bisect fix upstream job log (0) log
2020/12/07 15:34 24m bisect fix upstream job log (0) log
2020/11/07 13:38 25m bisect fix upstream job log (0) log
2020/10/08 08:53 26m bisect fix upstream job log (0) log

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	0-...!: (1 GPs behind) idle=5b54/1/0x4000000000000000 softirq=9325/9326 fqs=0
rcu: 	(detected by 1, t=10506 jiffies, g=10485, q=223 ncpus=2)
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 7 Comm: kworker/0:0 Not tainted 6.5.0-syzkaller-11191-g6e32dfcccfcc #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/26/2023
Workqueue: ipv6_addrconf addrconf_dad_work
RIP: 0010:do_raw_spin_unlock+0x12/0x230 kernel/locking/spinlock_debug.c:138
Code: c4 73 00 eb 89 e8 2e 3c d4 08 e8 79 c4 73 00 eb a5 0f 1f 80 00 00 00 00 66 0f 1f 00 48 b8 00 00 00 00 00 fc ff df 41 54 55 53 <48> 89 fb 48 83 c7 04 48 89 fa 48 c1 ea 03 0f b6 14 02 48 89 f8 83
RSP: 0018:ffffc90000007cd8 EFLAGS: 00000086
RAX: dffffc0000000000 RBX: ffffffff92439138 RCX: ffffffff81675885
RDX: 0000000000000000 RSI: ffffffff8ae8e1e0 RDI: ffffffff92439138
RBP: 0000000000000002 R08: 0000000000000000 R09: fffffbfff1d9a752
R10: ffffffff8ecd3a97 R11: 0000000000000000 R12: 0000000000000001
R13: ffff888076b7b340 R14: ffffffff8a8f1e80 R15: 1ffff92000000fa6
FS:  0000000000000000(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000020000600 CR3: 000000000c976000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 __raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:150 [inline]
 _raw_spin_unlock_irqrestore+0x22/0x70 kernel/locking/spinlock.c:194
 debug_object_activate+0x283/0x490 lib/debugobjects.c:745
 debug_hrtimer_activate kernel/time/hrtimer.c:422 [inline]
 debug_activate kernel/time/hrtimer.c:477 [inline]
 enqueue_hrtimer+0x23/0x310 kernel/time/hrtimer.c:1087
 __run_hrtimer kernel/time/hrtimer.c:1705 [inline]
 __hrtimer_run_queues+0xa0a/0xc10 kernel/time/hrtimer.c:1752
 hrtimer_interrupt+0x31b/0x800 kernel/time/hrtimer.c:1814
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1063 [inline]
 __sysvec_apic_timer_interrupt+0x105/0x3f0 arch/x86/kernel/apic/apic.c:1080
 sysvec_apic_timer_interrupt+0x8e/0xc0 arch/x86/kernel/apic/apic.c:1074
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:645
RIP: 0010:qlist_free_all+0xef/0x1b0 mm/kasan/quarantine.c:180
Code: ff ff ba 10 00 00 00 31 f6 4c 89 f7 e8 ca 89 60 08 e9 4a ff ff ff e8 80 a3 b1 ff 9c 58 f6 c4 02 0f 85 a8 00 00 00 fb 4d 85 e4 <0f> 85 74 ff ff ff 48 8b 04 24 48 c7 40 08 00 00 00 00 48 c7 00 00
RSP: 0018:ffffc900002df4e0 EFLAGS: 00000286
RAX: 0000000000000046 RBX: ffff8880710c7e00 RCX: 1ffffffff1d9adfe
RDX: 0000000000000000 RSI: ffffffff8ae8e1e0 RDI: ffffffff81dba820
RBP: 0000000000000200 R08: 0000000000000000 R09: 0000000000000000
R10: ffffffff8ecd3a97 R11: ffff88801ab60c60 R12: ffff8880710346c0
R13: 0000000000000000 R14: ffff8880710c7e00 R15: ffff888012c74700
 kasan_quarantine_reduce+0x18b/0x1d0 mm/kasan/quarantine.c:292
 __kasan_slab_alloc+0x65/0x90 mm/kasan/common.c:305
 kasan_slab_alloc include/linux/kasan.h:186 [inline]
 slab_post_alloc_hook mm/slab.h:762 [inline]
 slab_alloc_node mm/slab.c:3237 [inline]
 kmem_cache_alloc_node+0x179/0x540 mm/slab.c:3509
 __alloc_skb+0x287/0x330 net/core/skbuff.c:634
 alloc_skb include/linux/skbuff.h:1286 [inline]
 alloc_skb_with_frags+0xe4/0x710 net/core/skbuff.c:6299
 sock_alloc_send_pskb+0x7c8/0x950 net/core/sock.c:2794
 sock_alloc_send_skb include/net/sock.h:1879 [inline]
 mld_newpack.isra.0+0x1ee/0x790 net/ipv6/mcast.c:1746
 add_grhead+0x295/0x340 net/ipv6/mcast.c:1849
 add_grec+0x10bb/0x1680 net/ipv6/mcast.c:1987
 mld_send_initial_cr.part.0+0xe2/0x260 net/ipv6/mcast.c:2234
 mld_send_initial_cr include/linux/refcount.h:201 [inline]
 ipv6_mc_dad_complete+0x255/0x2b0 net/ipv6/mcast.c:2245
 addrconf_dad_completed+0xcd8/0xfe0 net/ipv6/addrconf.c:4271
 addrconf_dad_work+0x807/0x13e0 net/ipv6/addrconf.c:4199
 process_one_work+0x887/0x15d0 kernel/workqueue.c:2630
 process_scheduled_works kernel/workqueue.c:2703 [inline]
 worker_thread+0x8bb/0x1290 kernel/workqueue.c:2784
 kthread+0x33a/0x430 kernel/kthread.c:388
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:304
 </TASK>
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 3.365 msecs
rcu: rcu_preempt kthread starved for 10506 jiffies! g10485 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:29056 pid:16    ppid:2      flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5382 [inline]
 __schedule+0xee1/0x59f0 kernel/sched/core.c:6695
 schedule+0xe7/0x1b0 kernel/sched/core.c:6771
 schedule_timeout+0x157/0x2c0 kernel/time/timer.c:2167
 rcu_gp_fqs_loop+0x1ec/0xa50 kernel/rcu/tree.c:1613
 rcu_gp_kthread+0x249/0x380 kernel/rcu/tree.c:1812
 kthread+0x33a/0x430 kernel/kthread.c:388
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:304
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
CPU: 1 PID: 1025 Comm: kworker/u4:6 Not tainted 6.5.0-syzkaller-11191-g6e32dfcccfcc #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/26/2023
Workqueue: events_unbound toggle_allocation_gate
RIP: 0010:csd_lock_wait kernel/smp.c:300 [inline]
RIP: 0010:smp_call_function_many_cond+0x4d6/0x1570 kernel/smp.c:844
Code: 0b 00 85 ed 74 4d 48 b8 00 00 00 00 00 fc ff df 4d 89 fc 4c 89 fd 49 c1 ec 03 83 e5 07 49 01 c4 83 c5 03 e8 9c 8c 0b 00 f3 90 <41> 0f b6 04 24 40 38 c5 7c 08 84 c0 0f 85 3c 0e 00 00 8b 43 08 31
RSP: 0018:ffffc9000429f928 EFLAGS: 00000293
RAX: 0000000000000000 RBX: ffff8880b98414c0 RCX: 0000000000000000
RDX: ffff88801ea12000 RSI: ffffffff817b0e74 RDI: 0000000000000005
RBP: 0000000000000003 R08: 0000000000000005 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000000 R12: ffffed1017308299
R13: 0000000000000001 R14: ffff8880b993d900 R15: ffff8880b98414c8
FS:  0000000000000000(0000) GS:ffff8880b9900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000020000600 CR3: 000000000c976000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 </IRQ>
 <TASK>
 on_each_cpu_cond_mask+0x40/0x90 kernel/smp.c:1012
 on_each_cpu include/linux/smp.h:71 [inline]
 text_poke_sync arch/x86/kernel/alternative.c:1998 [inline]
 text_poke_bp_batch+0x2ce/0x960 arch/x86/kernel/alternative.c:2208
 text_poke_flush arch/x86/kernel/alternative.c:2399 [inline]
 text_poke_flush arch/x86/kernel/alternative.c:2396 [inline]
 text_poke_finish+0x30/0x40 arch/x86/kernel/alternative.c:2406
 arch_jump_label_transform_apply+0x1c/0x30 arch/x86/kernel/jump_label.c:146
 jump_label_update+0x32e/0x410 kernel/jump_label.c:829
 static_key_enable_cpuslocked+0x1b5/0x270 kernel/jump_label.c:205
 static_key_enable+0x1a/0x20 kernel/jump_label.c:218
 toggle_allocation_gate mm/kfence/core.c:829 [inline]
 toggle_allocation_gate+0xf4/0x250 mm/kfence/core.c:821
 process_one_work+0x887/0x15d0 kernel/workqueue.c:2630
 process_scheduled_works kernel/workqueue.c:2703 [inline]
 worker_thread+0x8bb/0x1290 kernel/workqueue.c:2784
 kthread+0x33a/0x430 kernel/kthread.c:388
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x11/0x20 arch/x86/entry/entry_64.S:304
 </TASK>

Crashes (7):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2023/09/04 01:04 upstream 6e32dfcccfcc 696ea0d2 .config console log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in addrconf_dad_work
2023/08/25 18:28 linux-next 626932085009 03d9c195 .config console log report syz C [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in addrconf_dad_work
2020/09/03 15:50 upstream fc3abb53250a abf9ba4f .config console log report syz C ci-upstream-kasan-gce-root
2022/02/13 10:24 upstream b81b1829e7e3 8b9ca619 .config console log report syz C ci-upstream-kasan-gce INFO: rcu detected stall in addrconf_dad_work
2022/02/13 10:23 net-next-old 5a8fb33e5305 8b9ca619 .config console log report syz C ci-upstream-net-kasan-gce INFO: rcu detected stall in addrconf_dad_work
2023/07/26 07:54 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci e40939bbfc68 6756545c .config console log report syz [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 BUG: soft lockup in addrconf_dad_work
2023/07/19 17:17 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci e40939bbfc68 022df2bb .config console log report syz [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 BUG: soft lockup in addrconf_dad_work
* Struck through repros no longer work on HEAD.