syzbot

 sign-in | mailing list | source | docs
🐞 Open [560] 🐞 Fixed [32] 🐞 Invalid [300] ⬇ Missing Backports [45] 📈 Kernel Health 📈 Bug Lifetimes 📈 Fuzzing 📈 Crashes 💬 Send us feedback

INFO: task hung in synchronize_rcu (2)

Status: upstream: reported on 2024/02/21 13:53
Reported-by: syzbot+22954a262e8b7dae5a94@syzkaller.appspotmail.com
First crash: 70d, last: 70d
▶ ▼ Similar bugs (10)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: task hung in synchronize_rcu (3) C done done 374 441d 1379d 22/26 fixed on 2023/02/24 13:50
linux-6.1 INFO: task hung in synchronize_rcu origin:lts-only C unreliable 6 3d07h 396d 0/3 upstream: reported C repro on 2023/04/01 04:49
android-49 INFO: task hung in synchronize_rcu 1 1615d 1615d 0/3 auto-closed as invalid on 2020/03/28 16:57
linux-4.14 INFO: task hung in synchronize_rcu vfs C error 162 456d 1637d 0/1 upstream: reported C repro on 2019/11/07 04:32
linux-4.19 INFO: task hung in synchronize_rcu C error 70 445d 1636d 0/1 upstream: reported C repro on 2019/11/08 21:27
linux-5.15 INFO: task hung in synchronize_rcu 1 382d 382d 0/3 auto-obsoleted due to no activity on 2023/08/14 02:05
upstream INFO: task hung in synchronize_rcu syz done 1206 1615d 1641d 14/26 fixed on 2019/11/29 15:48
upstream INFO: task hung in synchronize_rcu (4) net fs C done error 62 10h56m 364d 0/26 upstream: reported C repro on 2023/05/04 02:01
android-414 INFO: task hung in synchronize_rcu 1 1633d 1633d 0/1 auto-closed as invalid on 2020/03/10 10:42
upstream INFO: task hung in synchronize_rcu (2) trace 8 1614d 1615d 0/26 closed as invalid on 2019/11/30 16:54

Sample crash report:
INFO: task syz-executor.4:21555 blocked for more than 143 seconds.
      Not tainted 5.15.148-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor.4  state:D stack:27448 pid:21555 ppid:  3542 flags:0x00004006
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5030 [inline]
 __schedule+0x12c4/0x45b0 kernel/sched/core.c:6376
 schedule+0x11b/0x1f0 kernel/sched/core.c:6459
 synchronize_rcu_expedited+0x5c6/0x740 kernel/rcu/tree_exp.h:861
 synchronize_rcu+0x107/0x1a0 kernel/rcu/tree.c:3798
 packet_release+0x982/0xc40 net/packet/af_packet.c:3126
 __sock_release net/socket.c:649 [inline]
 sock_close+0xcd/0x230 net/socket.c:1336
 __fput+0x3bf/0x890 fs/file_table.c:280
 task_work_run+0x129/0x1a0 kernel/task_work.c:164
 tracehook_notify_resume include/linux/tracehook.h:189 [inline]
 exit_to_user_mode_loop+0x106/0x130 kernel/entry/common.c:175
 exit_to_user_mode_prepare+0xb1/0x140 kernel/entry/common.c:208
 __syscall_exit_to_user_mode_work kernel/entry/common.c:290 [inline]
 syscall_exit_to_user_mode+0x5d/0x250 kernel/entry/common.c:301
 do_syscall_64+0x49/0xb0 arch/x86/entry/common.c:86
 entry_SYSCALL_64_after_hwframe+0x61/0xcb
RIP: 0033:0x7fdf803a4c9a
RSP: 002b:00007ffe4a68bcd0 EFLAGS: 00000293 ORIG_RAX: 0000000000000003
RAX: 0000000000000000 RBX: 0000000000000005 RCX: 00007fdf803a4c9a
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000004
RBP: 00007fdf804d5980 R08: 0000001b30620000 R09: 0000000000000353
R10: 0000000089f838fd R11: 0000000000000293 R12: 00000000002ca098
R13: ffffffffffffffff R14: 00007fdf7ff28000 R15: 00000000002c9d57
 </TASK>
INFO: task syz-executor.0:21570 blocked for more than 143 seconds.
      Not tainted 5.15.148-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor.0  state:D stack:27168 pid:21570 ppid:  3554 flags:0x00004006
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5030 [inline]
 __schedule+0x12c4/0x45b0 kernel/sched/core.c:6376
 schedule+0x11b/0x1f0 kernel/sched/core.c:6459
 schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6518
 __mutex_lock_common+0xe34/0x25a0 kernel/locking/mutex.c:669
 __mutex_lock kernel/locking/mutex.c:729 [inline]
 mutex_lock_nested+0x17/0x20 kernel/locking/mutex.c:743
 exp_funnel_lock kernel/rcu/tree_exp.h:322 [inline]
 synchronize_rcu_expedited+0x350/0x740 kernel/rcu/tree_exp.h:845
 synchronize_rcu+0x107/0x1a0 kernel/rcu/tree.c:3798
 packet_release+0x982/0xc40 net/packet/af_packet.c:3126
 __sock_release net/socket.c:649 [inline]
 sock_close+0xcd/0x230 net/socket.c:1336
 __fput+0x3bf/0x890 fs/file_table.c:280
 task_work_run+0x129/0x1a0 kernel/task_work.c:164
 tracehook_notify_resume include/linux/tracehook.h:189 [inline]
 exit_to_user_mode_loop+0x106/0x130 kernel/entry/common.c:175
 exit_to_user_mode_prepare+0xb1/0x140 kernel/entry/common.c:208
 __syscall_exit_to_user_mode_work kernel/entry/common.c:290 [inline]
 syscall_exit_to_user_mode+0x5d/0x250 kernel/entry/common.c:301
 do_syscall_64+0x49/0xb0 arch/x86/entry/common.c:86
 entry_SYSCALL_64_after_hwframe+0x61/0xcb
RIP: 0033:0x7f4bd2507c9a
RSP: 002b:00007ffcb09daf20 EFLAGS: 00000293 ORIG_RAX: 0000000000000003
RAX: 0000000000000000 RBX: 0000000000000005 RCX: 00007f4bd2507c9a
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000004
RBP: 00007f4bd2638980 R08: 0000001b2f360000 R09: 0000000000000010
R10: 0000000000000000 R11: 0000000000000293 R12: 00000000002ca25b
R13: ffffffffffffffff R14: 00007f4bd208b000 R15: 00000000002c9f1a
 </TASK>

Showing all locks held in the system:
1 lock held by khungtaskd/27:
 #0: ffffffff8c91f220 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x0/0x30
1 lock held by khugepaged/33:
 #0: ffffffff8c9b59c8 (lock#4){+.+.}-{3:3}, at: __lru_add_drain_all+0x67/0x9e0 mm/swap.c:782
2 locks held by getty/3257:
 #0: ffff88814b548098 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x21/0x70 drivers/tty/tty_ldisc.c:252
 #1: ffffc900024b32e8 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0x6af/0x1db0 drivers/tty/n_tty.c:2158
4 locks held by kworker/0:7/4193:
2 locks held by kworker/u4:16/24109:
 #0: ffff888011c79138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x78a/0x10c0 kernel/workqueue.c:2283
 #1: ffffc9000af2fd20 ((reaper_work).work){+.+.}-{0:0}, at: process_one_work+0x7d0/0x10c0 kernel/workqueue.c:2285
2 locks held by kworker/u4:0/8671:
 #0: ffff888011c79138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x78a/0x10c0 kernel/workqueue.c:2283
 #1: ffffc9000137fd20 (connector_reaper_work){+.+.}-{0:0}, at: process_one_work+0x7d0/0x10c0 kernel/workqueue.c:2285
2 locks held by kworker/1:42/14176:
 #0: ffff888011c70d38 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0x78a/0x10c0 kernel/workqueue.c:2283
 #1: ffffc900034e7d20 (free_ipc_work){+.+.}-{0:0}, at: process_one_work+0x7d0/0x10c0 kernel/workqueue.c:2285
3 locks held by kworker/1:44/14178:
 #0: ffff888011c70d38 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0x78a/0x10c0 kernel/workqueue.c:2283
 #1: ffffc90003507d20 ((linkwatch_work).work){+.+.}-{0:0}, at: process_one_work+0x7d0/0x10c0 kernel/workqueue.c:2285
 #2: ffffffff8d9da688 (rtnl_mutex){+.+.}-{3:3}, at: linkwatch_event+0xa/0x50 net/core/link_watch.c:251
2 locks held by syz-executor.4/21555:
 #0: ffff8880abdd9a10 (&sb->s_type->i_mutex_key#11){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:787 [inline]
 #0: ffff8880abdd9a10 (&sb->s_type->i_mutex_key#11){+.+.}-{3:3}, at: __sock_release net/socket.c:648 [inline]
 #0: ffff8880abdd9a10 (&sb->s_type->i_mutex_key#11){+.+.}-{3:3}, at: sock_close+0x98/0x230 net/socket.c:1336
 #1: ffffffff8c9237e8 (rcu_state.exp_mutex){+.+.}-{3:3}, at: exp_funnel_lock kernel/rcu/tree_exp.h:290 [inline]
 #1: ffffffff8c9237e8 (rcu_state.exp_mutex){+.+.}-{3:3}, at: synchronize_rcu_expedited+0x280/0x740 kernel/rcu/tree_exp.h:845
2 locks held by syz-executor.0/21570:
 #0: ffff888098a96810 (&sb->s_type->i_mutex_key#11){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:787 [inline]
 #0: ffff888098a96810 (&sb->s_type->i_mutex_key#11){+.+.}-{3:3}, at: __sock_release net/socket.c:648 [inline]
 #0: ffff888098a96810 (&sb->s_type->i_mutex_key#11){+.+.}-{3:3}, at: sock_close+0x98/0x230 net/socket.c:1336
 #1: ffffffff8c9237e8 (rcu_state.exp_mutex){+.+.}-{3:3}, at: exp_funnel_lock kernel/rcu/tree_exp.h:322 [inline]
 #1: ffffffff8c9237e8 (rcu_state.exp_mutex){+.+.}-{3:3}, at: synchronize_rcu_expedited+0x350/0x740 kernel/rcu/tree_exp.h:845
1 lock held by syz-executor.0/21900:
 #0: ffffffff8d9da688 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
 #0: ffffffff8d9da688 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x94c/0xee0 net/core/rtnetlink.c:5627
1 lock held by syz-executor.2/21921:
 #0: ffff8880abc32010 (&sb->s_type->i_mutex_key#11){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:787 [inline]
 #0: ffff8880abc32010 (&sb->s_type->i_mutex_key#11){+.+.}-{3:3}, at: __sock_release net/socket.c:648 [inline]
 #0: ffff8880abc32010 (&sb->s_type->i_mutex_key#11){+.+.}-{3:3}, at: sock_close+0x98/0x230 net/socket.c:1336
1 lock held by syz-executor.4/21988:
 #0: ffffffff8d9da688 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
 #0: ffffffff8d9da688 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x94c/0xee0 net/core/rtnetlink.c:5627
1 lock held by dhcpcd/21990:
 #0: ffff88800f3d4120 (sk_lock-AF_PACKET){+.+.}-{0:0}, at: lock_sock include/net/sock.h:1668 [inline]
 #0: ffff88800f3d4120 (sk_lock-AF_PACKET){+.+.}-{0:0}, at: packet_do_bind+0x33/0xd50 net/packet/af_packet.c:3164
1 lock held by syz-executor.2/22134:
 #0: ffffffff8d9da688 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
 #0: ffffffff8d9da688 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x94c/0xee0 net/core/rtnetlink.c:5627
2 locks held by syz-executor.3/22164:
 #0: ffffffff8d9cead0 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x37b/0x5d0 net/core/net_namespace.c:469
 #1: ffffffff8d9da688 (rtnl_mutex){+.+.}-{3:3}, at: smc_pnet_create_pnetids_list net/smc/smc_pnet.c:799 [inline]
 #1: ffffffff8d9da688 (rtnl_mutex){+.+.}-{3:3}, at: smc_pnet_net_init+0x176/0x3b0 net/smc/smc_pnet.c:868
1 lock held by syz-executor.1/22165:
 #0: ffffffff8d9da688 (rtnl_mutex){+.+.}-{3:3}, at: tun_detach drivers/net/tun.c:698 [inline]
 #0: ffffffff8d9da688 (rtnl_mutex){+.+.}-{3:3}, at: tun_chr_close+0x3a/0x1b0 drivers/net/tun.c:3434
1 lock held by dhcpcd/22167:
 #0: ffff88807783e120 (sk_lock-AF_PACKET){+.+.}-{0:0}, at: lock_sock include/net/sock.h:1668 [inline]
 #0: ffff88807783e120 (sk_lock-AF_PACKET){+.+.}-{0:0}, at: packet_do_bind+0x33/0xd50 net/packet/af_packet.c:3164
1 lock held by dhcpcd/22168:
 #0: ffff8880938ae120 (sk_lock-AF_PACKET){+.+.}-{0:0}, at: lock_sock include/net/sock.h:1668 [inline]
 #0: ffff8880938ae120 (sk_lock-AF_PACKET){+.+.}-{0:0}, at: packet_do_bind+0x33/0xd50 net/packet/af_packet.c:3164
1 lock held by dhcpcd/22169:
 #0: ffff888063176120 (sk_lock-AF_PACKET){+.+.}-{0:0}, at: lock_sock include/net/sock.h:1668 [inline]
 #0: ffff888063176120 (sk_lock-AF_PACKET){+.+.}-{0:0}, at: packet_do_bind+0x33/0xd50 net/packet/af_packet.c:3164
1 lock held by syz-executor.0/22171:
 #0: ffffffff8d9da688 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
 #0: ffffffff8d9da688 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x94c/0xee0 net/core/rtnetlink.c:5627
1 lock held by syz-executor.3/22175:
 #0: ffffffff8d9da688 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
 #0: ffffffff8d9da688 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x94c/0xee0 net/core/rtnetlink.c:5627
1 lock held by syz-executor.1/22178:
 #0: ffffffff8d9da688 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
 #0: ffffffff8d9da688 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x94c/0xee0 net/core/rtnetlink.c:5627
1 lock held by dhcpcd/22180:
 #0: ffff888078d32120 (sk_lock-AF_PACKET){+.+.}-{0:0}, at: lock_sock include/net/sock.h:1668 [inline]
 #0: ffff888078d32120 (sk_lock-AF_PACKET){+.+.}-{0:0}, at: packet_do_bind+0x33/0xd50 net/packet/af_packet.c:3164
1 lock held by syz-executor.4/22182:
 #0: ffffffff8d9da688 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
 #0: ffffffff8d9da688 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x94c/0xee0 net/core/rtnetlink.c:5627
1 lock held by dhcpcd/22184:
 #0: ffff8880941e4120 (sk_lock-AF_PACKET){+.+.}-{0:0}, at: lock_sock include/net/sock.h:1668 [inline]
 #0: ffff8880941e4120 (sk_lock-AF_PACKET){+.+.}-{0:0}, at: packet_do_bind+0x33/0xd50 net/packet/af_packet.c:3164
1 lock held by syz-executor.2/22186:
 #0: ffffffff8d9da688 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
 #0: ffffffff8d9da688 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x94c/0xee0 net/core/rtnetlink.c:5627

=============================================

NMI backtrace for cpu 1
CPU: 1 PID: 27 Comm: khungtaskd Not tainted 5.15.148-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/25/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e3/0x2cb lib/dump_stack.c:106
 nmi_cpu_backtrace+0x46a/0x4a0 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x181/0x2a0 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:148 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:210 [inline]
 watchdog+0xe72/0xeb0 kernel/hung_task.c:295
 kthread+0x3f6/0x4f0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298
 </TASK>
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 4193 Comm: kworker/0:7 Not tainted 5.15.148-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/25/2024
Workqueue: events ipvlan_process_multicast
RIP: 0010:debug_lockdep_rcu_enabled+0x25/0x30 kernel/rcu/update.c:281
Code: cc cc cc cc cc 31 c0 83 3d 7b 11 c7 03 00 74 1d 83 3d f2 43 c7 03 00 74 14 65 48 8b 0d 64 e1 e5 75 31 c0 83 b9 ec 0a 00 00 00 <0f> 94 c0 c3 cc cc cc cc cc cc cc 41 56 53 89 fb e8 e6 0d 00 00 41
RSP: 0018:ffffc90000007108 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff88809f2b28a8 RCX: ffff888022a99dc0
RDX: ffff888022a99dc0 RSI: 0000000000000000 RDI: 0000000000000000
RBP: ffff88802de1e360 R08: ffffffff88fd73a0 R09: ffffc90000007430
R10: ffffc90000007420 R11: dffffc0000000001 R12: ffff88808468e000
R13: ffff88809f2b28b7 R14: 1ffff11013e56516 R15: dffffc0000000000
FS:  0000000000000000(0000) GS:ffff8880b9a00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd7189ca008 CR3: 0000000019f23000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <IRQ>
 __in6_dev_get include/net/addrconf.h:321 [inline]
 ip6_ignore_linkdown include/net/addrconf.h:412 [inline]
 find_match+0xc3/0xc20 net/ipv6/route.c:738
 __find_rr_leaf+0x271/0x8e0 net/ipv6/route.c:826
 find_rr_leaf net/ipv6/route.c:847 [inline]
 rt6_select net/ipv6/route.c:891 [inline]
 fib6_table_lookup+0x453/0xbf0 net/ipv6/route.c:2185
 ip6_pol_route+0x1e7/0x15a0 net/ipv6/route.c:2221
 pol_lookup_func include/net/ip6_fib.h:582 [inline]
 fib6_rule_lookup+0x265/0x620 net/ipv6/fib6_rules.c:115
 ip6_route_input_lookup net/ipv6/route.c:2291 [inline]
 ip6_route_input+0x830/0xd90 net/ipv6/route.c:2587
 ip6_rcv_finish+0x136/0x250 net/ipv6/ip6_input.c:77
 NF_HOOK+0x364/0x410 include/linux/netfilter.h:302
 __netif_receive_skb_one_core net/core/dev.c:5485 [inline]
 __netif_receive_skb+0x1c6/0x530 net/core/dev.c:5599
 process_backlog+0x363/0x7f0 net/core/dev.c:6476
 __napi_poll+0xc7/0x440 net/core/dev.c:7035
 napi_poll net/core/dev.c:7102 [inline]
 net_rx_action+0x617/0xda0 net/core/dev.c:7189
 __do_softirq+0x3b3/0x93a kernel/softirq.c:558
 do_softirq+0x162/0x240 kernel/softirq.c:459
 </IRQ>
 <TASK>
 __local_bh_enable_ip+0x1b1/0x1f0 kernel/softirq.c:383
 ipvlan_process_multicast+0xa0d/0xeb0 drivers/net/ipvlan/ipvlan_core.c:279
 process_one_work+0x8a1/0x10c0 kernel/workqueue.c:2310
 worker_thread+0xaca/0x1280 kernel/workqueue.c:2457
 kthread+0x3f6/0x4f0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298
 </TASK>

Crashes (1):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/02/21 13:52 linux-5.15.y 6139f2a02fe0 3af7dd65 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: task hung in synchronize_rcu
* Struck through repros no longer work on HEAD.