syzbot


INFO: task hung in rxrpc_destroy_all_calls (2)

Status: auto-closed as invalid on 2022/04/12 22:15
Subsystems: afs net
[Documentation on labels]
First crash: 1111d, last: 1111d
Similar bugs (3)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: task hung in rxrpc_destroy_all_calls afs net 13 1934d 1973d 0/28 auto-closed as invalid on 2020/01/11 01:03
upstream INFO: task hung in rxrpc_destroy_all_calls (3) afs net 352 768d 780d 0/28 auto-obsoleted due to no activity on 2023/03/21 16:19
linux-4.19 INFO: task hung in rxrpc_destroy_all_calls 1 1115d 1115d 0/1 auto-closed as invalid on 2022/05/08 23:28

Sample crash report:
INFO: task kworker/u4:8:4014 blocked for more than 143 seconds.
      Not tainted 5.16.0-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/u4:8    state:D stack:25016 pid: 4014 ppid:     2 flags:0x00004000
Workqueue: netns cleanup_net
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:4972 [inline]
 __schedule+0xa9a/0x4900 kernel/sched/core.c:6253
 schedule+0xd2/0x260 kernel/sched/core.c:6326
 rxrpc_destroy_all_calls+0x1ea/0x580 net/rxrpc/call_object.c:702
 rxrpc_exit_net+0x14d/0x2f0 net/rxrpc/net_ns.c:118
 ops_exit_list+0xb0/0x160 net/core/net_namespace.c:168
 cleanup_net+0x4ea/0xb00 net/core/net_namespace.c:595
 process_one_work+0x9b2/0x1660 kernel/workqueue.c:2298
 worker_thread+0x65d/0x1130 kernel/workqueue.c:2445
 kthread+0x405/0x4f0 kernel/kthread.c:327
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>

Showing all locks held in the system:
1 lock held by khungtaskd/27:
 #0: ffffffff8bb84620 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x53/0x260 kernel/locking/lockdep.c:6458
1 lock held by khugepaged/33:
 #0: ffffffff8bc6b508 (lock#5){+.+.}-{3:3}, at: __lru_add_drain_all+0x65/0x760 mm/swap.c:798
2 locks held by kworker/u4:3/54:
 #0: ffff888010c69138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
 #0: ffff888010c69138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: arch_atomic_long_set include/linux/atomic/atomic-long.h:41 [inline]
 #0: ffff888010c69138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: atomic_long_set include/linux/atomic/atomic-instrumented.h:1198 [inline]
 #0: ffff888010c69138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:635 [inline]
 #0: ffff888010c69138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:662 [inline]
 #0: ffff888010c69138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x896/0x1660 kernel/workqueue.c:2269
 #1: ffffc90001a2fdb0 (connector_reaper_work){+.+.}-{0:0}, at: process_one_work+0x8ca/0x1660 kernel/workqueue.c:2273
2 locks held by getty/3282:
 #0: ffff88807f36a098 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x22/0x80 drivers/tty/tty_ldisc.c:252
 #1: ffffc90002b962e8 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0xcf0/0x1230 drivers/tty/n_tty.c:2113
3 locks held by kworker/u4:6/3783:
2 locks held by kworker/u4:7/3912:
 #0: ffff888010c69138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
 #0: ffff888010c69138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: arch_atomic_long_set include/linux/atomic/atomic-long.h:41 [inline]
 #0: ffff888010c69138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: atomic_long_set include/linux/atomic/atomic-instrumented.h:1198 [inline]
 #0: ffff888010c69138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:635 [inline]
 #0: ffff888010c69138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:662 [inline]
 #0: ffff888010c69138 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x896/0x1660 kernel/workqueue.c:2269
 #1: ffffc90004dafdb0 ((reaper_work).work){+.+.}-{0:0}, at: process_one_work+0x8ca/0x1660 kernel/workqueue.c:2273
3 locks held by kworker/u4:8/4014:
 #0: ffff88814070b138 ((wq_completion)netns){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
 #0: ffff88814070b138 ((wq_completion)netns){+.+.}-{0:0}, at: arch_atomic_long_set include/linux/atomic/atomic-long.h:41 [inline]
 #0: ffff88814070b138 ((wq_completion)netns){+.+.}-{0:0}, at: atomic_long_set include/linux/atomic/atomic-instrumented.h:1198 [inline]
 #0: ffff88814070b138 ((wq_completion)netns){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:635 [inline]
 #0: ffff88814070b138 ((wq_completion)netns){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:662 [inline]
 #0: ffff88814070b138 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0x896/0x1660 kernel/workqueue.c:2269
 #1: ffffc90003acfdb0 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0x8ca/0x1660 kernel/workqueue.c:2273
 #2: ffffffff8d306bd0 (pernet_ops_rwsem){++++}-{3:3}, at: cleanup_net+0x9b/0xb00 net/core/net_namespace.c:557
2 locks held by kworker/1:15/8966:
 #0: ffff888010c66538 ((wq_completion)rcu_gp){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
 #0: ffff888010c66538 ((wq_completion)rcu_gp){+.+.}-{0:0}, at: arch_atomic_long_set include/linux/atomic/atomic-long.h:41 [inline]
 #0: ffff888010c66538 ((wq_completion)rcu_gp){+.+.}-{0:0}, at: atomic_long_set include/linux/atomic/atomic-instrumented.h:1198 [inline]
 #0: ffff888010c66538 ((wq_completion)rcu_gp){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:635 [inline]
 #0: ffff888010c66538 ((wq_completion)rcu_gp){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:662 [inline]
 #0: ffff888010c66538 ((wq_completion)rcu_gp){+.+.}-{0:0}, at: process_one_work+0x896/0x1660 kernel/workqueue.c:2269
 #1: ffffc9000499fdb0 ((work_completion)(&rew.rew_work)){+.+.}-{0:0}, at: process_one_work+0x8ca/0x1660 kernel/workqueue.c:2273
5 locks held by kworker/0:52/18187:
3 locks held by syz-executor.5/25676:
 #0: ffffffff8d306bd0 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x2f5/0x760 net/core/net_namespace.c:468
 #1: ffffffff8d31b6e8 (rtnl_mutex){+.+.}-{3:3}, at: ip_tunnel_delete_nets+0x8e/0x5b0 net/ipv4/ip_tunnel.c:1118
 #2: ffffffff8bb8d9a8 (rcu_state.exp_mutex){+.+.}-{3:3}, at: exp_funnel_lock kernel/rcu/tree_exp.h:290 [inline]
 #2: ffffffff8bb8d9a8 (rcu_state.exp_mutex){+.+.}-{3:3}, at: synchronize_rcu_expedited+0x4fa/0x620 kernel/rcu/tree_exp.h:836
2 locks held by syz-executor.5/25677:
 #0: ffffffff8d306bd0 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x2f5/0x760 net/core/net_namespace.c:468
 #1: ffffffff8bb8d8b0 (rcu_state.barrier_mutex){+.+.}-{3:3}, at: rcu_barrier+0x44/0x430 kernel/rcu/tree.c:3985

=============================================

NMI backtrace for cpu 1
CPU: 1 PID: 27 Comm: khungtaskd Not tainted 5.16.0-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:106
 nmi_cpu_backtrace.cold+0x47/0x144 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x1b3/0x230 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:210 [inline]
 watchdog+0xc1d/0xf50 kernel/hung_task.c:295
 kthread+0x405/0x4f0 kernel/kthread.c:327
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 18187 Comm: kworker/0:52 Not tainted 5.16.0-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: events ipvlan_process_multicast
RIP: 0010:write_comp_data kernel/kcov.c:221 [inline]
RIP: 0010:__sanitizer_cov_trace_const_cmp4+0x17/0x70 kernel/kcov.c:287
Code: 30 f0 4c 89 54 d8 20 48 89 10 5b c3 0f 1f 80 00 00 00 00 41 89 f8 bf 03 00 00 00 4c 8b 14 24 89 f1 65 48 8b 34 25 40 70 02 00 <e8> 14 f9 ff ff 84 c0 74 4b 48 8b 86 88 15 00 00 8b b6 84 15 00 00
RSP: 0018:ffffc900000075b8 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff888072a56ca0 RCX: 00000000fffffffd
RDX: ffff888029920000 RSI: ffff888029920000 RDI: 0000000000000003
RBP: ffffc90000007630 R08: 00000000fffffffd R09: 0000000000000000
R10: ffffffff87ea95d3 R11: 0000000000000000 R12: 0000000000000003
R13: 00000000fffffffd R14: 0000000000000000 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff8880b9c00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6c8d2371d0 CR3: 000000000b88e000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 find_match.part.0+0x163/0xd00 net/ipv6/route.c:747
 find_match net/ipv6/route.c:825 [inline]
 __find_rr_leaf+0x17f/0xd20 net/ipv6/route.c:826
 find_rr_leaf net/ipv6/route.c:847 [inline]
 rt6_select net/ipv6/route.c:891 [inline]
 fib6_table_lookup+0x649/0xa20 net/ipv6/route.c:2185
 ip6_pol_route+0x1c5/0x11e0 net/ipv6/route.c:2221
 pol_lookup_func include/net/ip6_fib.h:581 [inline]
 fib6_rule_lookup+0x111/0x6f0 net/ipv6/fib6_rules.c:115
 ip6_route_input_lookup net/ipv6/route.c:2291 [inline]
 ip6_route_input+0x63c/0xbc0 net/ipv6/route.c:2587
 ip6_rcv_finish_core.constprop.0.isra.0+0x168/0x570 net/ipv6/ip6_input.c:63
 ip6_rcv_finish net/ipv6/ip6_input.c:74 [inline]
 NF_HOOK include/linux/netfilter.h:307 [inline]
 NF_HOOK include/linux/netfilter.h:301 [inline]
 ipv6_rcv+0x21c/0x3b0 net/ipv6/ip6_input.c:297
 __netif_receive_skb_one_core+0x114/0x180 net/core/dev.c:5351
 __netif_receive_skb+0x24/0x1b0 net/core/dev.c:5465
 process_backlog+0x2a5/0x6c0 net/core/dev.c:5797
 __napi_poll+0xaf/0x440 net/core/dev.c:6365
 napi_poll net/core/dev.c:6432 [inline]
 net_rx_action+0x801/0xb40 net/core/dev.c:6519
 __do_softirq+0x29b/0x9c2 kernel/softirq.c:558
 do_softirq.part.0+0xde/0x130 kernel/softirq.c:459
 </IRQ>
 <TASK>
 do_softirq kernel/softirq.c:451 [inline]
 __local_bh_enable_ip+0x102/0x120 kernel/softirq.c:383
 local_bh_enable include/linux/bottom_half.h:33 [inline]
 ipvlan_process_multicast+0x821/0xd80 drivers/net/ipvlan/ipvlan_core.c:279
 process_one_work+0x9b2/0x1660 kernel/workqueue.c:2298
 worker_thread+0x65d/0x1130 kernel/workqueue.c:2445
 kthread+0x405/0x4f0 kernel/kthread.c:327
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>
----------------
Code disassembly (best guess):
   0:	30 f0                	xor    %dh,%al
   2:	4c 89 54 d8 20       	mov    %r10,0x20(%rax,%rbx,8)
   7:	48 89 10             	mov    %rdx,(%rax)
   a:	5b                   	pop    %rbx
   b:	c3                   	retq
   c:	0f 1f 80 00 00 00 00 	nopl   0x0(%rax)
  13:	41 89 f8             	mov    %edi,%r8d
  16:	bf 03 00 00 00       	mov    $0x3,%edi
  1b:	4c 8b 14 24          	mov    (%rsp),%r10
  1f:	89 f1                	mov    %esi,%ecx
  21:	65 48 8b 34 25 40 70 	mov    %gs:0x27040,%rsi
  28:	02 00
* 2a:	e8 14 f9 ff ff       	callq  0xfffff943 <-- trapping instruction
  2f:	84 c0                	test   %al,%al
  31:	74 4b                	je     0x7e
  33:	48 8b 86 88 15 00 00 	mov    0x1588(%rsi),%rax
  3a:	8b b6 84 15 00 00    	mov    0x1584(%rsi),%esi

Crashes (1):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2022/01/12 22:11 net-next-old fe8152b38d3a 44d1319a .config console log report info ci-upstream-net-kasan-gce INFO: task hung in rxrpc_destroy_all_calls
* Struck through repros no longer work on HEAD.