syzbot


INFO: task hung in io_uring_del_tctx_node

Status: auto-obsoleted due to no activity on 2023/08/23 09:02
Reported-by: syzbot+63270f80f6aff2067626@syzkaller.appspotmail.com
First crash: 596d, last: 592d
Similar bugs (3)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: task hung in io_uring_del_tctx_node (3) io-uring 1 157d 157d 0/28 auto-obsoleted due to no activity on 2024/10/02 07:14
upstream INFO: task hung in io_uring_del_tctx_node io-uring fs C unreliable 37 1008d 1171d 20/28 fixed on 2022/03/08 16:11
upstream INFO: task hung in io_uring_del_tctx_node (2) io-uring C error error 20 637d 999d 0/28 auto-obsoleted due to no activity on 2023/07/30 22:24

Sample crash report:
INFO: task syz-executor.4:15333 blocked for more than 143 seconds.
      Not tainted 5.15.108-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor.4  state:D stack:24576 pid:15333 ppid: 15274 flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5026 [inline]
 __schedule+0x12c4/0x4590 kernel/sched/core.c:6372
 schedule+0x11b/0x1f0 kernel/sched/core.c:6455
 schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6514
 __mutex_lock_common+0xe34/0x25a0 kernel/locking/mutex.c:669
 __mutex_lock kernel/locking/mutex.c:729 [inline]
 mutex_lock_nested+0x17/0x20 kernel/locking/mutex.c:743
 io_uring_del_tctx_node+0xe3/0x2b0 io_uring/io_uring.c:9966
 io_uring_clean_tctx io_uring/io_uring.c:9982 [inline]
 io_uring_cancel_generic+0x6a3/0x900 io_uring/io_uring.c:10059
 io_uring_files_cancel include/linux/io_uring.h:16 [inline]
 do_exit+0x278/0x2480 kernel/exit.c:827
 do_group_exit+0x144/0x310 kernel/exit.c:994
 get_signal+0xc66/0x14e0 kernel/signal.c:2889
 arch_do_signal_or_restart+0xc3/0x1890 arch/x86/kernel/signal.c:865
 handle_signal_work kernel/entry/common.c:148 [inline]
 exit_to_user_mode_loop+0x97/0x130 kernel/entry/common.c:172
 exit_to_user_mode_prepare+0xb1/0x140 kernel/entry/common.c:208
 __syscall_exit_to_user_mode_work kernel/entry/common.c:290 [inline]
 syscall_exit_to_user_mode+0x5d/0x250 kernel/entry/common.c:301
 do_syscall_64+0x49/0xb0 arch/x86/entry/common.c:86
 entry_SYSCALL_64_after_hwframe+0x61/0xcb
RIP: 0033:0x7f9ba7bc9169
RSP: 002b:00007f9ba60f9168 EFLAGS: 00000246 ORIG_RAX: 0000000000000148
RAX: 0000000000000001 RBX: 00007f9ba7ce9120 RCX: 00007f9ba7bc9169
RDX: 0000000000000001 RSI: 00000000200004c0 RDI: 0000000000000009
RBP: 00007f9ba7c24ca1 R08: 0000000000000000 R09: 0000000000000002
R10: 0000000000003f00 R11: 0000000000000246 R12: 0000000000000000
R13: 00007ffef811b34f R14: 00007f9ba60f9300 R15: 0000000000022000
 </TASK>

Showing all locks held in the system:
1 lock held by ksoftirqd/1/20:
1 lock held by khungtaskd/27:
 #0: ffffffff8c91b920 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x0/0x30
2 locks held by getty/3269:
 #0: ffff888024ef7098 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x21/0x70 drivers/tty/tty_ldisc.c:252
 #1: ffffc90002bb32e8 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0x6af/0x1da0 drivers/tty/n_tty.c:2147
2 locks held by kworker/u4:203/31018:
4 locks held by kworker/u4:239/31054:
 #0: ffff888011db5138 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0x78a/0x10c0 kernel/workqueue.c:2279
 #1: ffffc90008fefd20 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0x7d0/0x10c0 kernel/workqueue.c:2281
 #2: ffffffff8d9c7d90 (pernet_ops_rwsem){++++}-{3:3}, at: cleanup_net+0xf1/0xb60 net/core/net_namespace.c:558
 #3: ffffffff8c91fe68 (rcu_state.exp_mutex){+.+.}-{3:3}, at: exp_funnel_lock kernel/rcu/tree_exp.h:290 [inline]
 #3: ffffffff8c91fe68 (rcu_state.exp_mutex){+.+.}-{3:3}, at: synchronize_rcu_expedited+0x280/0x740 kernel/rcu/tree_exp.h:840
2 locks held by kworker/0:3/32091:
 #0: ffff888011c66538 ((wq_completion)rcu_gp){+.+.}-{0:0}, at: process_one_work+0x78a/0x10c0 kernel/workqueue.c:2279
 #1: ffffc900130ffd20 ((work_completion)(&rew.rew_work)){+.+.}-{0:0}, at: process_one_work+0x7d0/0x10c0 kernel/workqueue.c:2281
1 lock held by syz-executor.4/15203:
3 locks held by kworker/0:4/15284:
 #0: ffff888011c64d38 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0x78a/0x10c0 kernel/workqueue.c:2279
 #1: ffffc90004f9fd20 (fqdir_free_work){+.+.}-{0:0}, at: process_one_work+0x7d0/0x10c0 kernel/workqueue.c:2281
 #2: ffffffff8c91fd70 (rcu_state.barrier_mutex){+.+.}-{3:3}, at: rcu_barrier+0x9c/0x4e0 kernel/rcu/tree.c:3994
1 lock held by syz-executor.4/15327:
1 lock held by syz-executor.4/15333:
 #0: ffff88802fa220a8 (&ctx->uring_lock){+.+.}-{3:3}, at: io_uring_del_tctx_node+0xe3/0x2b0 io_uring/io_uring.c:9966
3 locks held by kworker/1:4/15455:
 #0: ffff888011c64d38 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0x78a/0x10c0 kernel/workqueue.c:2279
 #1: ffffc90005a17d20 ((work_completion)(&pwq->unbound_release_work)){+.+.}-{0:0}, at: process_one_work+0x7d0/0x10c0 kernel/workqueue.c:2281
 #2: ffffffff8c91fe68 (rcu_state.exp_mutex){+.+.}-{3:3}, at: exp_funnel_lock kernel/rcu/tree_exp.h:322 [inline]
 #2: ffffffff8c91fe68 (rcu_state.exp_mutex){+.+.}-{3:3}, at: synchronize_rcu_expedited+0x350/0x740 kernel/rcu/tree_exp.h:840
2 locks held by udevd/16001:
 #0: ffff88801bf75918 (&disk->open_mutex){+.+.}-{3:3}, at: blkdev_put+0xfb/0x790 block/bdev.c:912
 #1: ffff88801beef468 (&lo->lo_mutex){+.+.}-{3:3}, at: __loop_clr_fd+0xa9/0xbe0 drivers/block/loop.c:1365
1 lock held by syz-executor.4/16106:

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 27 Comm: khungtaskd Not tainted 5.15.108-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/14/2023
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e3/0x2cb lib/dump_stack.c:106
 nmi_cpu_backtrace+0x46a/0x4a0 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x181/0x2a0 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:148 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:210 [inline]
 watchdog+0xe72/0xeb0 kernel/hung_task.c:295
 kthread+0x3f6/0x4f0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298
 </TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 20 Comm: ksoftirqd/1 Not tainted 5.15.108-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/14/2023
RIP: 0010:preempt_count kernel/rcu/tree.c:1161 [inline]
RIP: 0010:rcu_lockdep_current_cpu_online+0x0/0x130 kernel/rcu/tree.c:1166
Code: c1 7c db 48 89 df e8 df 74 5e 00 eb d1 44 89 f6 48 c7 c7 80 00 92 8c e8 ce eb bc 02 eb 88 66 2e 0f 1f 84 00 00 00 00 00 66 90 <41> 57 41 56 41 54 53 65 8b 0d 8a d1 97 7e b0 01 f7 c1 00 00 f0 00
RSP: 0018:ffffc90000da63e8 EFLAGS: 00000202
RAX: 00000000003e6e01 RBX: 0000000000000000 RCX: ffff8880129b0000
RDX: ffff8880129b0000 RSI: ffffffff8ad84de0 RDI: ffffffff8ad84da0
RBP: ffffc90000da6578 R08: ffffffff88f88428 R09: ffffc90000da66b0
R10: ffffc90000da66a0 R11: dffffc0000000001 R12: ffff888077ed5800
R13: 0000000000000000 R14: ffffc90000da6680 R15: dffffc0000000000
FS:  0000000000000000(0000) GS:ffff8880b9b00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000555556fb76e8 CR3: 000000007d28a000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <TASK>
 rcu_read_lock_held_common kernel/rcu/update.c:112 [inline]
 rcu_read_lock_held+0x1a/0x40 kernel/rcu/update.c:309
 __find_rr_leaf+0x391/0x8e0 net/ipv6/route.c:791
 find_rr_leaf net/ipv6/route.c:847 [inline]
 rt6_select net/ipv6/route.c:891 [inline]
 fib6_table_lookup+0x453/0xbf0 net/ipv6/route.c:2185
 ip6_pol_route+0x1e7/0x15a0 net/ipv6/route.c:2221
 pol_lookup_func include/net/ip6_fib.h:582 [inline]
 fib6_rule_lookup+0x265/0x620 net/ipv6/fib6_rules.c:115
 ip6_route_input_lookup net/ipv6/route.c:2291 [inline]
 ip6_route_input+0x82d/0xd90 net/ipv6/route.c:2587
 ip6_rcv_finish+0x136/0x250 net/ipv6/ip6_input.c:77
 ip_sabotage_in+0x1b6/0x230 net/bridge/br_netfilter_hooks.c:875
 nf_hook_entry_hookfn include/linux/netfilter.h:142 [inline]
 nf_hook_slow+0xae/0x1e0 net/netfilter/core.c:590
 nf_hook include/linux/netfilter.h:262 [inline]
 NF_HOOK+0x26b/0x410 include/linux/netfilter.h:305
 __netif_receive_skb_one_core net/core/dev.c:5458 [inline]
 __netif_receive_skb+0x1c6/0x530 net/core/dev.c:5572
 netif_receive_skb_internal net/core/dev.c:5658 [inline]
 netif_receive_skb+0x1b4/0x760 net/core/dev.c:5717
 NF_HOOK+0x96/0x3d0 include/linux/netfilter.h:307
 br_handle_frame_finish+0x1302/0x1700 net/bridge/br_input.c:181
 br_nf_hook_thresh+0x46a/0x580 net/bridge/br_netfilter_hooks.c:1042
 br_nf_pre_routing_finish_ipv6+0xa08/0xcd0
 NF_HOOK include/linux/netfilter.h:307 [inline]
 br_nf_pre_routing_ipv6+0x35d/0x720 net/bridge/br_netfilter_ipv6.c:237
 nf_hook_entry_hookfn include/linux/netfilter.h:142 [inline]
 nf_hook_bridge_pre net/bridge/br_input.c:237 [inline]
 br_handle_frame+0x8a8/0x10b0 net/bridge/br_input.c:377
 __netif_receive_skb_core+0x13b9/0x3df0 net/core/dev.c:5352
 __netif_receive_skb_one_core net/core/dev.c:5456 [inline]
 __netif_receive_skb+0x11c/0x530 net/core/dev.c:5572
 process_backlog+0x363/0x7f0 net/core/dev.c:6449
 __napi_poll+0xc7/0x440 net/core/dev.c:7008
 napi_poll net/core/dev.c:7075 [inline]
 net_rx_action+0x617/0xda0 net/core/dev.c:7162
 __do_softirq+0x3b3/0x93a kernel/softirq.c:558
 run_ksoftirqd+0xc1/0x120 kernel/softirq.c:920
 smpboot_thread_fn+0x51b/0x9d0 kernel/smpboot.c:164
 kthread+0x3f6/0x4f0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298
 </TASK>

Crashes (2):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2023/04/26 09:50 linux-5.15.y 3299fb36854f 65320f8e .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: task hung in io_uring_del_tctx_node
2023/04/22 12:44 linux-5.15.y 3299fb36854f 2b32bd34 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 INFO: task hung in io_uring_del_tctx_node
* Struck through repros no longer work on HEAD.