syzbot


INFO: task hung in __lru_add_drain_all

Status: auto-obsoleted due to no activity on 2024/02/20 10:46
Subsystems: net
[Documentation on labels]
Reported-by: syzbot+a9b681dcbc06eb2bca04@syzkaller.appspotmail.com
First crash: 1304d, last: 398d
Cause bisection: introduced by (bisect log) :
commit 997acaf6b4b59c6a9c259740312a69ea549cc684
Author: Mark Rutland <mark.rutland@arm.com>
Date: Mon Jan 11 15:37:07 2021 +0000

  lockdep: report broken irq restoration

Crash: WARNING in kvm_wait (log)
Repro: C syz .config
  
Fix bisection: failed (error log)
  
Discussions (5)
Title Replies (including bot) Last reply
[syzbot] Monthly mm report 3 (4) 2023/04/06 10:02
Re: [syzbot] INFO: task hung in __lru_add_drain_all 1 (1) 2021/09/05 23:55
Re: [syzbot] INFO: task hung in __lru_add_drain_all 1 (1) 2021/09/05 23:36
Re: [syzbot] INFO: task hung in __lru_add_drain_all 1 (1) 2021/09/04 17:04
[syzbot] INFO: task hung in __lru_add_drain_all 0 (1) 2021/09/03 08:32
Similar bugs (4)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-5.15 INFO: task hung in __lru_add_drain_all 1 601d 601d 0/3 auto-obsoleted due to no activity on 2023/08/21 02:18
linux-6.1 INFO: task hung in __lru_add_drain_all 1 210d 210d 0/3 auto-obsoleted due to no activity on 2024/08/25 20:43
upstream INFO: task hung in __lru_add_drain_all (2) mm 267 10h14m 210d 0/28 upstream: reported on 2024/05/17 22:28
linux-5.15 INFO: task hung in __lru_add_drain_all (2) 1 141d 141d 0/3 auto-obsoleted due to no activity on 2024/11/03 03:56
Last patch testing requests (5)
Created Duration User Patch Repo Result
2023/09/04 13:17 22m retest repro net OK log
2021/09/05 02:21 14m hdanton@sina.com patch upstream report log
2021/09/04 08:07 14m hdanton@sina.com patch upstream report log
2021/09/04 00:57 14m hdanton@sina.com upstream report log
2021/09/03 11:10 15m hdanton@sina.com patch git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git c7d102232649 report log
Fix bisection attempts (9)
Created Duration User Patch Repo Result
2023/08/21 07:03 0m bisect fix net-old error
2023/03/30 12:35 33m bisect fix net-old OK (0) job log log
2023/02/07 06:13 35m bisect fix net-old OK (0) job log log
2022/06/28 23:01 31m bisect fix net-old OK (0) job log log
2022/05/29 15:45 27m bisect fix net-old OK (0) job log log
2022/04/29 04:18 25m bisect fix net-old OK (0) job log log
2022/03/17 07:13 25m bisect fix net-old OK (0) job log log
2022/02/15 06:45 27m bisect fix net-old OK (0) job log log
2022/01/04 12:46 28m bisect fix net-old OK (0) job log log

Sample crash report:
INFO: task khugepaged:1663 blocked for more than 143 seconds.
      Not tainted 5.14.0-rc3-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:khugepaged      state:D stack:28464 pid: 1663 ppid:     2 flags:0x00004000
Call Trace:
 context_switch kernel/sched/core.c:4683 [inline]
 __schedule+0x93a/0x26f0 kernel/sched/core.c:5940
 schedule+0xd3/0x270 kernel/sched/core.c:6019
 schedule_timeout+0x1db/0x2a0 kernel/time/timer.c:1855
 do_wait_for_common kernel/sched/completion.c:85 [inline]
 __wait_for_common kernel/sched/completion.c:106 [inline]
 wait_for_common kernel/sched/completion.c:117 [inline]
 wait_for_completion+0x176/0x280 kernel/sched/completion.c:138
 __flush_work+0x50e/0xad0 kernel/workqueue.c:3053
 __lru_add_drain_all+0x3fd/0x760 mm/swap.c:842
 khugepaged_do_scan mm/khugepaged.c:2214 [inline]
 khugepaged+0x10f/0x5590 mm/khugepaged.c:2275
 kthread+0x3e5/0x4d0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295

Showing all locks held in the system:
1 lock held by khungtaskd/1647:
 #0: ffffffff8b97ba40 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x53/0x260 kernel/locking/lockdep.c:6446
1 lock held by khugepaged/1663:
 #0: ffffffff8ba5e948 (lock#6){+.+.}-{3:3}, at: __lru_add_drain_all+0x65/0x760 mm/swap.c:791
1 lock held by in:imklog/8274:
 #0: ffff88801b6b4ff0 (&f->f_pos_lock){+.+.}-{3:3}, at: __fdget_pos+0xe9/0x100 fs/file.c:974
3 locks held by kworker/1:4/8673:
3 locks held by kworker/1:5/8674:
2 locks held by syz-executor764/31172:

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 1647 Comm: khungtaskd Not tainted 5.14.0-rc3-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:105
 nmi_cpu_backtrace.cold+0x44/0xd7 lib/nmi_backtrace.c:105
 nmi_trigger_cpumask_backtrace+0x1b3/0x230 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:210 [inline]
 watchdog+0xd0a/0xfc0 kernel/hung_task.c:295
 kthread+0x3e5/0x4d0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 19 Comm: ksoftirqd/1 Not tainted 5.14.0-rc3-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
RIP: 0010:__find_rr_leaf+0x2d4/0xd20 net/ipv6/route.c:794
Code: 08 0f 84 58 01 00 00 e8 aa 3d a0 f9 48 83 3c 24 00 74 43 e8 9e 3d a0 f9 48 8d bb 80 00 00 00 48 89 f8 48 c1 e8 03 0f b6 04 28 <84> c0 74 08 3c 03 0f 8e 49 08 00 00 44 8b a3 80 00 00 00 44 8b 74
RSP: 0018:ffffc90000d97508 EFLAGS: 00000a06
RAX: 0000000000000000 RBX: ffff88804f143400 RCX: 0000000000000100
RDX: ffff888010a7d4c0 RSI: ffffffff87d55dc2 RDI: ffff88804f143480
RBP: dffffc0000000000 R08: 0000000000000000 R09: 0000000000000000
R10: ffffffff87d55d8a R11: 0000000000000000 R12: 0000000000000001
R13: ffffc90000d977f0 R14: 0000000000000000 R15: ffff88804f142aa0
FS:  0000000000000000(0000) GS:ffff8880b9d00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000200000c0 CR3: 000000001ff23000 CR4: 00000000001506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 find_rr_leaf net/ipv6/route.c:846 [inline]
 rt6_select net/ipv6/route.c:890 [inline]
 fib6_table_lookup+0x649/0xa20 net/ipv6/route.c:2174
 ip6_pol_route+0x1c5/0x11d0 net/ipv6/route.c:2210
 pol_lookup_func include/net/ip6_fib.h:579 [inline]
 fib6_rule_lookup+0x111/0x6f0 net/ipv6/fib6_rules.c:115
 ip6_route_input_lookup net/ipv6/route.c:2280 [inline]
 ip6_route_input+0x63c/0xb30 net/ipv6/route.c:2576
 ip6_rcv_finish_core.constprop.0.isra.0+0x168/0x570 net/ipv6/ip6_input.c:63
 ip6_rcv_finish net/ipv6/ip6_input.c:74 [inline]
 NF_HOOK include/linux/netfilter.h:307 [inline]
 NF_HOOK include/linux/netfilter.h:301 [inline]
 ipv6_rcv+0x229/0x3c0 net/ipv6/ip6_input.c:297
 __netif_receive_skb_one_core+0x114/0x180 net/core/dev.c:5498
 __netif_receive_skb+0x24/0x1b0 net/core/dev.c:5612
 process_backlog+0x2a5/0x6c0 net/core/dev.c:6492
 __napi_poll+0xaf/0x440 net/core/dev.c:7047
 napi_poll net/core/dev.c:7114 [inline]
 net_rx_action+0x801/0xb40 net/core/dev.c:7201
 __do_softirq+0x29b/0x9c2 kernel/softirq.c:558
 run_ksoftirqd kernel/softirq.c:920 [inline]
 run_ksoftirqd+0x2d/0x60 kernel/softirq.c:912
 smpboot_thread_fn+0x645/0x9c0 kernel/smpboot.c:164
 kthread+0x3e5/0x4d0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295

Crashes (71):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2021/08/01 20:18 net-old c7d102232649 6c236867 .config console log report syz C ci-upstream-net-this-kasan-gce INFO: task hung in __lru_add_drain_all
2023/11/12 10:45 upstream 1b907d050735 6d6dbf8a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: task hung in __lru_add_drain_all
2023/07/18 11:38 upstream fdf0eaf11452 20f8b3c2 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: task hung in __lru_add_drain_all
2023/06/29 08:34 upstream e8f75c0270d9 ca69c785 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: task hung in __lru_add_drain_all
2023/06/22 19:46 upstream dad9774deaf1 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: task hung in __lru_add_drain_all
2023/06/15 03:40 upstream b6dad5178cea 76decb82 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: task hung in __lru_add_drain_all
2023/06/14 03:03 upstream 15adb51c04cc d2ee9228 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-fs INFO: task hung in __lru_add_drain_all
2023/06/08 14:45 upstream 5f63595ebd82 7086cdb9 .config console log report info ci-upstream-kasan-gce-smack-root INFO: task hung in __lru_add_drain_all
2023/05/30 19:21 upstream 8b817fded42d 8d5c7541 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: task hung in __lru_add_drain_all
2023/02/28 00:21 upstream 982818426a0f e792ae78 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: task hung in __lru_add_drain_all
2023/01/08 01:55 upstream 0a71553536d2 1dac8c7a .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in __lru_add_drain_all
2022/11/07 09:08 upstream 089d1c31224e 6d752409 .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in __lru_add_drain_all
2022/10/07 16:14 upstream 4c86114194e6 8a212197 .config console log report info [disk image] [vmlinux] ci-upstream-kasan-gce-root INFO: task hung in __lru_add_drain_all
2022/09/30 08:54 upstream 987a926c1d8a 1d385642 .config console log report info [disk image] [vmlinux] ci-upstream-kasan-gce-selinux-root INFO: task hung in __lru_add_drain_all
2022/09/19 16:27 upstream 521a547ced64 dd9a85ff .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in __lru_add_drain_all
2022/08/19 03:12 upstream 3b06a2755758 26a13b38 .config console log report info ci-upstream-kasan-gce-root INFO: task hung in __lru_add_drain_all
2022/08/06 03:43 upstream 200e340f2196 e853abd9 .config console log report info ci-upstream-kasan-gce-root INFO: task hung in __lru_add_drain_all
2022/07/28 09:57 upstream 6e7765cb477a fb95c74d .config console log report info ci-upstream-kasan-gce INFO: task hung in __lru_add_drain_all
2021/12/03 02:05 upstream a51e3ac43ddb 61f86278 .config console log report info ci-upstream-kasan-gce-smack-root INFO: task hung in __lru_add_drain_all
2021/11/27 03:29 upstream c5c17547b778 63eeac02 .config console log report info ci-upstream-kasan-gce-root INFO: task hung in __lru_add_drain_all
2021/11/11 19:26 upstream debe436e77c7 75b04091 .config console log report info ci-upstream-kasan-gce INFO: task hung in __lru_add_drain_all
2021/11/06 18:39 upstream fe91c4725aee 4c1be0be .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in __lru_add_drain_all
2021/11/06 03:03 upstream fe91c4725aee 4c1be0be .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in __lru_add_drain_all
2021/11/05 19:15 upstream d4439a1189f9 4c1be0be .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in __lru_add_drain_all
2021/11/05 09:57 upstream d4439a1189f9 4c1be0be .config console log report info ci-upstream-kasan-gce-root INFO: task hung in __lru_add_drain_all
2021/10/16 23:55 upstream 304040fb4909 0c5d9412 .config console log report info ci-upstream-kasan-gce INFO: task hung in __lru_add_drain_all
2021/08/30 10:26 upstream 90ac80dcd313 be2c130d .config console log report info ci-upstream-kasan-gce INFO: task hung in __lru_add_drain_all
2021/08/01 11:28 upstream f3438b4c4e69 6c236867 .config console log report info ci-upstream-kasan-gce-root INFO: task hung in __lru_add_drain_all
2021/07/24 23:52 upstream f0fddcec6b62 4d1b57d4 .config console log report info ci-upstream-kasan-gce INFO: task hung in __lru_add_drain_all
2023/04/16 14:59 upstream a7a55e27ad72 ec410564 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-386 INFO: task hung in __lru_add_drain_all
2022/01/16 06:45 upstream a33f5c380c4b 723cfaf0 .config console log report info ci-upstream-kasan-gce-386 INFO: task hung in __lru_add_drain_all
2021/09/12 02:15 upstream c605c39677b9 5ae8508a .config console log report info ci-upstream-kasan-gce-386 INFO: task hung in __lru_add_drain_all
2023/07/01 05:42 net 3674fbf0451d af3053d2 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in __lru_add_drain_all
2023/04/30 00:38 net 6686317855c6 62df2017 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in __lru_add_drain_all
2021/11/23 06:13 net-old 03a000bfd719 545ab074 .config console log report info ci-upstream-net-this-kasan-gce INFO: task hung in __lru_add_drain_all
2021/10/17 00:03 net-old fac3cb82a54a 0c5d9412 .config console log report info ci-upstream-net-this-kasan-gce INFO: task hung in __lru_add_drain_all
2021/10/16 23:54 bpf 732b74d64704 0c5d9412 .config console log report info ci-upstream-bpf-kasan-gce INFO: task hung in __lru_add_drain_all
2021/10/15 16:20 bpf 732b74d64704 0c5d9412 .config console log report info ci-upstream-bpf-kasan-gce INFO: task hung in __lru_add_drain_all
2021/10/15 16:17 net-old ec681c53f8d2 0c5d9412 .config console log report info ci-upstream-net-this-kasan-gce INFO: task hung in __lru_add_drain_all
2021/10/03 05:35 bpf b0e875bac0fa db0f5787 .config console log report info ci-upstream-bpf-kasan-gce INFO: task hung in __lru_add_drain_all
2021/10/03 03:30 bpf b0e875bac0fa db0f5787 .config console log report info ci-upstream-bpf-kasan-gce INFO: task hung in __lru_add_drain_all
2021/09/20 03:27 bpf bc23f7244817 70b76c1d .config console log report info ci-upstream-bpf-kasan-gce INFO: task hung in __lru_add_drain_all
2021/09/12 02:14 bpf 57f780f1c433 5ae8508a .config console log report info ci-upstream-bpf-kasan-gce INFO: task hung in __lru_add_drain_all
2021/06/09 09:03 bpf 11fc79fc9f2e 5c2fe346 .config console log report info ci-upstream-bpf-kasan-gce INFO: task hung in __lru_add_drain_all
2021/06/08 20:40 bpf 11fc79fc9f2e b718257f .config console log report info ci-upstream-bpf-kasan-gce INFO: task hung in __lru_add_drain_all
2021/05/27 01:09 bpf f5d287126f63 858ea628 .config console log report info ci-upstream-bpf-kasan-gce INFO: task hung in __lru_add_drain_all
2021/05/23 11:58 bpf 84316ca4e100 3c7fef33 .config console log report info ci-upstream-bpf-kasan-gce INFO: task hung in __lru_add_drain_all
2023/09/11 00:38 net-next 73be7fb14e83 6654cf89 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in __lru_add_drain_all
2023/09/06 18:58 net-next bd6c11bc43c4 72324844 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in __lru_add_drain_all
2023/06/26 00:41 net-next 9ae440b8fdd6 09ffe269 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in __lru_add_drain_all
2023/06/16 13:46 net-next 97c5209b3d37 f3921d4d .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in __lru_add_drain_all
2023/06/12 21:21 net-next 3a2cb45ca0cc 749afb64 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in __lru_add_drain_all
2021/12/05 12:46 bpf-next 866de4074443 a617004c .config console log report info ci-upstream-bpf-next-kasan-gce INFO: task hung in __lru_add_drain_all
2021/11/27 02:15 bpf-next e32cb12ff52a 63eeac02 .config console log report info ci-upstream-bpf-next-kasan-gce INFO: task hung in __lru_add_drain_all
2021/10/07 05:09 bpf-next 9d0578722391 62ee0987 .config console log report info ci-upstream-bpf-next-kasan-gce INFO: task hung in __lru_add_drain_all
2021/09/19 05:02 net-next-old 983e59a27b92 70b76c1d .config console log report info ci-upstream-net-kasan-gce INFO: task hung in __lru_add_drain_all
2021/09/04 02:47 bpf-next a16ef91aa61a d236a457 .config console log report info ci-upstream-bpf-next-kasan-gce INFO: task hung in __lru_add_drain_all
2021/07/23 12:03 bpf-next 724f17b7d45d bc5f1d88 .config console log report info ci-upstream-bpf-next-kasan-gce INFO: task hung in __lru_add_drain_all
2021/07/05 10:03 net-next-old 5e437416ff66 55aa55c2 .config console log report info ci-upstream-net-kasan-gce INFO: task hung in __lru_add_drain_all
2021/07/01 07:54 net-next-old dbe69e433722 38a885d1 .config console log report info ci-upstream-net-kasan-gce INFO: task hung in __lru_add_drain_all
2021/05/19 23:48 bpf-next 5d67f349590d a343ba6b .config console log report info ci-upstream-bpf-next-kasan-gce INFO: task hung in __lru_add_drain_all
2021/05/19 22:14 bpf-next 1a532eb28df3 a343ba6b .config console log report info ci-upstream-bpf-next-kasan-gce INFO: task hung in __lru_add_drain_all
2023/09/10 03:57 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci fe4469582053 6654cf89 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 INFO: task hung in __lru_add_drain_all
2023/06/01 18:04 linux-next 715abedee4cd a4ae4f42 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in __lru_add_drain_all
2023/05/06 00:17 linux-next 83e5775d7afd 4cec9341 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in __lru_add_drain_all
2022/03/29 22:34 linux-next c2528a0cdebd 6bdac766 .config console log report info ci-upstream-linux-next-kasan-gce-root INFO: task hung in __lru_add_drain_all
2021/12/04 14:43 linux-next f81e94e91878 a617004c .config console log report info ci-upstream-linux-next-kasan-gce-root INFO: task hung in __lru_add_drain_all
2021/11/11 17:56 linux-next ad8be4fa6e81 75b04091 .config console log report info ci-upstream-linux-next-kasan-gce-root INFO: task hung in __lru_add_drain_all
2021/11/11 07:39 linux-next ad8be4fa6e81 75b04091 .config console log report info ci-upstream-linux-next-kasan-gce-root INFO: task hung in __lru_add_drain_all
2021/11/06 20:52 linux-next 6a37ebbe07bf 4c1be0be .config console log report info ci-upstream-linux-next-kasan-gce-root INFO: task hung in __lru_add_drain_all
2021/09/25 12:42 linux-next 5a5d008887b4 8cac236e .config console log report info ci-upstream-linux-next-kasan-gce-root INFO: task hung in __lru_add_drain_all
* Struck through repros no longer work on HEAD.