syzbot


INFO: task hung in __lru_add_drain_all

Status: upstream: reported C repro on 2021/09/03 08:32
Reported-by: syzbot+a9b681dcbc06eb2bca04@syzkaller.appspotmail.com
First crash: 560d, last: 23d

Cause bisection: introduced by (bisect log) :
commit 997acaf6b4b59c6a9c259740312a69ea549cc684
Author: Mark Rutland <mark.rutland@arm.com>
Date: Mon Jan 11 15:37:07 2021 +0000

  lockdep: report broken irq restoration

Crash: WARNING in kvm_wait (log)
Repro: C syz .config
Patch testing requests:
Created Duration User Patch Repo Result
2021/09/05 02:21 14m hdanton@sina.com patch upstream report log
2021/09/04 08:07 14m hdanton@sina.com patch upstream report log
2021/09/04 00:57 14m hdanton@sina.com upstream report log
2021/09/03 11:10 15m hdanton@sina.com patch git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git c7d102232649 report log

Sample crash report:
INFO: task khugepaged:1663 blocked for more than 143 seconds.
      Not tainted 5.14.0-rc3-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:khugepaged      state:D stack:28464 pid: 1663 ppid:     2 flags:0x00004000
Call Trace:
 context_switch kernel/sched/core.c:4683 [inline]
 __schedule+0x93a/0x26f0 kernel/sched/core.c:5940
 schedule+0xd3/0x270 kernel/sched/core.c:6019
 schedule_timeout+0x1db/0x2a0 kernel/time/timer.c:1855
 do_wait_for_common kernel/sched/completion.c:85 [inline]
 __wait_for_common kernel/sched/completion.c:106 [inline]
 wait_for_common kernel/sched/completion.c:117 [inline]
 wait_for_completion+0x176/0x280 kernel/sched/completion.c:138
 __flush_work+0x50e/0xad0 kernel/workqueue.c:3053
 __lru_add_drain_all+0x3fd/0x760 mm/swap.c:842
 khugepaged_do_scan mm/khugepaged.c:2214 [inline]
 khugepaged+0x10f/0x5590 mm/khugepaged.c:2275
 kthread+0x3e5/0x4d0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295

Showing all locks held in the system:
1 lock held by khungtaskd/1647:
 #0: ffffffff8b97ba40 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x53/0x260 kernel/locking/lockdep.c:6446
1 lock held by khugepaged/1663:
 #0: ffffffff8ba5e948 (lock#6){+.+.}-{3:3}, at: __lru_add_drain_all+0x65/0x760 mm/swap.c:791
1 lock held by in:imklog/8274:
 #0: ffff88801b6b4ff0 (&f->f_pos_lock){+.+.}-{3:3}, at: __fdget_pos+0xe9/0x100 fs/file.c:974
3 locks held by kworker/1:4/8673:
3 locks held by kworker/1:5/8674:
2 locks held by syz-executor764/31172:

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 1647 Comm: khungtaskd Not tainted 5.14.0-rc3-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:105
 nmi_cpu_backtrace.cold+0x44/0xd7 lib/nmi_backtrace.c:105
 nmi_trigger_cpumask_backtrace+0x1b3/0x230 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:210 [inline]
 watchdog+0xd0a/0xfc0 kernel/hung_task.c:295
 kthread+0x3e5/0x4d0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 19 Comm: ksoftirqd/1 Not tainted 5.14.0-rc3-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
RIP: 0010:__find_rr_leaf+0x2d4/0xd20 net/ipv6/route.c:794
Code: 08 0f 84 58 01 00 00 e8 aa 3d a0 f9 48 83 3c 24 00 74 43 e8 9e 3d a0 f9 48 8d bb 80 00 00 00 48 89 f8 48 c1 e8 03 0f b6 04 28 <84> c0 74 08 3c 03 0f 8e 49 08 00 00 44 8b a3 80 00 00 00 44 8b 74
RSP: 0018:ffffc90000d97508 EFLAGS: 00000a06
RAX: 0000000000000000 RBX: ffff88804f143400 RCX: 0000000000000100
RDX: ffff888010a7d4c0 RSI: ffffffff87d55dc2 RDI: ffff88804f143480
RBP: dffffc0000000000 R08: 0000000000000000 R09: 0000000000000000
R10: ffffffff87d55d8a R11: 0000000000000000 R12: 0000000000000001
R13: ffffc90000d977f0 R14: 0000000000000000 R15: ffff88804f142aa0
FS:  0000000000000000(0000) GS:ffff8880b9d00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000200000c0 CR3: 000000001ff23000 CR4: 00000000001506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 find_rr_leaf net/ipv6/route.c:846 [inline]
 rt6_select net/ipv6/route.c:890 [inline]
 fib6_table_lookup+0x649/0xa20 net/ipv6/route.c:2174
 ip6_pol_route+0x1c5/0x11d0 net/ipv6/route.c:2210
 pol_lookup_func include/net/ip6_fib.h:579 [inline]
 fib6_rule_lookup+0x111/0x6f0 net/ipv6/fib6_rules.c:115
 ip6_route_input_lookup net/ipv6/route.c:2280 [inline]
 ip6_route_input+0x63c/0xb30 net/ipv6/route.c:2576
 ip6_rcv_finish_core.constprop.0.isra.0+0x168/0x570 net/ipv6/ip6_input.c:63
 ip6_rcv_finish net/ipv6/ip6_input.c:74 [inline]
 NF_HOOK include/linux/netfilter.h:307 [inline]
 NF_HOOK include/linux/netfilter.h:301 [inline]
 ipv6_rcv+0x229/0x3c0 net/ipv6/ip6_input.c:297
 __netif_receive_skb_one_core+0x114/0x180 net/core/dev.c:5498
 __netif_receive_skb+0x24/0x1b0 net/core/dev.c:5612
 process_backlog+0x2a5/0x6c0 net/core/dev.c:6492
 __napi_poll+0xaf/0x440 net/core/dev.c:7047
 napi_poll net/core/dev.c:7114 [inline]
 net_rx_action+0x801/0xb40 net/core/dev.c:7201
 __do_softirq+0x29b/0x9c2 kernel/softirq.c:558
 run_ksoftirqd kernel/softirq.c:920 [inline]
 run_ksoftirqd+0x2d/0x60 kernel/softirq.c:912
 smpboot_thread_fn+0x645/0x9c0 kernel/smpboot.c:164
 kthread+0x3e5/0x4d0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295

Fix bisection attempts:
Manager Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Title
ci-upstream-net-this-kasan-gce 2022/06/28 23:33 net ab84db251c04 6c236867 .config log report syz C
ci-upstream-net-this-kasan-gce 2022/05/29 16:13 net 90343f573252 6c236867 .config log report syz C
ci-upstream-net-this-kasan-gce 2022/04/29 04:43 net 249aca0d3d63 6c236867 .config log report syz C
ci-upstream-net-this-kasan-gce 2022/03/17 07:39 net 4ee06de7729d 6c236867 .config log report syz C
ci-upstream-net-this-kasan-gce 2022/02/15 07:13 net 032062f363b4 6c236867 .config log report syz C
ci-upstream-net-this-kasan-gce 2022/01/04 13:15 net 7d18a07897d0 6c236867 .config log report syz C
* Struck through repros no longer work on HEAD.
Crashes (50):
Manager Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Title
ci-upstream-net-this-kasan-gce 2021/08/01 20:18 net c7d102232649 6c236867 .config log report syz C INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-selinux-root 2022/11/07 09:08 upstream 089d1c31224e 6d752409 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-root 2022/10/07 16:14 upstream 4c86114194e6 8a212197 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-selinux-root 2022/09/30 08:54 upstream 987a926c1d8a 1d385642 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-selinux-root 2022/09/19 16:27 upstream 521a547ced64 dd9a85ff .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-root 2022/08/19 03:12 upstream 3b06a2755758 26a13b38 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-root 2022/08/06 03:43 upstream 200e340f2196 e853abd9 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce 2022/07/28 09:57 upstream 6e7765cb477a fb95c74d .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-smack-root 2021/12/03 02:05 upstream a51e3ac43ddb 61f86278 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-root 2021/11/27 03:29 upstream c5c17547b778 63eeac02 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce 2021/11/11 19:26 upstream debe436e77c7 75b04091 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-selinux-root 2021/11/06 18:39 upstream fe91c4725aee 4c1be0be .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-selinux-root 2021/11/06 03:03 upstream fe91c4725aee 4c1be0be .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-selinux-root 2021/11/05 19:15 upstream d4439a1189f9 4c1be0be .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-root 2021/11/05 09:57 upstream d4439a1189f9 4c1be0be .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce 2021/10/16 23:55 upstream 304040fb4909 0c5d9412 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce 2021/08/30 10:26 upstream 90ac80dcd313 be2c130d .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-root 2021/08/01 11:28 upstream f3438b4c4e69 6c236867 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce 2021/07/24 23:52 upstream f0fddcec6b62 4d1b57d4 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-386 2022/01/16 06:45 upstream a33f5c380c4b 723cfaf0 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-386 2021/09/12 02:15 upstream c605c39677b9 5ae8508a .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-net-this-kasan-gce 2021/11/23 06:13 net 03a000bfd719 545ab074 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-net-this-kasan-gce 2021/10/17 00:03 net fac3cb82a54a 0c5d9412 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-kasan-gce 2021/10/16 23:54 bpf 732b74d64704 0c5d9412 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-kasan-gce 2021/10/15 16:20 bpf 732b74d64704 0c5d9412 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-net-this-kasan-gce 2021/10/15 16:17 net ec681c53f8d2 0c5d9412 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-kasan-gce 2021/10/03 05:35 bpf b0e875bac0fa db0f5787 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-kasan-gce 2021/10/03 03:30 bpf b0e875bac0fa db0f5787 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-kasan-gce 2021/09/20 03:27 bpf bc23f7244817 70b76c1d .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-kasan-gce 2021/09/12 02:14 bpf 57f780f1c433 5ae8508a .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-kasan-gce 2021/06/09 09:03 bpf 11fc79fc9f2e 5c2fe346 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-kasan-gce 2021/06/08 20:40 bpf 11fc79fc9f2e b718257f .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-kasan-gce 2021/05/27 01:09 bpf f5d287126f63 858ea628 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-kasan-gce 2021/05/23 11:58 bpf 84316ca4e100 3c7fef33 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-next-kasan-gce 2021/12/05 12:46 bpf-next 866de4074443 a617004c .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-next-kasan-gce 2021/11/27 02:15 bpf-next e32cb12ff52a 63eeac02 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-next-kasan-gce 2021/10/07 05:09 bpf-next 9d0578722391 62ee0987 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-net-kasan-gce 2021/09/19 05:02 net-next 983e59a27b92 70b76c1d .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-next-kasan-gce 2021/09/04 02:47 bpf-next a16ef91aa61a d236a457 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-next-kasan-gce 2021/07/23 12:03 bpf-next 724f17b7d45d bc5f1d88 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-net-kasan-gce 2021/07/05 10:03 net-next 5e437416ff66 55aa55c2 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-net-kasan-gce 2021/07/01 07:54 net-next dbe69e433722 38a885d1 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-next-kasan-gce 2021/05/19 23:48 bpf-next 5d67f349590d a343ba6b .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-next-kasan-gce 2021/05/19 22:14 bpf-next 1a532eb28df3 a343ba6b .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-linux-next-kasan-gce-root 2022/03/29 22:34 linux-next c2528a0cdebd 6bdac766 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-linux-next-kasan-gce-root 2021/12/04 14:43 linux-next f81e94e91878 a617004c .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-linux-next-kasan-gce-root 2021/11/11 17:56 linux-next ad8be4fa6e81 75b04091 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-linux-next-kasan-gce-root 2021/11/11 07:39 linux-next ad8be4fa6e81 75b04091 .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-linux-next-kasan-gce-root 2021/11/06 20:52 linux-next 6a37ebbe07bf 4c1be0be .config log report info INFO: task hung in __lru_add_drain_all
ci-upstream-linux-next-kasan-gce-root 2021/09/25 12:42 linux-next 5a5d008887b4 8cac236e .config log report info INFO: task hung in __lru_add_drain_all
* Struck through repros no longer work on HEAD.