INFO: task hung in __lru_add_drain

INFO: task hung in __lru_add_drain_all
Status: upstream: reported C repro on 2021/09/03 08:32
Reported-by: syzbot+a9b681dcbc06eb2bca04@syzkaller.appspotmail.com
First crash: 253d, last: 11d

Cause bisection: introduced by (bisect log) :
commit 997acaf6b4b59c6a9c259740312a69ea549cc684
Author: Mark Rutland <mark.rutland@arm.com>
Date: Mon Jan 11 15:37:07 2021 +0000

lockdep: report broken irq restoration

Crash: WARNING in kvm_wait (log)
Repro: C syz .config

Patch testing requests:
Created	Duration	User	Patch	Repo	Result
2021/09/05 02:21	14m	hdanton@sina.com	patch	upstream	report log
2021/09/04 08:07	14m	hdanton@sina.com	patch	upstream	report log
2021/09/04 00:57	14m	hdanton@sina.com		upstream	report log
2021/09/03 11:10	15m	hdanton@sina.com	patch	git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git c7d102232649	report log

Sample crash report:

INFO: task khugepaged:1663 blocked for more than 143 seconds.
      Not tainted 5.14.0-rc3-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:khugepaged      state:D stack:28464 pid: 1663 ppid:     2 flags:0x00004000
Call Trace:
 context_switch kernel/sched/core.c:4683 [inline]
 __schedule+0x93a/0x26f0 kernel/sched/core.c:5940
 schedule+0xd3/0x270 kernel/sched/core.c:6019
 schedule_timeout+0x1db/0x2a0 kernel/time/timer.c:1855
 do_wait_for_common kernel/sched/completion.c:85 [inline]
 __wait_for_common kernel/sched/completion.c:106 [inline]
 wait_for_common kernel/sched/completion.c:117 [inline]
 wait_for_completion+0x176/0x280 kernel/sched/completion.c:138
 __flush_work+0x50e/0xad0 kernel/workqueue.c:3053
 __lru_add_drain_all+0x3fd/0x760 mm/swap.c:842
 khugepaged_do_scan mm/khugepaged.c:2214 [inline]
 khugepaged+0x10f/0x5590 mm/khugepaged.c:2275
 kthread+0x3e5/0x4d0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295

Showing all locks held in the system:
1 lock held by khungtaskd/1647:
 #0: ffffffff8b97ba40 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x53/0x260 kernel/locking/lockdep.c:6446
1 lock held by khugepaged/1663:
 #0: ffffffff8ba5e948 (lock#6){+.+.}-{3:3}, at: __lru_add_drain_all+0x65/0x760 mm/swap.c:791
1 lock held by in:imklog/8274:
 #0: ffff88801b6b4ff0 (&f->f_pos_lock){+.+.}-{3:3}, at: __fdget_pos+0xe9/0x100 fs/file.c:974
3 locks held by kworker/1:4/8673:
3 locks held by kworker/1:5/8674:
2 locks held by syz-executor764/31172:

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 1647 Comm: khungtaskd Not tainted 5.14.0-rc3-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:105
 nmi_cpu_backtrace.cold+0x44/0xd7 lib/nmi_backtrace.c:105
 nmi_trigger_cpumask_backtrace+0x1b3/0x230 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:210 [inline]
 watchdog+0xd0a/0xfc0 kernel/hung_task.c:295
 kthread+0x3e5/0x4d0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 19 Comm: ksoftirqd/1 Not tainted 5.14.0-rc3-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
RIP: 0010:__find_rr_leaf+0x2d4/0xd20 net/ipv6/route.c:794
Code: 08 0f 84 58 01 00 00 e8 aa 3d a0 f9 48 83 3c 24 00 74 43 e8 9e 3d a0 f9 48 8d bb 80 00 00 00 48 89 f8 48 c1 e8 03 0f b6 04 28 <84> c0 74 08 3c 03 0f 8e 49 08 00 00 44 8b a3 80 00 00 00 44 8b 74
RSP: 0018:ffffc90000d97508 EFLAGS: 00000a06
RAX: 0000000000000000 RBX: ffff88804f143400 RCX: 0000000000000100
RDX: ffff888010a7d4c0 RSI: ffffffff87d55dc2 RDI: ffff88804f143480
RBP: dffffc0000000000 R08: 0000000000000000 R09: 0000000000000000
R10: ffffffff87d55d8a R11: 0000000000000000 R12: 0000000000000001
R13: ffffc90000d977f0 R14: 0000000000000000 R15: ffff88804f142aa0
FS:  0000000000000000(0000) GS:ffff8880b9d00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000200000c0 CR3: 000000001ff23000 CR4: 00000000001506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 find_rr_leaf net/ipv6/route.c:846 [inline]
 rt6_select net/ipv6/route.c:890 [inline]
 fib6_table_lookup+0x649/0xa20 net/ipv6/route.c:2174
 ip6_pol_route+0x1c5/0x11d0 net/ipv6/route.c:2210
 pol_lookup_func include/net/ip6_fib.h:579 [inline]
 fib6_rule_lookup+0x111/0x6f0 net/ipv6/fib6_rules.c:115
 ip6_route_input_lookup net/ipv6/route.c:2280 [inline]
 ip6_route_input+0x63c/0xb30 net/ipv6/route.c:2576
 ip6_rcv_finish_core.constprop.0.isra.0+0x168/0x570 net/ipv6/ip6_input.c:63
 ip6_rcv_finish net/ipv6/ip6_input.c:74 [inline]
 NF_HOOK include/linux/netfilter.h:307 [inline]
 NF_HOOK include/linux/netfilter.h:301 [inline]
 ipv6_rcv+0x229/0x3c0 net/ipv6/ip6_input.c:297
 __netif_receive_skb_one_core+0x114/0x180 net/core/dev.c:5498
 __netif_receive_skb+0x24/0x1b0 net/core/dev.c:5612
 process_backlog+0x2a5/0x6c0 net/core/dev.c:6492
 __napi_poll+0xaf/0x440 net/core/dev.c:7047
 napi_poll net/core/dev.c:7114 [inline]
 net_rx_action+0x801/0xb40 net/core/dev.c:7201
 __do_softirq+0x29b/0x9c2 kernel/softirq.c:558
 run_ksoftirqd kernel/softirq.c:920 [inline]
 run_ksoftirqd+0x2d/0x60 kernel/softirq.c:912
 smpboot_thread_fn+0x645/0x9c0 kernel/smpboot.c:164
 kthread+0x3e5/0x4d0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295

Fix bisection attempts:
Manager	Time	Kernel	Commit	Syzkaller	Config	Log	Report	Syz repro	C repro	VM info	Title
ci-upstream-net-this-kasan-gce	2022/01/04 13:15	net	7d18a07897d0	6c236867	.config	log	report	syz	C

Crashes (42):
Manager	Time	Kernel	Commit	Syzkaller	Config	Log	Report	Syz repro	C repro	VM info	Title
ci-upstream-net-this-kasan-gce	2021/08/01 20:18	net	c7d102232649	6c236867	.config	log	report	syz	C		INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-smack-root	2021/12/03 02:05	upstream	a51e3ac43ddb	61f86278	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-root	2021/11/27 03:29	upstream	c5c17547b778	63eeac02	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce	2021/11/11 19:26	upstream	debe436e77c7	75b04091	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-selinux-root	2021/11/06 18:39	upstream	fe91c4725aee	4c1be0be	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-selinux-root	2021/11/06 03:03	upstream	fe91c4725aee	4c1be0be	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-selinux-root	2021/11/05 19:15	upstream	d4439a1189f9	4c1be0be	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-root	2021/11/05 09:57	upstream	d4439a1189f9	4c1be0be	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce	2021/10/16 23:55	upstream	304040fb4909	0c5d9412	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce	2021/08/30 10:26	upstream	90ac80dcd313	be2c130d	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-root	2021/08/01 11:28	upstream	f3438b4c4e69	6c236867	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce	2021/07/24 23:52	upstream	f0fddcec6b62	4d1b57d4	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-386	2022/01/16 06:45	upstream	a33f5c380c4b	723cfaf0	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-kasan-gce-386	2021/09/12 02:15	upstream	c605c39677b9	5ae8508a	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-net-this-kasan-gce	2021/11/23 06:13	net	03a000bfd719	545ab074	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-net-this-kasan-gce	2021/10/17 00:03	net	fac3cb82a54a	0c5d9412	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-kasan-gce	2021/10/16 23:54	bpf	732b74d64704	0c5d9412	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-kasan-gce	2021/10/15 16:20	bpf	732b74d64704	0c5d9412	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-net-this-kasan-gce	2021/10/15 16:17	net	ec681c53f8d2	0c5d9412	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-kasan-gce	2021/10/03 05:35	bpf	b0e875bac0fa	db0f5787	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-kasan-gce	2021/10/03 03:30	bpf	b0e875bac0fa	db0f5787	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-kasan-gce	2021/09/20 03:27	bpf	bc23f7244817	70b76c1d	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-kasan-gce	2021/09/12 02:14	bpf	57f780f1c433	5ae8508a	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-kasan-gce	2021/06/09 09:03	bpf	11fc79fc9f2e	5c2fe346	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-kasan-gce	2021/06/08 20:40	bpf	11fc79fc9f2e	b718257f	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-kasan-gce	2021/05/27 01:09	bpf	f5d287126f63	858ea628	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-kasan-gce	2021/05/23 11:58	bpf	84316ca4e100	3c7fef33	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-next-kasan-gce	2021/12/05 12:46	bpf-next	866de4074443	a617004c	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-next-kasan-gce	2021/11/27 02:15	bpf-next	e32cb12ff52a	63eeac02	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-next-kasan-gce	2021/10/07 05:09	bpf-next	9d0578722391	62ee0987	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-net-kasan-gce	2021/09/19 05:02	net-next	983e59a27b92	70b76c1d	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-next-kasan-gce	2021/09/04 02:47	bpf-next	a16ef91aa61a	d236a457	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-next-kasan-gce	2021/07/23 12:03	bpf-next	724f17b7d45d	bc5f1d88	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-net-kasan-gce	2021/07/05 10:03	net-next	5e437416ff66	55aa55c2	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-net-kasan-gce	2021/07/01 07:54	net-next	dbe69e433722	38a885d1	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-next-kasan-gce	2021/05/19 23:48	bpf-next	5d67f349590d	a343ba6b	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-bpf-next-kasan-gce	2021/05/19 22:14	bpf-next	1a532eb28df3	a343ba6b	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-linux-next-kasan-gce-root	2021/12/04 14:43	linux-next	f81e94e91878	a617004c	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-linux-next-kasan-gce-root	2021/11/11 17:56	linux-next	ad8be4fa6e81	75b04091	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-linux-next-kasan-gce-root	2021/11/11 07:39	linux-next	ad8be4fa6e81	75b04091	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-linux-next-kasan-gce-root	2021/11/06 20:52	linux-next	6a37ebbe07bf	4c1be0be	.config	log	report			info	INFO: task hung in __lru_add_drain_all
ci-upstream-linux-next-kasan-gce-root	2021/09/25 12:42	linux-next	5a5d008887b4	8cac236e	.config	log	report			info	INFO: task hung in __lru_add_drain_all