INFO: task hung in stop_sync_thread (2)

Status: fixed on 2018/05/08 18:30
Subsystems: lvs
[Documentation on labels]
Reported-by: syzbot+5fe074c01b2032ce9618@syzkaller.appspotmail.com
Fix commit: 5c64576a7789 ipvs: fix rtnl_lock lockups caused by start_sync_thread
First crash: 3040d, last: 3023d
▶ ▼ Discussions (6)
Title Replies (including bot) Last reply
[PATCH 4.4 00/56] 4.4.132-stable review 68 (68) 2018/07/05 16:20
[PATCH 4.14 00/62] 4.14.41-stable review 71 (71) 2018/05/16 07:57
[PATCH 4.16 00/72] 4.16.9-stable review 86 (86) 2018/05/15 06:47
[PATCH 4.9 00/36] 4.9.100-stable review 41 (41) 2018/05/15 05:40
[PATCH 00/12] Netfilter/IPVS fixes for net 15 (15) 2018/04/24 08:55
INFO: task hung in stop_sync_thread (2) 2 (4) 2018/03/30 16:54
▶ ▼ Similar bugs (4)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
android-44 INFO: task hung in stop_sync_thread 1 syz 12 3006d 2659d 0/2 public: reported syz repro on 2019/04/14 00:02
android-49 INFO: task hung in stop_sync_thread 1 1 3043d 3043d 0/3 closed as invalid on 2018/03/27 11:14
android-49 INFO: task hung in stop_sync_thread (2) 1 12 2889d 3039d 0/3 auto-closed as invalid on 2019/02/23 02:19
upstream INFO: task hung in stop_sync_thread lvs 1 C 2 3042d 3042d 0/29 closed as invalid on 2018/03/27 11:14

Sample crash report:
INFO: task syzkaller923914:4319 blocked for more than 120 seconds.
      Not tainted 4.16.0-rc6+ #286
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
syzkaller923914 D23312  4319   4316 0x00000000
Call Trace:
 context_switch kernel/sched/core.c:2862 [inline]
 __schedule+0x8fb/0x1ec0 kernel/sched/core.c:3440
 schedule+0xf5/0x430 kernel/sched/core.c:3499
 schedule_timeout+0x1a3/0x230 kernel/time/timer.c:1777
 do_wait_for_common kernel/sched/completion.c:86 [inline]
 __wait_for_common kernel/sched/completion.c:107 [inline]
 wait_for_common kernel/sched/completion.c:118 [inline]
 wait_for_completion+0x415/0x770 kernel/sched/completion.c:139
 kthread_stop+0x14a/0x7a0 kernel/kthread.c:530
 stop_sync_thread+0x3d9/0x740 net/netfilter/ipvs/ip_vs_sync.c:1996
 do_ip_vs_set_ctl+0x2b1/0x1cc0 net/netfilter/ipvs/ip_vs_ctl.c:2394
 nf_sockopt net/netfilter/nf_sockopt.c:106 [inline]
 nf_setsockopt+0x67/0xc0 net/netfilter/nf_sockopt.c:115
 ip_setsockopt+0x97/0xa0 net/ipv4/ip_sockglue.c:1253
 udp_setsockopt+0x45/0x80 net/ipv4/udp.c:2400
 sock_common_setsockopt+0x95/0xd0 net/core/sock.c:3039
 SYSC_setsockopt net/socket.c:1850 [inline]
 SyS_setsockopt+0x189/0x360 net/socket.c:1829
 do_syscall_64+0x281/0x940 arch/x86/entry/common.c:287
 entry_SYSCALL_64_after_hwframe+0x42/0xb7
RIP: 0033:0x4459d9
RSP: 002b:00007f1d6f47cdb8 EFLAGS: 00000297 ORIG_RAX: 0000000000000036
RAX: ffffffffffffffda RBX: 00000000006dac24 RCX: 00000000004459d9
RDX: 000000000000048c RSI: 0000000000000000 RDI: 0000000000000008
RBP: 00000000006dac20 R08: 0000000000000018 R09: 0000000000000000
R10: 0000000020000000 R11: 0000000000000297 R12: 0000000000000000
R13: 00007fffcf128acf R14: 00007f1d6f47d9c0 R15: 0000000000000001
INFO: lockdep is turned off.
NMI backtrace for cpu 1
CPU: 1 PID: 869 Comm: khungtaskd Not tainted 4.16.0-rc6+ #286
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:17 [inline]
 dump_stack+0x194/0x24d lib/dump_stack.c:53
 nmi_cpu_backtrace+0x1d2/0x210 lib/nmi_backtrace.c:103
 nmi_trigger_cpumask_backtrace+0x123/0x180 lib/nmi_backtrace.c:62
 arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38
 trigger_all_cpu_backtrace include/linux/nmi.h:138 [inline]
 check_hung_task kernel/hung_task.c:132 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:190 [inline]
 watchdog+0x90c/0xd60 kernel/hung_task.c:249
 kthread+0x33c/0x400 kernel/kthread.c:238
 ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:406
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 4317 Comm: syzkaller923914 Not tainted 4.16.0-rc6+ #286
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
RIP: 0010:__sanitizer_cov_trace_pc+0x0/0x50
RSP: 0018:ffff8801b56df138 EFLAGS: 00000093
RAX: ffff8801cbaae640 RBX: 0000000000000000 RCX: ffffffff866a3971
RDX: 0000000000000000 RSI: 0000000000000040 RDI: ffff8801db218038
RBP: ffff8801b56df168 R08: ffff88021fff801c R09: ffff88021fff8008
R10: ffff88021fff8010 R11: ffff88021fff801d R12: ffff8801db218038
R13: 0000000000000040 R14: ffff8801db218038 R15: 00000000ffffffff
FS:  0000000001968880(0000) GS:ffff8801db200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: ffffffffff600400 CR3: 00000001c94dc001 CR4: 00000000001606f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 cpumask_next+0x24/0x30 lib/cpumask.c:21
 select_idle_smt kernel/sched/fair.c:6116 [inline]
 select_idle_sibling+0x86d/0xda0 kernel/sched/fair.c:6238
 select_task_rq_fair+0xe0a/0x2910 kernel/sched/fair.c:6394
 select_task_rq kernel/sched/core.c:1554 [inline]
 try_to_wake_up+0x4ee/0x15f0 kernel/sched/core.c:2064
 default_wake_function+0x30/0x50 kernel/sched/core.c:3693
 autoremove_wake_function+0x78/0x350 kernel/sched/wait.c:377
 __wake_up_common+0x18e/0x780 kernel/sched/wait.c:97
 __wake_up_common_lock+0x1b4/0x310 kernel/sched/wait.c:125
 __wake_up_sync_key+0x19/0x20 kernel/sched/wait.c:203
 pipe_write+0x63b/0xd60 fs/pipe.c:477
 call_write_iter include/linux/fs.h:1782 [inline]
 new_sync_write fs/read_write.c:469 [inline]
 __vfs_write+0x684/0x970 fs/read_write.c:482
 vfs_write+0x189/0x510 fs/read_write.c:544
 SYSC_write fs/read_write.c:589 [inline]
 SyS_write+0xef/0x220 fs/read_write.c:581
 do_syscall_64+0x281/0x940 arch/x86/entry/common.c:287
 entry_SYSCALL_64_after_hwframe+0x42/0xb7
RIP: 0033:0x4459d9
RSP: 002b:00007fffcf128b48 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 00000000006dada0 RCX: 00000000004459d9
RDX: 0000000000000012 RSI: 00000000004adf28 RDI: 0000000000000001
RBP: 0000000000000010 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000006dad8c
R13: 00000000006dada0 R14: 0000000000000005 R15: 0000000000000001
Code: ff 4c 89 e7 e8 62 b2 38 00 e9 32 fd ff ff 48 8b bd d0 fe ff ff e8 f1 b2 38 00 e9 c3 fc ff ff 90 90 90 90 90 90 90 90 90 90 90 90 <55> 65 48 8b 04 25 c0 ed 01 00 48 89 e5 65 8b 15 8c 20 90 7e 81

Crashes (9):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2018/03/30 16:45 net-next-old 18845557fd6f d47f0ed6 .config console log report syz C ci-upstream-net-kasan-gce
2018/04/15 03:46 upstream 18b7fd1c93e5 7a67784c .config console log report ci-upstream-kasan-gce
2018/04/11 22:15 upstream b284d4d5a678 9cd56d71 .config console log report ci-upstream-kasan-gce
2018/04/04 02:27 upstream f2d285669aae 676bd07e .config console log report ci-upstream-kasan-gce
2018/04/03 20:35 upstream f2d285669aae 676bd07e .config console log report ci-upstream-kasan-gce-root
2018/04/07 20:52 upstream f2d285669aae 66f22a7f .config console log report ci-upstream-kasan-gce-386
2018/04/13 17:56 net-next-old 5d1365940a68 0a0c5db6 .config console log report ci-upstream-net-kasan-gce
2018/04/11 04:13 net-next-old 17dec0a94915 8b8de427 .config console log report ci-upstream-net-kasan-gce
2018/03/29 04:36 net-next-old 5d22d47b9ed9 bf5e585c .config console log report ci-upstream-net-kasan-gce
* Struck through repros no longer work on HEAD.