syzbot


INFO: task hung in tcf_action_init_1

Status: fixed on 2021/04/09 19:46
Subsystems: net
[Documentation on labels]
Reported-by: syzbot+82752bc5331601cf4899@syzkaller.appspotmail.com
Fix commit: d349f9976868 net_sched: fix RTNL deadlock again caused by request_module()
First crash: 1359d, last: 1223d
Cause bisection: introduced by (bisect log) :
commit 0fedc63fadf0404a729e73a35349481c8009c02f
Author: Cong Wang <xiyou.wangcong@gmail.com>
Date: Wed Sep 23 03:56:24 2020 +0000

  net_sched: commit action insertions together

Crash: INFO: task hung in addrconf_dad_work (log)
Repro: C syz .config
  
Discussions (4)
Title Replies (including bot) Last reply
[PATCH 5.10 000/663] 5.10.20-rc1 review 673 (673) 2021/03/05 18:03
[PATCH 5.11 000/775] 5.11.3-rc1 review 776 (776) 2021/03/01 16:15
[Patch net-next] net_sched: fix RTNL deadlock again caused by request_module() 4 (4) 2021/01/19 04:30
INFO: task hung in tcf_action_init_1 1 (3) 2020/10/01 06:40
Last patch testing requests (1)
Created Duration User Patch Repo Result
2020/10/01 01:10 17m xiyou.wangcong@gmail.com https://github.com/congwang/linux.git net OK
Fix bisection attempts (2)
Created Duration User Patch Repo Result
2021/01/07 08:29 26m bisect fix net-old job log (0) log
2020/12/07 00:50 26m bisect fix net-old job log (0) log

Sample crash report:
INFO: task syz-executor541:8453 blocked for more than 143 seconds.
      Not tainted 5.11.0-rc7-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor541 state:D stack:25216 pid: 8453 ppid:  8451 flags:0x00004004
Call Trace:
 context_switch kernel/sched/core.c:4327 [inline]
 __schedule+0x90c/0x21a0 kernel/sched/core.c:5078
 schedule+0xcf/0x270 kernel/sched/core.c:5157
 schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:5216
 __mutex_lock_common kernel/locking/mutex.c:1033 [inline]
 __mutex_lock+0x81a/0x1110 kernel/locking/mutex.c:1103
 tcf_action_init_1+0x743/0x990 net/sched/act_api.c:988
 tcf_action_init+0x265/0x4b0 net/sched/act_api.c:1063
 tcf_action_add+0xd9/0x360 net/sched/act_api.c:1476
 tc_ctl_action+0x33a/0x440 net/sched/act_api.c:1530
 rtnetlink_rcv_msg+0x44e/0xad0 net/core/rtnetlink.c:5553
 netlink_rcv_skb+0x153/0x420 net/netlink/af_netlink.c:2494
 netlink_unicast_kernel net/netlink/af_netlink.c:1304 [inline]
 netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1330
 netlink_sendmsg+0x856/0xd90 net/netlink/af_netlink.c:1919
 sock_sendmsg_nosec net/socket.c:652 [inline]
 sock_sendmsg+0xcf/0x120 net/socket.c:672
 ____sys_sendmsg+0x6e8/0x810 net/socket.c:2345
 ___sys_sendmsg+0xf3/0x170 net/socket.c:2399
 __sys_sendmsg+0xe5/0x1b0 net/socket.c:2432
 do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46
 entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x446b19
RSP: 002b:00007f6c74c20208 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 00000000004cb428 RCX: 0000000000446b19
RDX: 0000000000000000 RSI: 0000000020002980 RDI: 0000000000000003
RBP: 00000000004cb420 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000004cb42c
R13: 00007ffc49ded21f R14: 00007f6c74c20300 R15: 0000000000022000

Showing all locks held in the system:
3 locks held by kworker/1:1/34:
 #0: ffff8880230ca538 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
 #0: ffff8880230ca538 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: atomic64_set include/asm-generic/atomic-instrumented.h:856 [inline]
 #0: ffff8880230ca538 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: atomic_long_set include/asm-generic/atomic-long.h:41 [inline]
 #0: ffff8880230ca538 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:616 [inline]
 #0: ffff8880230ca538 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:643 [inline]
 #0: ffff8880230ca538 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: process_one_work+0x871/0x15f0 kernel/workqueue.c:2246
 #1: ffffc90000e37da8 ((addr_chk_work).work){+.+.}-{0:0}, at: process_one_work+0x8a5/0x15f0 kernel/workqueue.c:2250
 #2: ffffffff8d45c228 (rtnl_mutex){+.+.}-{3:3}, at: addrconf_verify_work+0xa/0x20 net/ipv6/addrconf.c:4570
1 lock held by khungtaskd/1620:
 #0: ffffffff8bd73da0 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x53/0x260 kernel/locking/lockdep.c:6259
1 lock held by in:imklog/8163:
 #0: ffff888013a36170 (&f->f_pos_lock){+.+.}-{3:3}, at: __fdget_pos+0xe9/0x100 fs/file.c:947
1 lock held by syz-executor541/8453:
 #0: ffffffff8d45c228 (rtnl_mutex){+.+.}-{3:3}, at: tcf_action_init_1+0x743/0x990 net/sched/act_api.c:988
1 lock held by syz-executor541/8455:

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 1620 Comm: khungtaskd Not tainted 5.11.0-rc7-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:79 [inline]
 dump_stack+0x107/0x163 lib/dump_stack.c:120
 nmi_cpu_backtrace.cold+0x44/0xd7 lib/nmi_backtrace.c:105
 nmi_trigger_cpumask_backtrace+0x1b3/0x230 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:209 [inline]
 watchdog+0xd43/0xfa0 kernel/hung_task.c:294
 kthread+0x3b1/0x4a0 kernel/kthread.c:292
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:296
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 8455 Comm: syz-executor541 Not tainted 5.11.0-rc7-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
RIP: 0010:__lock_acquire+0x1e8/0x54f0 kernel/locking/lockdep.c:4742
Code: 4c 8d 14 92 49 c1 e2 03 85 d2 74 47 4a 8d 54 16 d8 48 b9 00 00 00 00 00 fc ff df 48 8d 7a 20 48 89 fe 48 c1 ee 03 0f b6 0c 0e <84> c9 74 09 80 f9 03 0f 8e ca 3b 00 00 0f b7 4a 20 81 e1 ff 1f 00
RSP: 0018:ffffc90001736b70 EFLAGS: 00000806
RAX: 0000000000000003 RBX: 0000000000000001 RCX: 0000000000000000
RDX: ffff8880265daa88 RSI: 1ffff11004cbb555 RDI: ffff8880265daaa8
RBP: ffff8880265da100 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000028 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: ffff888023874068 R15: 0000000000000000
FS:  00007f6c74bff700(0000) GS:ffff8880b9d00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc4efd5a000 CR3: 000000001af15000 CR4: 00000000001506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 lock_acquire kernel/locking/lockdep.c:5442 [inline]
 lock_acquire+0x1a8/0x720 kernel/locking/lockdep.c:5407
 __mutex_lock_common kernel/locking/mutex.c:956 [inline]
 __mutex_lock+0x134/0x1110 kernel/locking/mutex.c:1103
 tcf_idr_check_alloc+0x78/0x3b0 net/sched/act_api.c:549
 tcf_connmark_init+0x2de/0x910 net/sched/act_connmark.c:124
 tcf_action_init_1+0x63b/0x990 net/sched/act_api.c:1010
 tcf_action_init+0x265/0x4b0 net/sched/act_api.c:1063
 tcf_action_add+0xd9/0x360 net/sched/act_api.c:1476
 tc_ctl_action+0x33a/0x440 net/sched/act_api.c:1530
 rtnetlink_rcv_msg+0x44e/0xad0 net/core/rtnetlink.c:5553
 netlink_rcv_skb+0x153/0x420 net/netlink/af_netlink.c:2494
 netlink_unicast_kernel net/netlink/af_netlink.c:1304 [inline]
 netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1330
 netlink_sendmsg+0x856/0xd90 net/netlink/af_netlink.c:1919
 sock_sendmsg_nosec net/socket.c:652 [inline]
 sock_sendmsg+0xcf/0x120 net/socket.c:672
 ____sys_sendmsg+0x6e8/0x810 net/socket.c:2345
 ___sys_sendmsg+0xf3/0x170 net/socket.c:2399
 __sys_sendmsg+0xe5/0x1b0 net/socket.c:2432
 do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46
 entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x446b19
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 11 15 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 bc ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f6c74bff208 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 00000000004cb438 RCX: 0000000000446b19
RDX: 0000000000000000 RSI: 0000000020002980 RDI: 0000000000000004
RBP: 00000000004cb430 R08: 0000000000000004 R09: 0000000000000000
R10: 0000000000000004 R11: 0000000000000246 R12: 00000000004cb43c
R13: 00007ffc49ded21f R14: 00007f6c74bff300 R15: 0000000000022000
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.226 msecs

Crashes (30):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2021/02/09 10:19 upstream e0756cfc7d7c 2bd9619f .config console log report syz C ci-upstream-kasan-gce-selinux-root INFO: task hung in tcf_action_init_1
2021/02/07 17:52 upstream 825b5991a46e 2ce644fc .config console log report syz C ci-upstream-kasan-gce-root INFO: task hung in tcf_action_init_1
2021/02/05 06:36 upstream 5c279c4cf206 23a562df .config console log report syz C ci-upstream-kasan-gce-smack-root INFO: task hung in tcf_action_init_1
2021/01/27 17:18 upstream 2ab38c17aac1 a0ebf917 .config console log report syz C ci-upstream-kasan-gce-root INFO: task hung in tcf_action_init_1
2020/10/13 04:36 net-old 8098bd69bc4e d32b0bbf .config console log report syz C ci-upstream-net-this-kasan-gce
2020/10/10 19:50 net-old 7b50ee3dad25 4a77ae0b .config console log report syz C ci-upstream-net-this-kasan-gce
2020/09/26 08:34 net-old 5e46e43c2ad9 4a006f63 .config console log report syz C ci-upstream-net-this-kasan-gce
2020/10/08 00:46 net-next-old 9faebeb2d800 1880b4a9 .config console log report syz C ci-upstream-net-kasan-gce
2020/10/06 22:27 net-next-old 8b0308fe319b 1880b4a9 .config console log report syz C ci-upstream-net-kasan-gce
2020/10/06 20:12 net-next-old 8b0308fe319b 1880b4a9 .config console log report syz C ci-upstream-net-kasan-gce
2020/10/06 17:28 net-next-old 8b0308fe319b 1880b4a9 .config console log report syz C ci-upstream-net-kasan-gce
2020/11/05 23:26 upstream 521b619acdc8 64069d48 .config console log report info ci-upstream-kasan-gce-root
2020/11/05 23:11 upstream 521b619acdc8 64069d48 .config console log report info ci-upstream-kasan-gce-smack-root
2020/10/30 17:19 upstream 07e088730245 a6e3ac3b .config console log report info ci-upstream-kasan-gce-root
2020/10/12 22:35 upstream bbf5c979011a d32b0bbf .config console log report info ci-upstream-kasan-gce-selinux-root
2020/10/06 02:19 upstream 7575fdda569b 1880b4a9 .config console log report info ci-upstream-kasan-gce-root
2020/10/30 17:19 net-old 07e088730245 a6e3ac3b .config console log report info ci-upstream-net-this-kasan-gce
2020/10/05 12:43 net-old 580e4273d7a8 5ef9c291 .config console log report info ci-upstream-net-this-kasan-gce
2020/10/02 19:04 net-old ef9da46ddef0 4969d6ca .config console log report info ci-upstream-net-this-kasan-gce
2020/10/02 04:15 net-old bb13a800620c 9602ddf4 .config console log report info ci-upstream-net-this-kasan-gce
2020/10/02 00:32 net-old bb13a800620c 9602ddf4 .config console log report info ci-upstream-net-this-kasan-gce
2020/10/01 15:16 net-old a59cf619787e a9767fb2 .config console log report info ci-upstream-net-this-kasan-gce
2020/10/01 08:47 net-old a59cf619787e a9767fb2 .config console log report info ci-upstream-net-this-kasan-gce
2020/09/30 15:46 net-old 2b3e981a94d8 8516f6d3 .config console log report info ci-upstream-net-this-kasan-gce
2020/09/28 19:40 net-old 709a16be0593 6bfdbe89 .config console log report info ci-upstream-net-this-kasan-gce
2020/09/27 15:35 net-old 059432495e20 5dd8aee8 .config console log report info ci-upstream-net-this-kasan-gce
2020/10/15 22:29 net-next-old 346e320cb210 6e262c73 .config console log report info ci-upstream-net-kasan-gce
2020/10/07 01:15 net-next-old 8b0308fe319b 1880b4a9 .config console log report info ci-upstream-net-kasan-gce
2020/10/06 20:16 net-next-old 8b0308fe319b 1880b4a9 .config console log report info ci-upstream-net-kasan-gce
2020/10/06 13:33 net-next-old 8b0308fe319b 1880b4a9 .config console log report info ci-upstream-net-kasan-gce
* Struck through repros no longer work on HEAD.