syzbot


possible deadlock in free_huge_page

Status: upstream: reported C repro on 2020/12/15 10:55
Reported-by: syzbot+2054819d6fda52ce059e@syzkaller.appspotmail.com
First crash: 1221d, last: 939d
Fix bisection: failed (error log, bisect log)
  
Similar bugs (1)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.14 possible deadlock in free_huge_page C 4 425d 1229d 0/1 upstream: reported C repro on 2020/12/07 03:16
Fix bisection attempts (10)
Created Duration User Patch Repo Result
2021/10/28 09:52 16m bisect fix linux-4.19.y error job log (0)
2021/09/23 07:22 24m bisect fix linux-4.19.y job log (0) log
2021/08/24 06:39 26m bisect fix linux-4.19.y job log (0) log
2021/07/25 02:09 23m bisect fix linux-4.19.y job log (0) log
2021/06/24 23:53 23m bisect fix linux-4.19.y job log (0) log
2021/05/25 22:41 26m bisect fix linux-4.19.y job log (0) log
2021/04/25 22:09 31m bisect fix linux-4.19.y job log (0) log
2021/03/26 20:07 22m bisect fix linux-4.19.y job log (0) log
2021/02/24 17:35 24m bisect fix linux-4.19.y job log (0) log
2021/01/25 16:53 23m bisect fix linux-4.19.y job log (0) log

Sample crash report:
nf_conntrack: default automatic helper assignment has been turned off for security reasons and CT-based  firewall rule not found. Use the iptables CT target to attach helpers instead.
=====================================================
WARNING: SOFTIRQ-safe -> SOFTIRQ-unsafe lock order detected
4.19.163-syzkaller #0 Not tainted
-----------------------------------------------------
syz-executor483/8114 [HC0[0]:SC0[2]:HE1:SE0] is trying to acquire:
00000000ee0b61b2 (hugetlb_lock){+.+.}, at: spin_lock include/linux/spinlock.h:329 [inline]
00000000ee0b61b2 (hugetlb_lock){+.+.}, at: free_huge_page+0x482/0xd20 mm/hugetlb.c:1276

and this task is already holding:
00000000864bec6d (slock-AF_INET){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline]
00000000864bec6d (slock-AF_INET){+.-.}, at: tcp_close+0x5bd/0xfd0 net/ipv4/tcp.c:2434
which would create a new lock dependency:
 (slock-AF_INET){+.-.} -> (hugetlb_lock){+.+.}

but this new dependency connects a SOFTIRQ-irq-safe lock:
 (slock-AF_INET){+.-.}

... which became SOFTIRQ-irq-safe at:
  __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
  _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
  spin_lock include/linux/spinlock.h:329 [inline]
  sk_clone_lock+0x40b/0x1430 net/core/sock.c:1671
  inet_csk_clone_lock+0x1f/0x3e0 net/ipv4/inet_connection_sock.c:821
  tcp_create_openreq_child+0x2c/0x19f0 net/ipv4/tcp_minisocks.c:452
  tcp_v4_syn_recv_sock+0xb6/0x1030 net/ipv4/tcp_ipv4.c:1426
  tcp_check_req+0x601/0x16b0 net/ipv4/tcp_minisocks.c:789
  tcp_v4_rcv+0x1e3c/0x3b80 net/ipv4/tcp_ipv4.c:1770
  ip_local_deliver_finish+0x495/0xc00 net/ipv4/ip_input.c:215
  NF_HOOK include/linux/netfilter.h:289 [inline]
  ip_local_deliver+0x188/0x500 net/ipv4/ip_input.c:256
  dst_input include/net/dst.h:461 [inline]
  ip_rcv_finish+0x1ca/0x2e0 net/ipv4/ip_input.c:414
  NF_HOOK include/linux/netfilter.h:289 [inline]
  ip_rcv+0xca/0x3c0 net/ipv4/ip_input.c:524
  __netif_receive_skb_one_core+0x114/0x180 net/core/dev.c:4954
  __netif_receive_skb+0x27/0x1c0 net/core/dev.c:5066
  netif_receive_skb_internal+0xf0/0x3f0 net/core/dev.c:5156
  napi_skb_finish net/core/dev.c:5600 [inline]
  napi_gro_receive+0x2e6/0x450 net/core/dev.c:5631
  receive_buf+0xf1d/0x6120 drivers/net/virtio_net.c:1072
  virtnet_receive drivers/net/virtio_net.c:1336 [inline]
  virtnet_poll+0x568/0xd70 drivers/net/virtio_net.c:1441
  napi_poll net/core/dev.c:6272 [inline]
  net_rx_action+0x4ac/0xfb0 net/core/dev.c:6338
  __do_softirq+0x26c/0x9a0 kernel/softirq.c:292
  run_ksoftirqd+0x57/0x110 kernel/softirq.c:653
  smpboot_thread_fn+0x655/0x9e0 kernel/smpboot.c:164
  kthread+0x33f/0x460 kernel/kthread.c:259
  ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:415

to a SOFTIRQ-irq-unsafe lock:
 (hugetlb_lock){+.+.}

... which became SOFTIRQ-irq-unsafe at:
...
  __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
  _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
  spin_lock include/linux/spinlock.h:329 [inline]
  hugetlb_overcommit_handler+0x2d4/0x460 mm/hugetlb.c:2999
  proc_sys_call_handler.isra.0+0x1f3/0x3b0 fs/proc/proc_sysctl.c:597
  __vfs_write+0xf7/0x770 fs/read_write.c:485
  vfs_write+0x1f3/0x540 fs/read_write.c:549
  ksys_write+0x12b/0x2a0 fs/read_write.c:599
  do_syscall_64+0xf9/0x620 arch/x86/entry/common.c:293
  entry_SYSCALL_64_after_hwframe+0x49/0xbe

other info that might help us debug this:

 Possible interrupt unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(hugetlb_lock);
                               local_irq_disable();
                               lock(slock-AF_INET);
                               lock(hugetlb_lock);
  <Interrupt>
    lock(slock-AF_INET);

 *** DEADLOCK ***

3 locks held by syz-executor483/8114:
 #0: 000000001e5e6bcd (&sb->s_type->i_mutex_key#13){+.+.}, at: inode_lock include/linux/fs.h:748 [inline]
 #0: 000000001e5e6bcd (&sb->s_type->i_mutex_key#13){+.+.}, at: __sock_release+0x86/0x2a0 net/socket.c:578
 #1: 00000000ae8cebee (sk_lock-AF_INET){+.+.}, at: lock_sock include/net/sock.h:1510 [inline]
 #1: 00000000ae8cebee (sk_lock-AF_INET){+.+.}, at: tcp_close+0x25/0xfd0 net/ipv4/tcp.c:2344
 #2: 00000000864bec6d (slock-AF_INET){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline]
 #2: 00000000864bec6d (slock-AF_INET){+.-.}, at: tcp_close+0x5bd/0xfd0 net/ipv4/tcp.c:2434

the dependencies between SOFTIRQ-irq-safe lock and the holding lock:
-> (slock-AF_INET){+.-.} ops: 6455 {
   HARDIRQ-ON-W at:
                    __raw_spin_lock_bh include/linux/spinlock_api_smp.h:135 [inline]
                    _raw_spin_lock_bh+0x2f/0x40 kernel/locking/spinlock.c:168
                    spin_lock_bh include/linux/spinlock.h:334 [inline]
                    lock_sock_nested+0x3b/0x110 net/core/sock.c:2864
                    lock_sock include/net/sock.h:1510 [inline]
                    inet_autobind+0x1a/0x190 net/ipv4/af_inet.c:179
                    inet_dgram_connect+0x245/0x2d0 net/ipv4/af_inet.c:569
                    __sys_connect+0x265/0x2c0 net/socket.c:1663
                    __do_sys_connect net/socket.c:1674 [inline]
                    __se_sys_connect net/socket.c:1671 [inline]
                    __x64_sys_connect+0x6f/0xb0 net/socket.c:1671
                    do_syscall_64+0xf9/0x620 arch/x86/entry/common.c:293
                    entry_SYSCALL_64_after_hwframe+0x49/0xbe
   IN-SOFTIRQ-W at:
                    __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
                    _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
                    spin_lock include/linux/spinlock.h:329 [inline]
                    sk_clone_lock+0x40b/0x1430 net/core/sock.c:1671
                    inet_csk_clone_lock+0x1f/0x3e0 net/ipv4/inet_connection_sock.c:821
                    tcp_create_openreq_child+0x2c/0x19f0 net/ipv4/tcp_minisocks.c:452
                    tcp_v4_syn_recv_sock+0xb6/0x1030 net/ipv4/tcp_ipv4.c:1426
                    tcp_check_req+0x601/0x16b0 net/ipv4/tcp_minisocks.c:789
                    tcp_v4_rcv+0x1e3c/0x3b80 net/ipv4/tcp_ipv4.c:1770
                    ip_local_deliver_finish+0x495/0xc00 net/ipv4/ip_input.c:215
                    NF_HOOK include/linux/netfilter.h:289 [inline]
                    ip_local_deliver+0x188/0x500 net/ipv4/ip_input.c:256
                    dst_input include/net/dst.h:461 [inline]
                    ip_rcv_finish+0x1ca/0x2e0 net/ipv4/ip_input.c:414
                    NF_HOOK include/linux/netfilter.h:289 [inline]
                    ip_rcv+0xca/0x3c0 net/ipv4/ip_input.c:524
                    __netif_receive_skb_one_core+0x114/0x180 net/core/dev.c:4954
                    __netif_receive_skb+0x27/0x1c0 net/core/dev.c:5066
                    netif_receive_skb_internal+0xf0/0x3f0 net/core/dev.c:5156
                    napi_skb_finish net/core/dev.c:5600 [inline]
                    napi_gro_receive+0x2e6/0x450 net/core/dev.c:5631
                    receive_buf+0xf1d/0x6120 drivers/net/virtio_net.c:1072
                    virtnet_receive drivers/net/virtio_net.c:1336 [inline]
                    virtnet_poll+0x568/0xd70 drivers/net/virtio_net.c:1441
                    napi_poll net/core/dev.c:6272 [inline]
                    net_rx_action+0x4ac/0xfb0 net/core/dev.c:6338
                    __do_softirq+0x26c/0x9a0 kernel/softirq.c:292
                    run_ksoftirqd+0x57/0x110 kernel/softirq.c:653
                    smpboot_thread_fn+0x655/0x9e0 kernel/smpboot.c:164
                    kthread+0x33f/0x460 kernel/kthread.c:259
                    ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:415
   INITIAL USE at:
                   __raw_spin_lock_bh include/linux/spinlock_api_smp.h:135 [inline]
                   _raw_spin_lock_bh+0x2f/0x40 kernel/locking/spinlock.c:168
                   spin_lock_bh include/linux/spinlock.h:334 [inline]
                   lock_sock_nested+0x3b/0x110 net/core/sock.c:2864
                   lock_sock include/net/sock.h:1510 [inline]
                   inet_autobind+0x1a/0x190 net/ipv4/af_inet.c:179
                   inet_dgram_connect+0x245/0x2d0 net/ipv4/af_inet.c:569
                   __sys_connect+0x265/0x2c0 net/socket.c:1663
                   __do_sys_connect net/socket.c:1674 [inline]
                   __se_sys_connect net/socket.c:1671 [inline]
                   __x64_sys_connect+0x6f/0xb0 net/socket.c:1671
                   do_syscall_64+0xf9/0x620 arch/x86/entry/common.c:293
                   entry_SYSCALL_64_after_hwframe+0x49/0xbe
 }
 ... key      at: [<ffffffff8dd6c510>] af_family_slock_keys+0x10/0x1a0
 ... acquired at:
   __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
   _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
   spin_lock include/linux/spinlock.h:329 [inline]
   free_huge_page+0x482/0xd20 mm/hugetlb.c:1276
   __put_page+0xe2/0x3a0 mm/swap.c:112
   put_page include/linux/mm.h:963 [inline]
   __skb_frag_unref include/linux/skbuff.h:2828 [inline]
   skb_release_data+0x2f3/0x920 net/core/skbuff.c:568
   skb_release_all net/core/skbuff.c:631 [inline]
   __kfree_skb+0x46/0x60 net/core/skbuff.c:645
   sk_wmem_free_skb include/net/sock.h:1466 [inline]
   tcp_write_queue_purge+0x24d/0x800 net/ipv4/tcp.c:2543
   tcp_v4_destroy_sock+0x101/0x770 net/ipv4/tcp_ipv4.c:1986
   inet_csk_destroy_sock+0x189/0x400 net/ipv4/inet_connection_sock.c:872
   tcp_close+0x95f/0xfd0 net/ipv4/tcp.c:2498
   inet_release+0xd7/0x1e0 net/ipv4/af_inet.c:427
   __sock_release+0xcd/0x2a0 net/socket.c:579
   sock_close+0x15/0x20 net/socket.c:1140
   __fput+0x2ce/0x890 fs/file_table.c:278
   task_work_run+0x148/0x1c0 kernel/task_work.c:113
   exit_task_work include/linux/task_work.h:22 [inline]
   do_exit+0xbed/0x2be0 kernel/exit.c:890
   do_group_exit+0x125/0x310 kernel/exit.c:993
   get_signal+0x3f2/0x1f70 kernel/signal.c:2589
   do_signal+0x8f/0x1670 arch/x86/kernel/signal.c:821
   exit_to_usermode_loop+0x204/0x2a0 arch/x86/entry/common.c:163
   prepare_exit_to_usermode+0x277/0x2d0 arch/x86/entry/common.c:198
   retint_user+0x8/0x18


the dependencies between the lock to be acquired
 and SOFTIRQ-irq-unsafe lock:
-> (hugetlb_lock){+.+.} ops: 30 {
   HARDIRQ-ON-W at:
                    __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
                    _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
                    spin_lock include/linux/spinlock.h:329 [inline]
                    hugetlb_overcommit_handler+0x2d4/0x460 mm/hugetlb.c:2999
                    proc_sys_call_handler.isra.0+0x1f3/0x3b0 fs/proc/proc_sysctl.c:597
                    __vfs_write+0xf7/0x770 fs/read_write.c:485
                    vfs_write+0x1f3/0x540 fs/read_write.c:549
                    ksys_write+0x12b/0x2a0 fs/read_write.c:599
                    do_syscall_64+0xf9/0x620 arch/x86/entry/common.c:293
                    entry_SYSCALL_64_after_hwframe+0x49/0xbe
   SOFTIRQ-ON-W at:
                    __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
                    _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
                    spin_lock include/linux/spinlock.h:329 [inline]
                    hugetlb_overcommit_handler+0x2d4/0x460 mm/hugetlb.c:2999
                    proc_sys_call_handler.isra.0+0x1f3/0x3b0 fs/proc/proc_sysctl.c:597
                    __vfs_write+0xf7/0x770 fs/read_write.c:485
                    vfs_write+0x1f3/0x540 fs/read_write.c:549
                    ksys_write+0x12b/0x2a0 fs/read_write.c:599
                    do_syscall_64+0xf9/0x620 arch/x86/entry/common.c:293
                    entry_SYSCALL_64_after_hwframe+0x49/0xbe
   INITIAL USE at:
                   __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
                   _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
                   spin_lock include/linux/spinlock.h:329 [inline]
                   hugetlb_overcommit_handler+0x2d4/0x460 mm/hugetlb.c:2999
                   proc_sys_call_handler.isra.0+0x1f3/0x3b0 fs/proc/proc_sysctl.c:597
                   __vfs_write+0xf7/0x770 fs/read_write.c:485
                   vfs_write+0x1f3/0x540 fs/read_write.c:549
                   ksys_write+0x12b/0x2a0 fs/read_write.c:599
                   do_syscall_64+0xf9/0x620 arch/x86/entry/common.c:293
                   entry_SYSCALL_64_after_hwframe+0x49/0xbe
 }
 ... key      at: [<ffffffff8a020978>] hugetlb_lock+0x18/0x17a0
 ... acquired at:
   __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
   _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
   spin_lock include/linux/spinlock.h:329 [inline]
   free_huge_page+0x482/0xd20 mm/hugetlb.c:1276
   __put_page+0xe2/0x3a0 mm/swap.c:112
   put_page include/linux/mm.h:963 [inline]
   __skb_frag_unref include/linux/skbuff.h:2828 [inline]
   skb_release_data+0x2f3/0x920 net/core/skbuff.c:568
   skb_release_all net/core/skbuff.c:631 [inline]
   __kfree_skb+0x46/0x60 net/core/skbuff.c:645
   sk_wmem_free_skb include/net/sock.h:1466 [inline]
   tcp_write_queue_purge+0x24d/0x800 net/ipv4/tcp.c:2543
   tcp_v4_destroy_sock+0x101/0x770 net/ipv4/tcp_ipv4.c:1986
   inet_csk_destroy_sock+0x189/0x400 net/ipv4/inet_connection_sock.c:872
   tcp_close+0x95f/0xfd0 net/ipv4/tcp.c:2498
   inet_release+0xd7/0x1e0 net/ipv4/af_inet.c:427
   __sock_release+0xcd/0x2a0 net/socket.c:579
   sock_close+0x15/0x20 net/socket.c:1140
   __fput+0x2ce/0x890 fs/file_table.c:278
   task_work_run+0x148/0x1c0 kernel/task_work.c:113
   exit_task_work include/linux/task_work.h:22 [inline]
   do_exit+0xbed/0x2be0 kernel/exit.c:890
   do_group_exit+0x125/0x310 kernel/exit.c:993
   get_signal+0x3f2/0x1f70 kernel/signal.c:2589
   do_signal+0x8f/0x1670 arch/x86/kernel/signal.c:821
   exit_to_usermode_loop+0x204/0x2a0 arch/x86/entry/common.c:163
   prepare_exit_to_usermode+0x277/0x2d0 arch/x86/entry/common.c:198
   retint_user+0x8/0x18


stack backtrace:
CPU: 0 PID: 8114 Comm: syz-executor483 Not tainted 4.19.163-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x1fc/0x2fe lib/dump_stack.c:118
 print_bad_irq_dependency kernel/locking/lockdep.c:1572 [inline]
 check_usage.cold+0x7ea/0xbad kernel/locking/lockdep.c:1604
 check_irq_usage kernel/locking/lockdep.c:1660 [inline]
 check_prev_add_irq kernel/locking/lockdep_states.h:8 [inline]
 check_prev_add kernel/locking/lockdep.c:1870 [inline]
 check_prevs_add kernel/locking/lockdep.c:1978 [inline]
 validate_chain kernel/locking/lockdep.c:2419 [inline]
 __lock_acquire+0x1da1/0x3ff0 kernel/locking/lockdep.c:3415
 lock_acquire+0x170/0x3c0 kernel/locking/lockdep.c:3907
 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
 _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
 spin_lock include/linux/spinlock.h:329 [inline]
 free_huge_page+0x482/0xd20 mm/hugetlb.c:1276
 __put_page+0xe2/0x3a0 mm/swap.c:112
 put_page include/linux/mm.h:963 [inline]
 __skb_frag_unref include/linux/skbuff.h:2828 [inline]
 skb_release_data+0x2f3/0x920 net/core/skbuff.c:568
 skb_release_all net/core/skbuff.c:631 [inline]
 __kfree_skb+0x46/0x60 net/core/skbuff.c:645
 sk_wmem_free_skb include/net/sock.h:1466 [inline]
 tcp_write_queue_purge+0x24d/0x800 net/ipv4/tcp.c:2543
 tcp_v4_destroy_sock+0x101/0x770 net/ipv4/tcp_ipv4.c:1986
 inet_csk_destroy_sock+0x189/0x400 net/ipv4/inet_connection_sock.c:872
 tcp_close+0x95f/0xfd0 net/ipv4/tcp.c:2498
 inet_release+0xd7/0x1e0 net/ipv4/af_inet.c:427
 __sock_release+0xcd/0x2a0 net/socket.c:579
 sock_close+0x15/0x20 net/socket.c:1140
 __fput+0x2ce/0x890 fs/file_table.c:278
 task_work_run+0x148/0x1c0 kernel/task_work.c:113
 exit_task_work include/linux/task_work.h:22 [inline]
 do_exit+0xbed/0x2be0 kernel/exit.c:890
 do_group_exit+0x125/0x310 kernel/exit.c:993
 get_signal+0x3f2/0x1f70 kernel/signal.c:2589
 do_signal+0x8f/0x1670 arch/x86/kernel/signal.c:821
 exit_to_usermode_loop+0x204/0x2a0 arch/x86/entry/common.c:163
 prepare_exit_to_usermode+0x277/0x2d0 arch/x86/entry/common.c:198
 retint_user+0x8/0x18
RIP: 0033:0x401072
Code: Bad RIP value.
RSP: 002b:00007f2a80855da0 EFLAGS: 00010293
RAX: 0000000000000003 RBX: 00000000006dbc48 RCX: 0000000000446b39
RDX: 0000000000000000 RSI: 0000000000000006 RDI: 0000000000000002
RBP: 00000000006dbc40 R08: 000000000000204e R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000006dbc4c
R13: 44d9effa3f410d0d R14: b1c59e31819ff2c8 R15: 8effb11713110d7c

Crashes (3):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2020/12/26 16:44 linux-4.19.y 13d2ce42de8c 821e0b09 .config console log report syz C ci2-linux-4-19
2020/12/15 11:13 linux-4.19.y 13d2ce42de8c 97183ed7 .config console log report syz C ci2-linux-4-19
2020/12/15 10:54 linux-4.19.y 13d2ce42de8c 97183ed7 .config console log report info ci2-linux-4-19
* Struck through repros no longer work on HEAD.