DATA RACE in stack.(*PacketBuffer).reset

Status: fixed on 2021/11/11 02:22
Fix commit: bb1ae811f4eb Prevent PacketBuffers from being returned to the pool too early in nic.
First crash: 1752d, last: 1752d
▶ ▼ Similar bugs (1)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
gvisor DATA RACE in stack.(*PacketBuffer).reset (2) -1 syz 2 1694d 1694d 14/26 fixed on 2022/02/01 22:27

Sample crash report:
WARNING: DATA RACE
Write at 0x00c08cd9d9e8 by goroutine 2279:
  gvisor.dev/gvisor/pkg/tcpip/stack.(*PacketBuffer).reset()
      pkg/tcpip/stack/packet_buffer.go:201 +0x78
  gvisor.dev/gvisor/pkg/tcpip/stack.NewPacketBuffer()
      pkg/tcpip/stack/packet_buffer.go:167 +0x66
  gvisor.dev/gvisor/pkg/tcpip/stack.(*nic).deliverOutboundPacket.func1()
      pkg/tcpip/stack/nic.go:813 +0x2e4
  gvisor.dev/gvisor/pkg/tcpip/stack.(*packetEndpointList).forEach()
      pkg/tcpip/stack/nic.go:133 +0xf5
  gvisor.dev/gvisor/pkg/tcpip/stack.(*nic).deliverOutboundPacket()
      pkg/tcpip/stack/nic.go:802 +0x214
  gvisor.dev/gvisor/pkg/tcpip/stack.(*nic).writePacket()
      pkg/tcpip/stack/nic.go:380 +0x1c6
  gvisor.dev/gvisor/pkg/tcpip/stack.(*nic).writePacketBuffer()
      pkg/tcpip/stack/nic.go:326 +0x111
  gvisor.dev/gvisor/pkg/tcpip/stack.(*nic).enqueuePacketBuffer()
      pkg/tcpip/stack/nic.go:341 +0x2dc
  gvisor.dev/gvisor/pkg/tcpip/stack.(*nic).WritePacket()
      pkg/tcpip/stack/nic.go:319 +0x64
  gvisor.dev/gvisor/pkg/tcpip/network/ipv6.(*endpoint).writePacket()
      pkg/tcpip/network/ipv6/ipv6.go:824 +0x4ab
  gvisor.dev/gvisor/pkg/tcpip/network/ipv6.(*endpoint).WritePacket()
      pkg/tcpip/network/ipv6/ipv6.go:775 +0x2cc
  gvisor.dev/gvisor/pkg/tcpip/stack.(*Route).WritePacket()
      pkg/tcpip/stack/route.go:462 +0x11b
  gvisor.dev/gvisor/pkg/tcpip/transport/internal/network.(*WriteContext).WritePacket()
      pkg/tcpip/transport/internal/network/endpoint.go:248 +0x19a
  gvisor.dev/gvisor/pkg/tcpip/transport/udp.(*endpoint).write()
      pkg/tcpip/transport/udp/endpoint.go:467 +0x544
  gvisor.dev/gvisor/pkg/tcpip/transport/udp.(*endpoint).Write()
      pkg/tcpip/transport/udp/endpoint.go:332 +0x84
  gvisor.dev/gvisor/pkg/sentry/socket/netstack.(*SocketVFS2).Write()
      pkg/sentry/socket/netstack/netstack_vfs2.go:132 +0x19c
  gvisor.dev/gvisor/pkg/sentry/vfs.(*FileDescription).Write()
      pkg/sentry/vfs/file_description.go:657 +0x130
  gvisor.dev/gvisor/pkg/sentry/syscalls/linux/vfs2.write()
      pkg/sentry/syscalls/linux/vfs2/read_write.go:345 +0xed
  gvisor.dev/gvisor/pkg/sentry/syscalls/linux/vfs2.Write()
      pkg/sentry/syscalls/linux/vfs2/read_write.go:314 +0x307
  gvisor.dev/gvisor/pkg/sentry/kernel.(*Task).executeSyscall()
      pkg/sentry/kernel/task_syscall.go:103 +0x452
  gvisor.dev/gvisor/pkg/sentry/kernel.(*Task).doSyscallInvoke()
      pkg/sentry/kernel/task_syscall.go:238 +0xb3
  gvisor.dev/gvisor/pkg/sentry/kernel.(*Task).doSyscallEnter()
      pkg/sentry/kernel/task_syscall.go:198 +0x10e
  gvisor.dev/gvisor/pkg/sentry/kernel.(*Task).doSyscall()
      pkg/sentry/kernel/task_syscall.go:173 +0x213
  gvisor.dev/gvisor/pkg/sentry/kernel.(*runApp).execute()
      pkg/sentry/kernel/task_run.go:282 +0x12c6
  gvisor.dev/gvisor/pkg/sentry/kernel.(*Task).run()
      pkg/sentry/kernel/task_run.go:97 +0x352

Previous read at 0x00c08cd9d9e8 by goroutine 2533:
  gvisor.dev/gvisor/pkg/tcpip/stack.(*PacketBuffer).Clone()
      pkg/tcpip/stack/packet_buffer.go:327 +0x93
  gvisor.dev/gvisor/pkg/tcpip/stack.(*nic).DeliverNetworkPacket.func1()
      pkg/tcpip/stack/nic.go:765 +0xca
  gvisor.dev/gvisor/pkg/tcpip/stack.(*packetEndpointList).forEach()
      pkg/tcpip/stack/nic.go:133 +0xf5
  gvisor.dev/gvisor/pkg/tcpip/stack.(*nic).DeliverNetworkPacket()
      pkg/tcpip/stack/nic.go:781 +0x477
  gvisor.dev/gvisor/pkg/tcpip/link/nested.(*Endpoint).DeliverNetworkPacket()
      pkg/tcpip/link/nested/nested.go:59 +0xef
  gvisor.dev/gvisor/pkg/tcpip/link/sniffer.(*endpoint).DeliverNetworkPacket()
      pkg/tcpip/link/sniffer/sniffer.go:140 +0xa4
  gvisor.dev/gvisor/pkg/tcpip/link/nested.(*Endpoint).DeliverNetworkPacket()
      pkg/tcpip/link/nested/nested.go:59 +0xef
  gvisor.dev/gvisor/pkg/tcpip/link/ethernet.(*Endpoint).DeliverNetworkPacket()
      pkg/tcpip/link/ethernet/ethernet.go:63 +0x224
  gvisor.dev/gvisor/pkg/tcpip/link/loopback.(*endpoint).WriteRawPacket()
      pkg/tcpip/link/loopback/loopback.go:108 +0x1f5
  gvisor.dev/gvisor/pkg/tcpip/link/loopback.(*endpoint).WritePacket()
      pkg/tcpip/link/loopback/loopback.go:80 +0x45
  gvisor.dev/gvisor/pkg/tcpip/link/nested.(*Endpoint).WritePacket()
      pkg/tcpip/link/nested/nested.go:107 +0x1cc
  gvisor.dev/gvisor/pkg/tcpip/link/ethernet.(*Endpoint).WritePacket()
      pkg/tcpip/link/ethernet/ethernet.go:78 +0xd6
  gvisor.dev/gvisor/pkg/tcpip/link/nested.(*Endpoint).WritePacket()
      pkg/tcpip/link/nested/nested.go:107 +0x16c
  gvisor.dev/gvisor/pkg/tcpip/link/sniffer.(*endpoint).WritePacket()
      pkg/tcpip/link/sniffer/sniffer.go:169 +0x77
  gvisor.dev/gvisor/pkg/tcpip/stack.(*nic).writePacket()
      pkg/tcpip/stack/nic.go:382 +0x28c
  gvisor.dev/gvisor/pkg/tcpip/stack.(*nic).writePacketBuffer()
      pkg/tcpip/stack/nic.go:326 +0x111
  gvisor.dev/gvisor/pkg/tcpip/stack.(*nic).enqueuePacketBuffer()
      pkg/tcpip/stack/nic.go:341 +0x2dc
  gvisor.dev/gvisor/pkg/tcpip/stack.(*nic).WritePacket()
      pkg/tcpip/stack/nic.go:319 +0x64
  gvisor.dev/gvisor/pkg/tcpip/network/ipv6.(*endpoint).writePacket()
      pkg/tcpip/network/ipv6/ipv6.go:824 +0x4ab
  gvisor.dev/gvisor/pkg/tcpip/network/ipv6.(*endpoint).WritePacket()
      pkg/tcpip/network/ipv6/ipv6.go:775 +0x2cc
  gvisor.dev/gvisor/pkg/tcpip/stack.(*Route).WritePacket()
      pkg/tcpip/stack/route.go:462 +0x11b
  gvisor.dev/gvisor/pkg/tcpip/transport/internal/network.(*WriteContext).WritePacket()
      pkg/tcpip/transport/internal/network/endpoint.go:248 +0x19a
  gvisor.dev/gvisor/pkg/tcpip/transport/udp.(*endpoint).write()
      pkg/tcpip/transport/udp/endpoint.go:467 +0x544
  gvisor.dev/gvisor/pkg/tcpip/transport/udp.(*endpoint).Write()
      pkg/tcpip/transport/udp/endpoint.go:332 +0x84
  gvisor.dev/gvisor/pkg/sentry/socket/netstack.(*SocketVFS2).Write()
      pkg/sentry/socket/netstack/netstack_vfs2.go:132 +0x19c
  gvisor.dev/gvisor/pkg/sentry/vfs.(*FileDescription).Write()
      pkg/sentry/vfs/file_description.go:657 +0x130
  gvisor.dev/gvisor/pkg/sentry/syscalls/linux/vfs2.write()
      pkg/sentry/syscalls/linux/vfs2/read_write.go:345 +0xed
  gvisor.dev/gvisor/pkg/sentry/syscalls/linux/vfs2.Write()
      pkg/sentry/syscalls/linux/vfs2/read_write.go:314 +0x307
  gvisor.dev/gvisor/pkg/sentry/kernel.(*Task).executeSyscall()
      pkg/sentry/kernel/task_syscall.go:103 +0x452
  gvisor.dev/gvisor/pkg/sentry/kernel.(*Task).doSyscallInvoke()
      pkg/sentry/kernel/task_syscall.go:238 +0xb3
  gvisor.dev/gvisor/pkg/sentry/kernel.(*Task).doSyscallEnter()
      pkg/sentry/kernel/task_syscall.go:198 +0x10e
  gvisor.dev/gvisor/pkg/sentry/kernel.(*Task).doSyscall()
      pkg/sentry/kernel/task_syscall.go:173 +0x213
  gvisor.dev/gvisor/pkg/sentry/kernel.(*runApp).execute()
      pkg/sentry/kernel/task_run.go:282 +0x12c6
  gvisor.dev/gvisor/pkg/sentry/kernel.(*Task).run()
      pkg/sentry/kernel/task_run.go:97 +0x352

Crashes (9):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2021/11/11 01:42 gvisor 37792ee1e6e1 75b04091 .config console log report info ci-gvisor-ptrace-1-race DATA RACE in stack.(*PacketBuffer).reset
2021/11/11 00:50 gvisor 37792ee1e6e1 75b04091 .config console log report info ci-gvisor-ptrace-2-race DATA RACE in stack.(*PacketBuffer).reset
2021/11/10 23:52 gvisor 37792ee1e6e1 75b04091 .config console log report info ci-gvisor-ptrace-1-race DATA RACE in stack.(*PacketBuffer).reset
2021/11/10 22:42 gvisor 37792ee1e6e1 75b04091 .config console log report info ci-gvisor-ptrace-2-race DATA RACE in stack.(*PacketBuffer).reset
2021/11/10 22:16 gvisor 37792ee1e6e1 75b04091 .config console log report info ci-gvisor-ptrace-1-race DATA RACE in stack.(*PacketBuffer).reset
2021/11/10 18:17 gvisor 37792ee1e6e1 75b04091 .config console log report info ci-gvisor-ptrace-1-race DATA RACE in stack.(*PacketBuffer).reset
2021/11/10 17:09 gvisor 37792ee1e6e1 75b04091 .config console log report info ci-gvisor-ptrace-2-race DATA RACE in stack.(*PacketBuffer).reset
2021/11/10 16:58 gvisor 37792ee1e6e1 75b04091 .config console log report info ci-gvisor-ptrace-2-race DATA RACE in stack.(*PacketBuffer).reset
2021/11/10 16:27 gvisor 37792ee1e6e1 75b04091 .config console log report info ci-gvisor-ptrace-2-race DATA RACE in stack.(*PacketBuffer).reset
* Struck through repros no longer work on HEAD.