syzbot


possible deadlock in ppp_do_recv

Status: upstream: reported on 2024/09/25 02:48
Reported-by: syzbot+36d5f3b70054c192fafa@syzkaller.appspotmail.com
First crash: 87d, last: 66d
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-5.15 possible deadlock in ppp_do_recv 1 83d 83d 0/3 upstream: reported on 2024/09/28 15:36
upstream possible deadlock in ppp_do_recv ppp 8 74d 89d 28/28 fixed on 2024/11/12 23:31

Sample crash report:
========================================================
WARNING: possible irq lock inversion dependency detected
6.1.111-syzkaller #0 Not tainted
--------------------------------------------------------
ksoftirqd/1/21 just changed the state of lock:
ffff88805c2f0dd0 (&ppp->rlock){+.-.}-{2:2}, at: spin_lock_bh include/linux/spinlock.h:356 [inline]
ffff88805c2f0dd0 (&ppp->rlock){+.-.}-{2:2}, at: ppp_do_recv+0x38/0x26a0 drivers/net/ppp/ppp_generic.c:2210
but this lock took another, SOFTIRQ-unsafe lock in the past:
 (&pch->downl){+.+.}-{2:2}


and interrupts could create inverse lock ordering between them.


other info that might help us debug this:
 Possible interrupt unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&pch->downl);
                               local_irq_disable();
                               lock(&ppp->rlock);
                               lock(&pch->downl);
  <Interrupt>
    lock(&ppp->rlock);

 *** DEADLOCK ***

4 locks held by ksoftirqd/1/21:
 #0: ffffffff8d32b000 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire include/linux/rcupdate.h:350 [inline]
 #0: ffffffff8d32b000 (rcu_read_lock){....}-{1:2}, at: rcu_read_lock include/linux/rcupdate.h:791 [inline]
 #0: ffffffff8d32b000 (rcu_read_lock){....}-{1:2}, at: process_backlog+0x2c9/0x760 net/core/dev.c:5969
 #1: ffff88805c7990b0 (slock-AF_PPPOX){+.-.}-{2:2}, at: spin_lock include/linux/spinlock.h:351 [inline]
 #1: ffff88805c7990b0 (slock-AF_PPPOX){+.-.}-{2:2}, at: __sk_receive_skb+0x188/0x9d0 net/core/sock.c:562
 #2: ffff88805c799130 (sk_lock-AF_PPPOX){+.+.}-{0:0}, at: __netif_receive_skb_one_core net/core/dev.c:5528 [inline]
 #2: ffff88805c799130 (sk_lock-AF_PPPOX){+.+.}-{0:0}, at: __netif_receive_skb+0x1c6/0x530 net/core/dev.c:5642
 #3: ffff888026f3e248 (&pch->upl){++.-}-{2:2}, at: ppp_input+0x3ca/0xa00 drivers/net/ppp/ppp_generic.c:2307

the shortest dependencies between 2nd lock and 1st lock:
 -> (&pch->downl){+.+.}-{2:2} {
    HARDIRQ-ON-W at:
                      lock_acquire+0x1f8/0x5a0 kernel/locking/lockdep.c:5662
                      __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
                      _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:154
                      spin_lock include/linux/spinlock.h:351 [inline]
                      __ppp_channel_push+0x31/0x1d0 drivers/net/ppp/ppp_generic.c:2161
                      ppp_channel_push+0x196/0x220 drivers/net/ppp/ppp_generic.c:2192
                      ppp_write+0x294/0x3a0 drivers/net/ppp/ppp_generic.c:538
                      do_iter_write+0x503/0xc40 fs/read_write.c:863
                      vfs_writev fs/read_write.c:934 [inline]
                      do_pwritev+0x216/0x360 fs/read_write.c:1031
                      do_syscall_x64 arch/x86/entry/common.c:51 [inline]
                      do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:81
                      entry_SYSCALL_64_after_hwframe+0x68/0xd2
    SOFTIRQ-ON-W
 at:
                      lock_acquire+0x1f8/0x5a0 kernel/locking/lockdep.c:5662
                      __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
                      _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:154
                      spin_lock include/linux/spinlock.h:351 [inline]
                      ppp_channel_bridge_input drivers/net/ppp/ppp_generic.c:2272 [inline]
                      ppp_input+0x185/0xa00 drivers/net/ppp/ppp_generic.c:2304
                      pppoe_rcv_core+0x112/0x300 drivers/net/ppp/pppoe.c:379
                      sk_backlog_rcv include/net/sock.h:1117 [inline]
                      __release_sock+0x198/0x4b0 net/core/sock.c:2936
                      release_sock+0x5d/0x1c0 net/core/sock.c:3500
                      pppoe_sendmsg+0xd1/0x740 drivers/net/ppp/pppoe.c:903
                      sock_sendmsg_nosec net/socket.c:718 [inline]
                      __sock_sendmsg net/socket.c:730 [inline]
                      ____sys_sendmsg+0x5a5/0x8f0 net/socket.c:2514
                      ___sys_sendmsg net/socket.c:2568 [inline]
                      __sys_sendmmsg+0x3ab/0x730 net/socket.c:2654
                      __do_sys_sendmmsg net/socket.c:2683 [inline]
                      __se_sys_sendmmsg net/socket.c:2680 [inline]
                      __x64_sys_sendmmsg+0x9c/0xb0 net/socket.c:2680
                      do_syscall_x64 arch/x86/entry/common.c:51 [inline]
                      do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:81
                      entry_SYSCALL_64_after_hwframe+0x68/0xd2
    INITIAL USE at:
                     lock_acquire+0x1f8/0x5a0 kernel/locking/lockdep.c:5662
                     __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
                     _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:154
                     spin_lock include/linux/spinlock.h:351 [inline]
                     __ppp_channel_push+0x31/0x1d0 drivers/net/ppp/ppp_generic.c:2161
                     ppp_channel_push+0x196/0x220 drivers/net/ppp/ppp_generic.c:2192
                     ppp_write+0x294/0x3a0 drivers/net/ppp/ppp_generic.c:538
                     do_iter_write+0x503/0xc40 fs/read_write.c:863
                     vfs_writev fs/read_write.c:934 [inline]
                     do_pwritev+0x216/0x360 fs/read_write.c:1031
                     do_syscall_x64 arch/x86/entry/common.c:51 [inline]
                     do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:81
                     entry_SYSCALL_64_after_hwframe+0x68/0xd2
  }
  ... key      at: [<ffffffff976db040>] ppp_register_net_channel.__key.1+0x0/0x20
  ... acquired at:
   lock_acquire+0x1f8/0x5a0 kernel/locking/lockdep.c:5662
   __raw_spin_lock_bh include/linux/spinlock_api_smp.h:126 [inline]
   _raw_spin_lock_bh+0x31/0x40 kernel/locking/spinlock.c:178
   spin_lock_bh include/linux/spinlock.h:356 [inline]
   ppp_connect_channel+0x190/0x640 drivers/net/ppp/ppp_generic.c:3485
   ppp_ioctl+0xbe5/0x1c90 drivers/net/ppp/ppp_generic.c:759
   vfs_ioctl fs/ioctl.c:51 [inline]
   __do_sys_ioctl fs/ioctl.c:870 [inline]
   __se_sys_ioctl+0xf1/0x160 fs/ioctl.c:856
   do_syscall_x64 arch/x86/entry/common.c:51 [inline]
   do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:81
   entry_SYSCALL_64_after_hwframe+0x68/0xd2

-> (&ppp->rlock){+.-.}-{2:2} {
   HARDIRQ-ON-W at:
                    lock_acquire+0x1f8/0x5a0 kernel/locking/lockdep.c:5662
                    __raw_spin_lock_bh include/linux/spinlock_api_smp.h:126 [inline]
                    _raw_spin_lock_bh+0x31/0x40 kernel/locking/spinlock.c:178
                    spin_lock_bh include/linux/spinlock.h:356 [inline]
                    ppp_get_stats64+0x2f/0x290 drivers/net/ppp/ppp_generic.c:1534
                    dev_get_stats+0xa7/0x490 net/core/dev.c:10471
                    rtnl_fill_stats+0x47/0x880 net/core/rtnetlink.c:1244
                    rtnl_fill_ifinfo+0x18aa/0x2090 net/core/rtnetlink.c:1825
                    rtmsg_ifinfo_build_skb+0xdc/0x180 net/core/rtnetlink.c:3938
                    rtmsg_ifinfo_event net/core/rtnetlink.c:3970 [inline]
                    rtmsg_ifinfo+0x71/0x120 net/core/rtnetlink.c:3979
                    register_netdevice+0x13dc/0x1720 net/core/dev.c:10146
                    ppp_unit_register drivers/net/ppp/ppp_generic.c:1217 [inline]
                    ppp_dev_configure+0x850/0xab0 drivers/net/ppp/ppp_generic.c:1273
                    ppp_create_interface drivers/net/ppp/ppp_generic.c:3348 [inline]
                    ppp_unattached_ioctl drivers/net/ppp/ppp_generic.c:1058 [inline]
                    ppp_ioctl+0x702/0x1c90 drivers/net/ppp/ppp_generic.c:731
                    vfs_ioctl fs/ioctl.c:51 [inline]
                    __do_sys_ioctl fs/ioctl.c:870 [inline]
                    __se_sys_ioctl+0xf1/0x160 fs/ioctl.c:856
                    do_syscall_x64 arch/x86/entry/common.c:51 [inline]
                    do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:81
                    entry_SYSCALL_64_after_hwframe+0x68/0xd2
   IN-SOFTIRQ-W at:
                    lock_acquire+0x1f8/0x5a0 kernel/locking/lockdep.c:5662
                    __raw_spin_lock_bh include/linux/spinlock_api_smp.h:126 [inline]
                    _raw_spin_lock_bh+0x31/0x40 kernel/locking/spinlock.c:178
                    spin_lock_bh include/linux/spinlock.h:356 [inline]
                    ppp_do_recv+0x38/0x26a0 drivers/net/ppp/ppp_generic.c:2210
                    ppp_input+0x5cc/0xa00 drivers/net/ppp/ppp_generic.c:2327
                    pppoe_rcv_core+0x112/0x300 drivers/net/ppp/pppoe.c:379
                    sk_backlog_rcv include/net/sock.h:1117 [inline]
                    __sk_receive_skb+0x41a/0x9d0 net/core/sock.c:569
                    __netif_receive_skb_one_core net/core/dev.c:5528 [inline]
                    __netif_receive_skb+0x1c6/0x530 net/core/dev.c:5642
                    process_backlog+0x381/0x760 net/core/dev.c:5970
                    __napi_poll+0xc7/0x470 net/core/dev.c:6537
                    napi_poll net/core/dev.c:6604 [inline]
                    net_rx_action+0x70f/0xeb0 net/core/dev.c:6718
                    handle_softirqs+0x2ee/0xa40 kernel/softirq.c:571
                    run_ksoftirqd+0xc6/0x120 kernel/softirq.c:938
                    smpboot_thread_fn+0x52c/0xa30 kernel/smpboot.c:164
                    kthread+0x28d/0x320 kernel/kthread.c:376
                    ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
   INITIAL USE at:
                   lock_acquire+0x1f8/0x5a0 kernel/locking/lockdep.c:5662
                   __raw_spin_lock_bh include/linux/spinlock_api_smp.h:126 [inline]
                   _raw_spin_lock_bh+0x31/0x40 kernel/locking/spinlock.c:178
                   spin_lock_bh include/linux/spinlock.h:356 [inline]
                   ppp_get_stats64+0x2f/0x290 drivers/net/ppp/ppp_generic.c:1534
                   dev_get_stats+0xa7/0x490 net/core/dev.c:10471
                   rtnl_fill_stats+0x47/0x880 net/core/rtnetlink.c:1244
                   rtnl_fill_ifinfo+0x18aa/0x2090 net/core/rtnetlink.c:1825
                   rtmsg_ifinfo_build_skb+0xdc/0x180 net/core/rtnetlink.c:3938
                   rtmsg_ifinfo_event net/core/rtnetlink.c:3970 [inline]
                   rtmsg_ifinfo+0x71/0x120 net/core/rtnetlink.c:3979
                   register_netdevice+0x13dc/0x1720 net/core/dev.c:10146
                   ppp_unit_register drivers/net/ppp/ppp_generic.c:1217 [inline]
                   ppp_dev_configure+0x850/0xab0 drivers/net/ppp/ppp_generic.c:1273
                   ppp_create_interface drivers/net/ppp/ppp_generic.c:3348 [inline]
                   ppp_unattached_ioctl drivers/net/ppp/ppp_generic.c:1058 [inline]
                   ppp_ioctl+0x702/0x1c90 drivers/net/ppp/ppp_generic.c:731
                   vfs_ioctl fs/ioctl.c:51 [inline]
                   __do_sys_ioctl fs/ioctl.c:870 [inline]
                   __se_sys_ioctl+0xf1/0x160 fs/ioctl.c:856
                   do_syscall_x64 arch/x86/entry/common.c:51 [inline]
                   do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:81
                   entry_SYSCALL_64_after_hwframe+0x68/0xd2
 }
 ... key      at: [<ffffffff976db180>] ppp_dev_configure.__key+0x0/0x20
 ... acquired at:
   mark_lock+0x21c/0x340 kernel/locking/lockdep.c:4628
   __lock_acquire+0xb7f/0x1f80 kernel/locking/lockdep.c:5003
   lock_acquire+0x1f8/0x5a0 kernel/locking/lockdep.c:5662
   __raw_spin_lock_bh include/linux/spinlock_api_smp.h:126 [inline]
   _raw_spin_lock_bh+0x31/0x40 kernel/locking/spinlock.c:178
   spin_lock_bh include/linux/spinlock.h:356 [inline]
   ppp_do_recv+0x38/0x26a0 drivers/net/ppp/ppp_generic.c:2210
   ppp_input+0x5cc/0xa00 drivers/net/ppp/ppp_generic.c:2327
   pppoe_rcv_core+0x112/0x300 drivers/net/ppp/pppoe.c:379
   sk_backlog_rcv include/net/sock.h:1117 [inline]
   __sk_receive_skb+0x41a/0x9d0 net/core/sock.c:569
   __netif_receive_skb_one_core net/core/dev.c:5528 [inline]
   __netif_receive_skb+0x1c6/0x530 net/core/dev.c:5642
   process_backlog+0x381/0x760 net/core/dev.c:5970
   __napi_poll+0xc7/0x470 net/core/dev.c:6537
   napi_poll net/core/dev.c:6604 [inline]
   net_rx_action+0x70f/0xeb0 net/core/dev.c:6718
   handle_softirqs+0x2ee/0xa40 kernel/softirq.c:571
   run_ksoftirqd+0xc6/0x120 kernel/softirq.c:938
   smpboot_thread_fn+0x52c/0xa30 kernel/smpboot.c:164
   kthread+0x28d/0x320 kernel/kthread.c:376
   ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295


stack backtrace:
CPU: 1 PID: 21 Comm: ksoftirqd/1 Not tainted 6.1.111-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/06/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e3/0x2cb lib/dump_stack.c:106
 mark_lock_irq+0x7de/0xba0 kernel/locking/lockdep.c:4199
 mark_lock+0x21c/0x340 kernel/locking/lockdep.c:4628
 __lock_acquire+0xb7f/0x1f80 kernel/locking/lockdep.c:5003
 lock_acquire+0x1f8/0x5a0 kernel/locking/lockdep.c:5662
 __raw_spin_lock_bh include/linux/spinlock_api_smp.h:126 [inline]
 _raw_spin_lock_bh+0x31/0x40 kernel/locking/spinlock.c:178
 spin_lock_bh include/linux/spinlock.h:356 [inline]
 ppp_do_recv+0x38/0x26a0 drivers/net/ppp/ppp_generic.c:2210
 ppp_input+0x5cc/0xa00 drivers/net/ppp/ppp_generic.c:2327
 pppoe_rcv_core+0x112/0x300 drivers/net/ppp/pppoe.c:379
 sk_backlog_rcv include/net/sock.h:1117 [inline]
 __sk_receive_skb+0x41a/0x9d0 net/core/sock.c:569
 __netif_receive_skb_one_core net/core/dev.c:5528 [inline]
 __netif_receive_skb+0x1c6/0x530 net/core/dev.c:5642
 process_backlog+0x381/0x760 net/core/dev.c:5970
 __napi_poll+0xc7/0x470 net/core/dev.c:6537
 napi_poll net/core/dev.c:6604 [inline]
 net_rx_action+0x70f/0xeb0 net/core/dev.c:6718
 handle_softirqs+0x2ee/0xa40 kernel/softirq.c:571
 run_ksoftirqd+0xc6/0x120 kernel/softirq.c:938
 smpboot_thread_fn+0x52c/0xa30 kernel/smpboot.c:164
 kthread+0x28d/0x320 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>

Crashes (2):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/09/25 02:48 linux-6.1.y e526b12bf916 5643e0e9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan possible deadlock in ppp_do_recv
2024/10/15 16:05 linux-6.1.y aa4cd140bba5 7eb57b4a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-6-1-kasan-arm64 possible deadlock in ppp_do_recv
* Struck through repros no longer work on HEAD.