syzbot


possible deadlock in ppp_do_recv

Status: upstream: reported on 2024/09/28 15:36
Reported-by: syzbot+4d2d52a72cd1d9aca63d@syzkaller.appspotmail.com
First crash: 83d, last: 83d
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-6.1 possible deadlock in ppp_do_recv 2 66d 87d 0/3 upstream: reported on 2024/09/25 02:48
upstream possible deadlock in ppp_do_recv ppp 8 74d 89d 28/28 fixed on 2024/11/12 23:31

Sample crash report:
========================================================
WARNING: possible irq lock inversion dependency detected
5.15.167-syzkaller #0 Not tainted
--------------------------------------------------------
ksoftirqd/1/20 just changed the state of lock:
ffff88807b504dd0 (&ppp->rlock){+.-.}-{2:2}, at: spin_lock_bh include/linux/spinlock.h:368 [inline]
ffff88807b504dd0 (&ppp->rlock){+.-.}-{2:2}, at: ppp_do_recv+0x38/0x2690 drivers/net/ppp/ppp_generic.c:2209
but this lock took another, SOFTIRQ-unsafe lock in the past:
 (&pch->downl){+.+.}-{2:2}


and interrupts could create inverse lock ordering between them.


other info that might help us debug this:
 Possible interrupt unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&pch->downl);
                               local_irq_disable();
                               lock(&ppp->rlock);
                               lock(&pch->downl);
  <Interrupt>
    lock(&ppp->rlock);

 *** DEADLOCK ***

4 locks held by ksoftirqd/1/20:
 #0: ffffffff8c91fc60 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x9/0x30 include/linux/rcupdate.h:312
 #1: ffff8880251b90a0 (slock-AF_PPPOX){+.-.}-{2:2}, at: spin_lock include/linux/spinlock.h:363 [inline]
 #1: ffff8880251b90a0 (slock-AF_PPPOX){+.-.}-{2:2}, at: __sk_receive_skb+0x17c/0x9c0 net/core/sock.c:521
 #2: ffff8880251b9120 (sk_lock-AF_PPPOX){+.+.}-{0:0}, at: __netif_receive_skb_one_core net/core/dev.c:5485 [inline]
 #2: ffff8880251b9120 (sk_lock-AF_PPPOX){+.+.}-{0:0}, at: __netif_receive_skb+0x1c6/0x530 net/core/dev.c:5599
 #3: ffff888020a30240 (&pch->upl){++.-}-{2:2}, at: ppp_input+0x361/0x980 drivers/net/ppp/ppp_generic.c:2306

the shortest dependencies between 2nd lock and 1st lock:
 -> (&pch->downl){+.+.}-{2:2} {
    HARDIRQ-ON-W at:
                      lock_acquire+0x1db/0x4f0 kernel/locking/lockdep.c:5623
                      __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
                      _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:154
                      spin_lock include/linux/spinlock.h:363 [inline]
                      __ppp_channel_push+0x31/0x1d0 drivers/net/ppp/ppp_generic.c:2160
                      ppp_channel_push+0x196/0x220 drivers/net/ppp/ppp_generic.c:2191
                      ppp_write+0x291/0x390 drivers/net/ppp/ppp_generic.c:537
                      do_iter_write+0x39c/0x760 fs/read_write.c:857
                      vfs_writev fs/read_write.c:928 [inline]
                      do_pwritev+0x219/0x360 fs/read_write.c:1025
                      do_syscall_x64 arch/x86/entry/common.c:50 [inline]
                      do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:80
                      entry_SYSCALL_64_after_hwframe+0x66/0xd0
    SOFTIRQ-ON-W at:
                      lock_acquire+0x1db/0x4f0 kernel/locking/lockdep.c:5623
                      __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
                      _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:154
                      spin_lock include/linux/spinlock.h:363 [inline]
                      ppp_channel_bridge_input drivers/net/ppp/ppp_generic.c:2271 [inline]
                      ppp_input+0x160/0x980 drivers/net/ppp/ppp_generic.c:2303
                      pppoe_rcv_core+0x109/0x2f0 drivers/net/ppp/pppoe.c:379
                      sk_backlog_rcv include/net/sock.h:1059 [inline]
                      __release_sock+0x198/0x4b0 net/core/sock.c:2724
                      release_sock+0x5d/0x1c0 net/core/sock.c:3265
                      pppoe_sendmsg+0xd1/0x740 drivers/net/ppp/pppoe.c:903
                      sock_sendmsg_nosec net/socket.c:704 [inline]
                      __sock_sendmsg net/socket.c:716 [inline]
                      ____sys_sendmsg+0x59e/0x8f0 net/socket.c:2431
                      ___sys_sendmsg+0x252/0x2e0 net/socket.c:2485
                      __sys_sendmmsg+0x2bf/0x560 net/socket.c:2571
                      __do_sys_sendmmsg net/socket.c:2600 [inline]
                      __se_sys_sendmmsg net/socket.c:2597 [inline]
                      __x64_sys_sendmmsg+0x9c/0xb0 net/socket.c:2597
                      do_syscall_x64 arch/x86/entry/common.c:50 [inline]
                      do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:80
                      entry_SYSCALL_64_after_hwframe+0x66/0xd0
    INITIAL USE at:
                     lock_acquire+0x1db/0x4f0 kernel/locking/lockdep.c:5623
                     __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
                     _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:154
                     spin_lock include/linux/spinlock.h:363 [inline]
                     __ppp_channel_push+0x31/0x1d0 drivers/net/ppp/ppp_generic.c:2160
                     ppp_channel_push+0x196/0x220 drivers/net/ppp/ppp_generic.c:2191
                     ppp_write+0x291/0x390 drivers/net/ppp/ppp_generic.c:537
                     do_iter_write+0x39c/0x760 fs/read_write.c:857
                     vfs_writev fs/read_write.c:928 [inline]
                     do_pwritev+0x219/0x360 fs/read_write.c:1025
                     do_syscall_x64 arch/x86/entry/common.c:50 [inline]
                     do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:80
                     entry_SYSCALL_64_after_hwframe+0x66/0xd0
  }
  ... key      at: [<ffffffff96bb1560>] ppp_register_net_channel.__key.1+0x0/0x20
  ... acquired at:
   lock_acquire+0x1db/0x4f0 kernel/locking/lockdep.c:5623
   __raw_spin_lock_bh include/linux/spinlock_api_smp.h:135 [inline]
   _raw_spin_lock_bh+0x31/0x40 kernel/locking/spinlock.c:178
   spin_lock_bh include/linux/spinlock.h:368 [inline]
   ppp_connect_channel+0x190/0x640 drivers/net/ppp/ppp_generic.c:3484
   ppp_ioctl+0xbe5/0x1cd0 drivers/net/ppp/ppp_generic.c:758
   vfs_ioctl fs/ioctl.c:51 [inline]
   __do_sys_ioctl fs/ioctl.c:874 [inline]
   __se_sys_ioctl+0xf1/0x160 fs/ioctl.c:860
   do_syscall_x64 arch/x86/entry/common.c:50 [inline]
   do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:80
   entry_SYSCALL_64_after_hwframe+0x66/0xd0

-> (&ppp->rlock){+.-.}-{2:2} {
   HARDIRQ-ON-W at:
                    lock_acquire+0x1db/0x4f0 kernel/locking/lockdep.c:5623
                    __raw_spin_lock_bh include/linux/spinlock_api_smp.h:135 [inline]
                    _raw_spin_lock_bh+0x31/0x40 kernel/locking/spinlock.c:178
                    spin_lock_bh include/linux/spinlock.h:368 [inline]
                    ppp_get_stats64+0x2f/0x290 drivers/net/ppp/ppp_generic.c:1533
                    dev_get_stats+0x90/0x370 net/core/dev.c:10704
                    rtnl_fill_stats+0x47/0x880 net/core/rtnetlink.c:1205
                    rtnl_fill_ifinfo+0x1598/0x1d60 net/core/rtnetlink.c:1782
                    rtmsg_ifinfo_build_skb+0xdc/0x180 net/core/rtnetlink.c:3856
                    rtmsg_ifinfo_event net/core/rtnetlink.c:3888 [inline]
                    rtmsg_ifinfo+0x71/0x120 net/core/rtnetlink.c:3897
                    register_netdevice+0x13d2/0x1720 net/core/dev.c:10402
                    ppp_unit_register drivers/net/ppp/ppp_generic.c:1216 [inline]
                    ppp_dev_configure+0x853/0xad0 drivers/net/ppp/ppp_generic.c:1272
                    ppp_create_interface drivers/net/ppp/ppp_generic.c:3347 [inline]
                    ppp_unattached_ioctl drivers/net/ppp/ppp_generic.c:1057 [inline]
                    ppp_ioctl+0x702/0x1cd0 drivers/net/ppp/ppp_generic.c:730
                    vfs_ioctl fs/ioctl.c:51 [inline]
                    __do_sys_ioctl fs/ioctl.c:874 [inline]
                    __se_sys_ioctl+0xf1/0x160 fs/ioctl.c:860
                    do_syscall_x64 arch/x86/entry/common.c:50 [inline]
                    do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:80
                    entry_SYSCALL_64_after_hwframe+0x66/0xd0
   IN-SOFTIRQ-W at:
                    lock_acquire+0x1db/0x4f0 kernel/locking/lockdep.c:5623
                    __raw_spin_lock_bh include/linux/spinlock_api_smp.h:135 [inline]
                    _raw_spin_lock_bh+0x31/0x40 kernel/locking/spinlock.c:178
                    spin_lock_bh include/linux/spinlock.h:368 [inline]
                    ppp_do_recv+0x38/0x2690 drivers/net/ppp/ppp_generic.c:2209
                    ppp_input+0x577/0x980 drivers/net/ppp/ppp_generic.c:2326
                    pppoe_rcv_core+0x109/0x2f0 drivers/net/ppp/pppoe.c:379
                    sk_backlog_rcv include/net/sock.h:1059 [inline]
                    __sk_receive_skb+0x416/0x9c0 net/core/sock.c:528
                    __netif_receive_skb_one_core net/core/dev.c:5485 [inline]
                    __netif_receive_skb+0x1c6/0x530 net/core/dev.c:5599
                    process_backlog+0x363/0x7f0 net/core/dev.c:6476
                    __napi_poll+0xc7/0x440 net/core/dev.c:7035
                    napi_poll net/core/dev.c:7102 [inline]
                    net_rx_action+0x617/0xda0 net/core/dev.c:7192
                    handle_softirqs+0x3a7/0x930 kernel/softirq.c:558
                    run_ksoftirqd+0xc6/0x120 kernel/softirq.c:925
                    smpboot_thread_fn+0x51b/0x9d0 kernel/smpboot.c:164
                    kthread+0x3f6/0x4f0 kernel/kthread.c:334
                    ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:287
   INITIAL USE at:
                   lock_acquire+0x1db/0x4f0 kernel/locking/lockdep.c:5623
                   __raw_spin_lock_bh include/linux/spinlock_api_smp.h:135 [inline]
                   _raw_spin_lock_bh+0x31/0x40 kernel/locking/spinlock.c:178
                   spin_lock_bh include/linux/spinlock.h:368 [inline]
                   ppp_get_stats64+0x2f/0x290 drivers/net/ppp/ppp_generic.c:1533
                   dev_get_stats+0x90/0x370 net/core/dev.c:10704
                   rtnl_fill_stats+0x47/0x880 net/core/rtnetlink.c:1205
                   rtnl_fill_ifinfo+0x1598/0x1d60 net/core/rtnetlink.c:1782
                   rtmsg_ifinfo_build_skb+0xdc/0x180 net/core/rtnetlink.c:3856
                   rtmsg_ifinfo_event net/core/rtnetlink.c:3888 [inline]
                   rtmsg_ifinfo+0x71/0x120 net/core/rtnetlink.c:3897
                   register_netdevice+0x13d2/0x1720 net/core/dev.c:10402
                   ppp_unit_register drivers/net/ppp/ppp_generic.c:1216 [inline]
                   ppp_dev_configure+0x853/0xad0 drivers/net/ppp/ppp_generic.c:1272
                   ppp_create_interface drivers/net/ppp/ppp_generic.c:3347 [inline]
                   ppp_unattached_ioctl drivers/net/ppp/ppp_generic.c:1057 [inline]
                   ppp_ioctl+0x702/0x1cd0 drivers/net/ppp/ppp_generic.c:730
                   vfs_ioctl fs/ioctl.c:51 [inline]
                   __do_sys_ioctl fs/ioctl.c:874 [inline]
                   __se_sys_ioctl+0xf1/0x160 fs/ioctl.c:860
                   do_syscall_x64 arch/x86/entry/common.c:50 [inline]
                   do_syscall_64+0x3b/0xb0 arch/x86/entry/common.c:80
                   entry_SYSCALL_64_after_hwframe+0x66/0xd0
 }
 ... key      at: [<ffffffff96bb16c0>] ppp_dev_configure.__key+0x0/0x20
 ... acquired at:
   mark_lock+0x21a/0x340 kernel/locking/lockdep.c:4591
   __lock_acquire+0xb9d/0x1ff0 kernel/locking/lockdep.c:4966
   lock_acquire+0x1db/0x4f0 kernel/locking/lockdep.c:5623
   __raw_spin_lock_bh include/linux/spinlock_api_smp.h:135 [inline]
   _raw_spin_lock_bh+0x31/0x40 kernel/locking/spinlock.c:178
   spin_lock_bh include/linux/spinlock.h:368 [inline]
   ppp_do_recv+0x38/0x2690 drivers/net/ppp/ppp_generic.c:2209
   ppp_input+0x577/0x980 drivers/net/ppp/ppp_generic.c:2326
   pppoe_rcv_core+0x109/0x2f0 drivers/net/ppp/pppoe.c:379
   sk_backlog_rcv include/net/sock.h:1059 [inline]
   __sk_receive_skb+0x416/0x9c0 net/core/sock.c:528
   __netif_receive_skb_one_core net/core/dev.c:5485 [inline]
   __netif_receive_skb+0x1c6/0x530 net/core/dev.c:5599
   process_backlog+0x363/0x7f0 net/core/dev.c:6476
   __napi_poll+0xc7/0x440 net/core/dev.c:7035
   napi_poll net/core/dev.c:7102 [inline]
   net_rx_action+0x617/0xda0 net/core/dev.c:7192
   handle_softirqs+0x3a7/0x930 kernel/softirq.c:558
   run_ksoftirqd+0xc6/0x120 kernel/softirq.c:925
   smpboot_thread_fn+0x51b/0x9d0 kernel/smpboot.c:164
   kthread+0x3f6/0x4f0 kernel/kthread.c:334
   ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:287


stack backtrace:
CPU: 1 PID: 20 Comm: ksoftirqd/1 Not tainted 5.15.167-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e3/0x2d0 lib/dump_stack.c:106
 mark_lock_irq+0x7d8/0xba0 kernel/locking/lockdep.c:4162
 mark_lock+0x21a/0x340 kernel/locking/lockdep.c:4591
 __lock_acquire+0xb9d/0x1ff0 kernel/locking/lockdep.c:4966
 lock_acquire+0x1db/0x4f0 kernel/locking/lockdep.c:5623
 __raw_spin_lock_bh include/linux/spinlock_api_smp.h:135 [inline]
 _raw_spin_lock_bh+0x31/0x40 kernel/locking/spinlock.c:178
 spin_lock_bh include/linux/spinlock.h:368 [inline]
 ppp_do_recv+0x38/0x2690 drivers/net/ppp/ppp_generic.c:2209
 ppp_input+0x577/0x980 drivers/net/ppp/ppp_generic.c:2326
 pppoe_rcv_core+0x109/0x2f0 drivers/net/ppp/pppoe.c:379
 sk_backlog_rcv include/net/sock.h:1059 [inline]
 __sk_receive_skb+0x416/0x9c0 net/core/sock.c:528
 __netif_receive_skb_one_core net/core/dev.c:5485 [inline]
 __netif_receive_skb+0x1c6/0x530 net/core/dev.c:5599
 process_backlog+0x363/0x7f0 net/core/dev.c:6476
 __napi_poll+0xc7/0x440 net/core/dev.c:7035
 napi_poll net/core/dev.c:7102 [inline]
 net_rx_action+0x617/0xda0 net/core/dev.c:7192
 handle_softirqs+0x3a7/0x930 kernel/softirq.c:558
 run_ksoftirqd+0xc6/0x120 kernel/softirq.c:925
 smpboot_thread_fn+0x51b/0x9d0 kernel/smpboot.c:164
 kthread+0x3f6/0x4f0 kernel/kthread.c:334
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:287
 </TASK>

Crashes (1):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/09/28 15:35 linux-5.15.y 3a5928702e71 ba29ff75 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan possible deadlock in ppp_do_recv
* Struck through repros no longer work on HEAD.