syzbot


possible deadlock in team_device_event (3)

Status: upstream: reported on 2024/05/09 16:54
Subsystems: net
[Documentation on labels]
Reported-by: syzbot+b668da2bc4cb9670bf58@syzkaller.appspotmail.com
First crash: 51d, last: 51d
Discussions (1)
Title Replies (including bot) Last reply
[syzbot] [net?] possible deadlock in team_device_event (3) 0 (1) 2024/05/09 16:54
Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream possible deadlock in team_device_event net 30 1392d 1460d 0/27 auto-closed as invalid on 2020/12/31 13:59
upstream possible deadlock in team_device_event (2) net 8 939d 1176d 0/27 auto-closed as invalid on 2022/03/29 04:55

Sample crash report:
mac80211_hwsim hwsim28 wlan0 (unregistering): left allmulticast mode
======================================================
WARNING: possible circular locking dependency detected
6.9.0-rc6-syzkaller-00234-g7367539ad4b0 #0 Not tainted
------------------------------------------------------
kworker/u8:9/5208 is trying to acquire lock:
ffff88806325cd20 (team->team_lock_key#12){+.+.}-{3:3}, at: team_port_change_check drivers/net/team/team.c:2995 [inline]
ffff88806325cd20 (team->team_lock_key#12){+.+.}-{3:3}, at: team_device_event+0x11d/0x770 drivers/net/team/team.c:3021

but task is already holding lock:
ffff888051578768 (&rdev->wiphy.mtx){+.+.}-{3:3}, at: wiphy_lock include/net/cfg80211.h:5953 [inline]
ffff888051578768 (&rdev->wiphy.mtx){+.+.}-{3:3}, at: ieee80211_remove_interfaces+0xfe/0x760 net/mac80211/iface.c:2277

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (&rdev->wiphy.mtx){+.+.}-{3:3}:
       __mutex_lock_common kernel/locking/mutex.c:608 [inline]
       __mutex_lock+0x175/0x9c0 kernel/locking/mutex.c:752
       wiphy_lock include/net/cfg80211.h:5953 [inline]
       cfg80211_netdev_notifier_call+0x367/0x1110 net/wireless/core.c:1524
       notifier_call_chain+0xb9/0x410 kernel/notifier.c:93
       call_netdevice_notifiers_info+0xbe/0x140 net/core/dev.c:1950
       call_netdevice_notifiers_extack net/core/dev.c:1988 [inline]
       call_netdevice_notifiers net/core/dev.c:2002 [inline]
       dev_open net/core/dev.c:1471 [inline]
       dev_open+0x144/0x160 net/core/dev.c:1459
       team_port_add drivers/net/team/team.c:1214 [inline]
       team_add_slave+0xadc/0x2110 drivers/net/team/team.c:1974
       do_set_master+0x1bc/0x230 net/core/rtnetlink.c:2685
       do_setlink+0xcaf/0x3ff0 net/core/rtnetlink.c:2891
       __rtnl_newlink+0xc35/0x1960 net/core/rtnetlink.c:3680
       rtnl_newlink+0x67/0xa0 net/core/rtnetlink.c:3727
       rtnetlink_rcv_msg+0x3c7/0xe60 net/core/rtnetlink.c:6595
       netlink_rcv_skb+0x16b/0x440 net/netlink/af_netlink.c:2559
       netlink_unicast_kernel net/netlink/af_netlink.c:1335 [inline]
       netlink_unicast+0x542/0x820 net/netlink/af_netlink.c:1361
       netlink_sendmsg+0x8b8/0xd70 net/netlink/af_netlink.c:1905
       sock_sendmsg_nosec net/socket.c:730 [inline]
       __sock_sendmsg net/socket.c:745 [inline]
       ____sys_sendmsg+0xab5/0xc90 net/socket.c:2584
       ___sys_sendmsg+0x135/0x1e0 net/socket.c:2638
       __sys_sendmsg+0x117/0x1f0 net/socket.c:2667
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xcf/0x260 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

-> #0 (team->team_lock_key#12){+.+.}-{3:3}:
       check_prev_add kernel/locking/lockdep.c:3134 [inline]
       check_prevs_add kernel/locking/lockdep.c:3253 [inline]
       validate_chain kernel/locking/lockdep.c:3869 [inline]
       __lock_acquire+0x2478/0x3b30 kernel/locking/lockdep.c:5137
       lock_acquire kernel/locking/lockdep.c:5754 [inline]
       lock_acquire+0x1b1/0x560 kernel/locking/lockdep.c:5719
       __mutex_lock_common kernel/locking/mutex.c:608 [inline]
       __mutex_lock+0x175/0x9c0 kernel/locking/mutex.c:752
       team_port_change_check drivers/net/team/team.c:2995 [inline]
       team_device_event+0x11d/0x770 drivers/net/team/team.c:3021
       notifier_call_chain+0xb9/0x410 kernel/notifier.c:93
       call_netdevice_notifiers_info+0xbe/0x140 net/core/dev.c:1950
       call_netdevice_notifiers_extack net/core/dev.c:1988 [inline]
       call_netdevice_notifiers net/core/dev.c:2002 [inline]
       dev_close_many+0x333/0x6a0 net/core/dev.c:1543
       unregister_netdevice_many_notify+0x46d/0x19f0 net/core/dev.c:11080
       macvlan_device_event+0x4ed/0x880 drivers/net/macvlan.c:1828
       notifier_call_chain+0xb9/0x410 kernel/notifier.c:93
       call_netdevice_notifiers_info+0xbe/0x140 net/core/dev.c:1950
       call_netdevice_notifiers_extack net/core/dev.c:1988 [inline]
       call_netdevice_notifiers net/core/dev.c:2002 [inline]
       unregister_netdevice_many_notify+0x8a1/0x19f0 net/core/dev.c:11105
       unregister_netdevice_many net/core/dev.c:11163 [inline]
       unregister_netdevice_queue+0x307/0x3f0 net/core/dev.c:11042
       unregister_netdevice include/linux/netdevice.h:3115 [inline]
       _cfg80211_unregister_wdev+0x624/0x7f0 net/wireless/core.c:1206
       ieee80211_remove_interfaces+0x36d/0x760 net/mac80211/iface.c:2302
       ieee80211_unregister_hw+0x55/0x3a0 net/mac80211/main.c:1652
       mac80211_hwsim_del_radio drivers/net/wireless/virtual/mac80211_hwsim.c:5560 [inline]
       hwsim_exit_net+0x3ad/0x7d0 drivers/net/wireless/virtual/mac80211_hwsim.c:6437
       ops_exit_list+0xb0/0x180 net/core/net_namespace.c:170
       cleanup_net+0x5b7/0xbf0 net/core/net_namespace.c:637
       process_one_work+0x9a9/0x1ac0 kernel/workqueue.c:3267
       process_scheduled_works kernel/workqueue.c:3348 [inline]
       worker_thread+0x6c8/0xf70 kernel/workqueue.c:3429
       kthread+0x2c1/0x3a0 kernel/kthread.c:388
       ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
       ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&rdev->wiphy.mtx);
                               lock(team->team_lock_key#12);
                               lock(&rdev->wiphy.mtx);
  lock(team->team_lock_key#12);

 *** DEADLOCK ***

5 locks held by kworker/u8:9/5208:
 #0: ffff888015ecb148 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0x1296/0x1ac0 kernel/workqueue.c:3242
 #1: ffffc90003e5fd80 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0x906/0x1ac0 kernel/workqueue.c:3243
 #2: ffffffff8f2ec950 (pernet_ops_rwsem){++++}-{3:3}, at: cleanup_net+0xbb/0xbf0 net/core/net_namespace.c:591
 #3: ffffffff8f301748 (rtnl_mutex){+.+.}-{3:3}, at: ieee80211_unregister_hw+0x4d/0x3a0 net/mac80211/main.c:1645
 #4: ffff888051578768 (&rdev->wiphy.mtx){+.+.}-{3:3}, at: wiphy_lock include/net/cfg80211.h:5953 [inline]
 #4: ffff888051578768 (&rdev->wiphy.mtx){+.+.}-{3:3}, at: ieee80211_remove_interfaces+0xfe/0x760 net/mac80211/iface.c:2277

stack backtrace:
CPU: 1 PID: 5208 Comm: kworker/u8:9 Not tainted 6.9.0-rc6-syzkaller-00234-g7367539ad4b0 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/27/2024
Workqueue: netns cleanup_net
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x116/0x1f0 lib/dump_stack.c:114
 check_noncircular+0x31a/0x400 kernel/locking/lockdep.c:2187
 check_prev_add kernel/locking/lockdep.c:3134 [inline]
 check_prevs_add kernel/locking/lockdep.c:3253 [inline]
 validate_chain kernel/locking/lockdep.c:3869 [inline]
 __lock_acquire+0x2478/0x3b30 kernel/locking/lockdep.c:5137
 lock_acquire kernel/locking/lockdep.c:5754 [inline]
 lock_acquire+0x1b1/0x560 kernel/locking/lockdep.c:5719
 __mutex_lock_common kernel/locking/mutex.c:608 [inline]
 __mutex_lock+0x175/0x9c0 kernel/locking/mutex.c:752
 team_port_change_check drivers/net/team/team.c:2995 [inline]
 team_device_event+0x11d/0x770 drivers/net/team/team.c:3021
 notifier_call_chain+0xb9/0x410 kernel/notifier.c:93
 call_netdevice_notifiers_info+0xbe/0x140 net/core/dev.c:1950
 call_netdevice_notifiers_extack net/core/dev.c:1988 [inline]
 call_netdevice_notifiers net/core/dev.c:2002 [inline]
 dev_close_many+0x333/0x6a0 net/core/dev.c:1543
 unregister_netdevice_many_notify+0x46d/0x19f0 net/core/dev.c:11080
 macvlan_device_event+0x4ed/0x880 drivers/net/macvlan.c:1828
 notifier_call_chain+0xb9/0x410 kernel/notifier.c:93
 call_netdevice_notifiers_info+0xbe/0x140 net/core/dev.c:1950
 call_netdevice_notifiers_extack net/core/dev.c:1988 [inline]
 call_netdevice_notifiers net/core/dev.c:2002 [inline]
 unregister_netdevice_many_notify+0x8a1/0x19f0 net/core/dev.c:11105
 unregister_netdevice_many net/core/dev.c:11163 [inline]
 unregister_netdevice_queue+0x307/0x3f0 net/core/dev.c:11042
 unregister_netdevice include/linux/netdevice.h:3115 [inline]
 _cfg80211_unregister_wdev+0x624/0x7f0 net/wireless/core.c:1206
 ieee80211_remove_interfaces+0x36d/0x760 net/mac80211/iface.c:2302
 ieee80211_unregister_hw+0x55/0x3a0 net/mac80211/main.c:1652
 mac80211_hwsim_del_radio drivers/net/wireless/virtual/mac80211_hwsim.c:5560 [inline]
 hwsim_exit_net+0x3ad/0x7d0 drivers/net/wireless/virtual/mac80211_hwsim.c:6437
 ops_exit_list+0xb0/0x180 net/core/net_namespace.c:170
 cleanup_net+0x5b7/0xbf0 net/core/net_namespace.c:637
 process_one_work+0x9a9/0x1ac0 kernel/workqueue.c:3267
 process_scheduled_works kernel/workqueue.c:3348 [inline]
 worker_thread+0x6c8/0xf70 kernel/workqueue.c:3429
 kthread+0x2c1/0x3a0 kernel/kthread.c:388
 ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
 </TASK>
team0: Port device macvlan2 removed
hsr_slave_0: left promiscuous mode
hsr_slave_1: left promiscuous mode
batman_adv: batadv0: Interface deactivated: batadv_slave_0
batman_adv: batadv0: Removing interface: batadv_slave_0
batman_adv: batadv0: Interface deactivated: batadv_slave_1
batman_adv: batadv0: Removing interface: batadv_slave_1
veth1_macvtap: left promiscuous mode
veth0_macvtap: left promiscuous mode
veth1_vlan: left promiscuous mode
veth0_vlan: left promiscuous mode
team0 (unregistering): Port device virt_wifi0 removed
team0 (unregistering): Port device team_slave_1 removed
team0 (unregistering): Port device team_slave_0 removed

Crashes (1):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/05/05 16:52 upstream 7367539ad4b0 610f2a54 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root possible deadlock in team_device_event
* Struck through repros no longer work on HEAD.