syzbot


possible deadlock in rtnl_lock (3)

Status: fixed on 2018/02/07 13:48
Subsystems: net
[Documentation on labels]
Reported-by: syzbot+63682ce11532e0da2b9d@syzkaller.appspotmail.com
Fix commit: 3f34cfae1238 netfilter: on sockopt() acquire sock lock only in the required scope
First crash: 2270d, last: 2268d
Discussions (1)
Title Replies (including bot) Last reply
possible deadlock in rtnl_lock (3) 3 (4) 2018/02/07 10:11
Similar bugs (9)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
android-44 possible deadlock in rtnl_lock C 4676 1647d 1839d 0/2 public: reported C repro on 2019/04/12 00:00
upstream possible deadlock in rtnl_lock (5) net C 1009 2191d 2220d 5/26 fixed on 2018/05/08 18:30
linux-4.19 possible deadlock in rtnl_lock C error 31 1320d 1570d 0/1 upstream: reported C repro on 2020/01/06 01:18
upstream possible deadlock in rtnl_lock (4) net C 73333 2251d 2267d 4/26 fixed on 2018/02/26 20:04
upstream possible deadlock in rtnl_lock (6) net C done error 551 1510d 1752d 0/26 auto-obsoleted due to no activity on 2022/12/21 11:47
android-49 possible deadlock in rtnl_lock C 2906 2069d 2221d 0/3 closed as invalid on 2019/04/05 05:43
upstream possible deadlock in rtnl_lock net C 15711 2274d 2331d 4/26 fixed on 2018/02/01 04:00
upstream possible deadlock in rtnl_lock (2) net C 10369 2270d 2274d 4/26 fixed on 2018/02/04 23:45
linux-4.14 possible deadlock in rtnl_lock C inconclusive 312 1429d 1570d 0/1 upstream: reported C repro on 2020/01/06 01:45

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
4.15.0+ #300 Not tainted
------------------------------------------------------
syzkaller428858/4154 is trying to acquire lock:
 (rtnl_mutex){+.+.}, at: [<00000000da6f19c2>] rtnl_lock+0x17/0x20 net/core/rtnetlink.c:74

but task is already holding lock:
 (&xt[i].mutex){+.+.}, at: [<0000000097c1c439>] xt_find_table_lock+0x3e/0x3e0 net/netfilter/x_tables.c:1041

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #2 (&xt[i].mutex){+.+.}:
       __mutex_lock_common kernel/locking/mutex.c:756 [inline]
       __mutex_lock+0x16f/0x1a80 kernel/locking/mutex.c:893
       mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
       xt_find_table_lock+0x3e/0x3e0 net/netfilter/x_tables.c:1041
       xt_request_find_table_lock+0x28/0xc0 net/netfilter/x_tables.c:1088
       get_info+0x154/0x690 net/ipv6/netfilter/ip6_tables.c:989
       do_ip6t_get_ctl+0x159/0xaf0 net/ipv6/netfilter/ip6_tables.c:1710
       nf_sockopt net/netfilter/nf_sockopt.c:104 [inline]
       nf_getsockopt+0x6a/0xc0 net/netfilter/nf_sockopt.c:122
       ipv6_getsockopt+0x1df/0x2e0 net/ipv6/ipv6_sockglue.c:1371
       tcp_getsockopt+0x82/0xd0 net/ipv4/tcp.c:3359
       sock_common_getsockopt+0x95/0xd0 net/core/sock.c:2934
       SYSC_getsockopt net/socket.c:1880 [inline]
       SyS_getsockopt+0x178/0x340 net/socket.c:1862
       do_syscall_64+0x282/0x940 arch/x86/entry/common.c:287
       entry_SYSCALL_64_after_hwframe+0x26/0x9b

-> #1 (sk_lock-AF_INET6){+.+.}:
       lock_sock_nested+0xc2/0x110 net/core/sock.c:2777
       lock_sock include/net/sock.h:1463 [inline]
       do_ipv6_setsockopt.isra.8+0x3c5/0x39d0 net/ipv6/ipv6_sockglue.c:167
       ipv6_setsockopt+0xd7/0x130 net/ipv6/ipv6_sockglue.c:922
       dccp_setsockopt+0x85/0xd0 net/dccp/proto.c:576
       sock_common_setsockopt+0x95/0xd0 net/core/sock.c:2975
       SYSC_setsockopt net/socket.c:1849 [inline]
       SyS_setsockopt+0x189/0x360 net/socket.c:1828
       do_syscall_64+0x282/0x940 arch/x86/entry/common.c:287
       entry_SYSCALL_64_after_hwframe+0x26/0x9b

-> #0 (rtnl_mutex){+.+.}:
       lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:3920
       __mutex_lock_common kernel/locking/mutex.c:756 [inline]
       __mutex_lock+0x16f/0x1a80 kernel/locking/mutex.c:893
       mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
       rtnl_lock+0x17/0x20 net/core/rtnetlink.c:74
       unregister_netdevice_notifier+0x91/0x4e0 net/core/dev.c:1673
       clusterip_config_entry_put net/ipv4/netfilter/ipt_CLUSTERIP.c:114 [inline]
       clusterip_tg_destroy+0x389/0x6e0 net/ipv4/netfilter/ipt_CLUSTERIP.c:518
       cleanup_entry+0x218/0x350 net/ipv4/netfilter/ip_tables.c:654
       __do_replace+0x79d/0xa50 net/ipv4/netfilter/ip_tables.c:1089
       do_replace net/ipv4/netfilter/ip_tables.c:1145 [inline]
       do_ipt_set_ctl+0x40f/0x5f0 net/ipv4/netfilter/ip_tables.c:1675
       nf_sockopt net/netfilter/nf_sockopt.c:106 [inline]
       nf_setsockopt+0x67/0xc0 net/netfilter/nf_sockopt.c:115
       ip_setsockopt+0x97/0xa0 net/ipv4/ip_sockglue.c:1259
       tcp_setsockopt+0x82/0xd0 net/ipv4/tcp.c:2905
       sock_common_setsockopt+0x95/0xd0 net/core/sock.c:2975
       SYSC_setsockopt net/socket.c:1849 [inline]
       SyS_setsockopt+0x189/0x360 net/socket.c:1828
       do_syscall_64+0x282/0x940 arch/x86/entry/common.c:287
       entry_SYSCALL_64_after_hwframe+0x26/0x9b

other info that might help us debug this:

Chain exists of:
  rtnl_mutex --> sk_lock-AF_INET6 --> &xt[i].mutex

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&xt[i].mutex);
                               lock(sk_lock-AF_INET6);
                               lock(&xt[i].mutex);
  lock(rtnl_mutex);

 *** DEADLOCK ***

1 lock held by syzkaller428858/4154:
 #0:  (&xt[i].mutex){+.+.}, at: [<0000000097c1c439>] xt_find_table_lock+0x3e/0x3e0 net/netfilter/x_tables.c:1041

stack backtrace:
CPU: 0 PID: 4154 Comm: syzkaller428858 Not tainted 4.15.0+ #300
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 __dump_stack lib/dump_stack.c:17 [inline]
 dump_stack+0x194/0x257 lib/dump_stack.c:53
 print_circular_bug.isra.38+0x2cd/0x2dc kernel/locking/lockdep.c:1223
 check_prev_add kernel/locking/lockdep.c:1863 [inline]
 check_prevs_add kernel/locking/lockdep.c:1976 [inline]
 validate_chain kernel/locking/lockdep.c:2417 [inline]
 __lock_acquire+0x30a8/0x3e00 kernel/locking/lockdep.c:3431
 lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:3920
 __mutex_lock_common kernel/locking/mutex.c:756 [inline]
 __mutex_lock+0x16f/0x1a80 kernel/locking/mutex.c:893
 mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
 rtnl_lock+0x17/0x20 net/core/rtnetlink.c:74
 unregister_netdevice_notifier+0x91/0x4e0 net/core/dev.c:1673
 clusterip_config_entry_put net/ipv4/netfilter/ipt_CLUSTERIP.c:114 [inline]
 clusterip_tg_destroy+0x389/0x6e0 net/ipv4/netfilter/ipt_CLUSTERIP.c:518
 cleanup_entry+0x218/0x350 net/ipv4/netfilter/ip_tables.c:654
 __do_replace+0x79d/0xa50 net/ipv4/netfilter/ip_tables.c:1089
 do_replace net/ipv4/netfilter/ip_tables.c:1145 [inline]
 do_ipt_set_ctl+0x40f/0x5f0 net/ipv4/netfilter/ip_tables.c:1675
 nf_sockopt net/netfilter/nf_sockopt.c:106 [inline]
 nf_setsockopt+0x67/0xc0 net/netfilter/nf_sockopt.c:115
 ip_setsockopt+0x97/0xa0 net/ipv4/ip_sockglue.c:1259
 tcp_setsockopt+0x82/0xd0 net/ipv4/tcp.c:2905
 sock_common_setsockopt+0x95/0xd0 net/core/sock.c:2975
 SYSC_setsockopt net/socket.c:1849 [inline]
 SyS_setsockopt+0x189/0x360 net/socket.c:1828
 do_syscall_64+0x282/0x940 arch/x86/entry/common.c:287
 entry_SYSCALL_64_after_hwframe+0x26/0x9b
RIP: 0033:0x44438a
RSP: 002b:00007fff404a0a08 EFLAGS: 00000206 ORIG_RAX: 0000000000000036
RAX: ffffffffffffffda RBX: 00000000006cc100 RCX: 000000000044438a
RDX: 0000000000000040 RSI: 0000000000000000 RDI: 0000000000000003
RBP: 00000000006cc100 R08: 00000000000002d8 R09: 0000000002006880
R10: 00000000006cc528 R11: 0000000000000206 R12: 0000000000000003
R13: 00000000006cf0a8 R14: 00000000006cf050 R15: 00000000004a332e

Crashes (3633):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2018/02/07 10:07 upstream cbd7b8a76b79 66c15deb .config console log report syz C ci-upstream-kasan-gce
2018/02/06 18:38 upstream e237f98a9c13 66c15deb .config console log report syz C ci-upstream-kasan-gce
2018/02/06 17:19 upstream e237f98a9c13 66c15deb .config console log report syz C ci-upstream-kasan-gce
2018/02/06 11:45 upstream e237f98a9c13 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/06 11:12 upstream e237f98a9c13 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/06 10:54 upstream e237f98a9c13 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/06 09:34 upstream 2deb41b24532 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/06 07:31 upstream 2deb41b24532 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/06 07:14 upstream 2deb41b24532 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/06 06:47 upstream 2deb41b24532 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/06 05:37 upstream 2deb41b24532 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/06 01:57 upstream 2deb41b24532 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/06 00:45 upstream 2deb41b24532 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 23:15 upstream 2deb41b24532 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 21:31 upstream 2deb41b24532 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 20:33 upstream 35277995e179 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 19:42 upstream 35277995e179 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 19:21 upstream 35277995e179 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 19:04 upstream 35277995e179 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 18:15 upstream 35277995e179 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 17:12 upstream 35277995e179 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 16:36 upstream 35277995e179 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 16:20 upstream 35277995e179 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 14:54 upstream 35277995e179 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 14:37 upstream 35277995e179 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 10:04 upstream 35277995e179 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 07:23 upstream 35277995e179 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 06:30 upstream 35277995e179 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 05:20 upstream 35277995e179 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 04:36 upstream 35277995e179 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 04:17 upstream 35277995e179 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 03:06 upstream 35277995e179 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 02:18 upstream 35277995e179 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 01:45 upstream 35277995e179 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/05 01:09 upstream 35277995e179 a1bc9d40 .config console log report syz C ci-upstream-kasan-gce
2018/02/07 10:35 net-next-old 617aebe6a97e 66c15deb .config console log report syz C ci-upstream-net-kasan-gce
2018/02/07 03:50 net-next-old 617aebe6a97e 66c15deb .config console log report syz C ci-upstream-net-kasan-gce
2018/02/06 18:09 net-next-old 617aebe6a97e 66c15deb .config console log report syz C ci-upstream-net-kasan-gce
2018/02/06 16:47 net-next-old 617aebe6a97e 66c15deb .config console log report syz C ci-upstream-net-kasan-gce
2018/02/06 11:18 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/06 11:01 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/06 10:20 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/06 09:32 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/06 07:55 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/06 06:55 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/06 05:54 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/06 05:19 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/06 01:39 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 22:55 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 21:02 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 20:07 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 19:16 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 17:49 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 16:55 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 15:53 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 15:40 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 14:55 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 14:28 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 13:36 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 09:42 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 06:48 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 06:12 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 05:20 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 05:02 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 04:19 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 02:35 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 01:42 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 00:40 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 00:32 net-next-old 617aebe6a97e a1bc9d40 .config console log report syz C ci-upstream-net-kasan-gce
2018/02/05 13:24 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 13:24 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 13:20 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 13:18 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 13:13 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 13:11 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 13:09 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 13:09 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 13:07 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 13:04 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 13:01 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 13:01 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:56 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:54 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:53 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:51 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:51 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:50 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:50 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:49 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:47 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:44 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:43 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:39 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:38 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:37 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:36 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:34 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:32 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:20 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:19 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:17 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:17 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:15 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:12 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:11 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:10 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:09 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:07 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/05 12:07 upstream 35277995e179 a1bc9d40 .config console log report ci-upstream-kasan-gce
2018/02/04 23:56 net-next-old 617aebe6a97e a1bc9d40 .config console log report ci-upstream-net-kasan-gce
* Struck through repros no longer work on HEAD.