bisecting cause commit starting from c92a9a461dff6140c539c61e457aa97df29517d6 building syzkaller on c9e7aeaef64e4e16a32baac1c66d772afbaf8ed0 testing commit c92a9a461dff6140c539c61e457aa97df29517d6 with gcc (GCC) 8.1.0 kernel signature: 45cabe4c0ce89d9d7ff1b88c5f855dff0b44a3a7 all runs: crashed: possible deadlock in sch_direct_xmit testing release v4.14 testing commit bebc6082da0a9f5d47a1ea2edc099bf671058bd4 with gcc (GCC) 8.1.0 kernel signature: 880163a83721df0692e2f5143266ff4c8045f06b all runs: crashed: possible deadlock in sch_direct_xmit testing release v4.13 testing commit 569dbb88e80deb68974ef6fdd6a13edb9d686261 with gcc (GCC) 8.1.0 kernel signature: d893e511c4f0a2d77f81c13e65ff75a658d6e39b all runs: OK # git bisect start bebc6082da0a9f5d47a1ea2edc099bf671058bd4 569dbb88e80deb68974ef6fdd6a13edb9d686261 Bisecting: 7300 revisions left to test after this (roughly 13 steps) [15d8ffc96464f6571ecf22043c45fad659f11bdd] Merge tag 'mmc-v4.14' of git://git.kernel.org/pub/scm/linux/kernel/git/ulfh/mmc testing commit 15d8ffc96464f6571ecf22043c45fad659f11bdd with gcc (GCC) 8.1.0 kernel signature: ec2b5fffc4c22c42f804c2bc9c1308d729c7a5ab all runs: OK # git bisect good 15d8ffc96464f6571ecf22043c45fad659f11bdd Bisecting: 3465 revisions left to test after this (roughly 12 steps) [e90937e756938f03d37d4cae7c82316a3a425944] Merge tag 'armsoc-devicetree' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc testing commit e90937e756938f03d37d4cae7c82316a3a425944 with gcc (GCC) 8.1.0 kernel signature: 7a8e0a4dd01a5db3822286f253814bff1b43e0c9 all runs: OK # git bisect good e90937e756938f03d37d4cae7c82316a3a425944 Bisecting: 1724 revisions left to test after this (roughly 11 steps) [a3583202e8292540fae38ed44bd49d77e35286c2] Merge tag 'drm-fixes-for-v4.14-rc3' of git://people.freedesktop.org/~airlied/linux testing commit a3583202e8292540fae38ed44bd49d77e35286c2 with gcc (GCC) 8.1.0 kernel signature: 745a70e4d6cd8160fb4433b67ee4af888590e0c1 all runs: OK # git bisect good a3583202e8292540fae38ed44bd49d77e35286c2 Bisecting: 861 revisions left to test after this (roughly 10 steps) [2cb3a34abd035756f9ba3cde12f44f4b3e4c234b] Merge tag 'drm-intel-fixes-2017-10-18-1' of git://anongit.freedesktop.org/drm/drm-intel into drm-fixes testing commit 2cb3a34abd035756f9ba3cde12f44f4b3e4c234b with gcc (GCC) 8.1.0 kernel signature: de7fda0332d5aab14d1d26d3674e2131c9b3af05 all runs: crashed: possible deadlock in sch_direct_xmit # git bisect bad 2cb3a34abd035756f9ba3cde12f44f4b3e4c234b Bisecting: 431 revisions left to test after this (roughly 9 steps) [00a534e5ea5c21b95f58cbb2f7918cc9fa82dd47] doc: Fix typo "8023.ad" in bonding documentation testing commit 00a534e5ea5c21b95f58cbb2f7918cc9fa82dd47 with gcc (GCC) 8.1.0 kernel signature: 39e49ea3b7777022e66a47d69eb3975eba3e1a9c all runs: crashed: possible deadlock in sch_direct_xmit # git bisect bad 00a534e5ea5c21b95f58cbb2f7918cc9fa82dd47 Bisecting: 215 revisions left to test after this (roughly 8 steps) [eae3df7e82318d798f45dedf111e241805ec7a4a] sh: sh7264: remove nonexistent GPIO_PH[0-7] to fix pinctrl registration testing commit eae3df7e82318d798f45dedf111e241805ec7a4a with gcc (GCC) 8.1.0 kernel signature: 2c77db04f146766202e62e9ae0c95411f7962084 all runs: OK # git bisect good eae3df7e82318d798f45dedf111e241805ec7a4a Bisecting: 124 revisions left to test after this (roughly 7 steps) [42b76d0e6b1fe0fcb90e0ff6b4d053d50597b031] Merge tag 'armsoc-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc testing commit 42b76d0e6b1fe0fcb90e0ff6b4d053d50597b031 with gcc (GCC) 8.1.0 kernel signature: 17c99f4976e0861de5cacd2552c281e01c01ea8f all runs: OK # git bisect good 42b76d0e6b1fe0fcb90e0ff6b4d053d50597b031 Bisecting: 62 revisions left to test after this (roughly 6 steps) [d51711c0557d6dbd26c63144aef32c7b3ec264b9] ip_gre: ipgre_tap device should keep dst testing commit d51711c0557d6dbd26c63144aef32c7b3ec264b9 with gcc (GCC) 8.1.0 kernel signature: 4d0d27ae82f413e43fe2a358aee5b72fb74452bd all runs: OK # git bisect good d51711c0557d6dbd26c63144aef32c7b3ec264b9 Bisecting: 36 revisions left to test after this (roughly 5 steps) [9a431ef9629fa6276aa8bd9ea87fb0728922bd6d] Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net testing commit 9a431ef9629fa6276aa8bd9ea87fb0728922bd6d with gcc (GCC) 8.1.0 kernel signature: 8da4623f0ed8386cf9199f4484ce1d3596d98be1 all runs: crashed: possible deadlock in sch_direct_xmit # git bisect bad 9a431ef9629fa6276aa8bd9ea87fb0728922bd6d Bisecting: 12 revisions left to test after this (roughly 4 steps) [9f775ead5e570e7e19015b9e4e2f3dd6e71a5935] l2tp: fix l2tp_eth module loading testing commit 9f775ead5e570e7e19015b9e4e2f3dd6e71a5935 with gcc (GCC) 8.1.0 kernel signature: 20b5bea0c1cf674d35ca66317330a3f4ffdb5b8b all runs: crashed: possible deadlock in sch_direct_xmit # git bisect bad 9f775ead5e570e7e19015b9e4e2f3dd6e71a5935 Bisecting: 6 revisions left to test after this (roughly 3 steps) [5a59a3a0ef0e546626a762d49dc06feaa204bab3] ppp: fix __percpu annotation testing commit 5a59a3a0ef0e546626a762d49dc06feaa204bab3 with gcc (GCC) 8.1.0 kernel signature: 873e7b15d0b96d3b6703377e949d30540f7196d8 all runs: OK # git bisect good 5a59a3a0ef0e546626a762d49dc06feaa204bab3 Bisecting: 3 revisions left to test after this (roughly 2 steps) [5513d08d29511c263c00933c00dd7a82fffda3c9] ip_gre: check packet length and mtu correctly in erspan_xmit testing commit 5513d08d29511c263c00933c00dd7a82fffda3c9 with gcc (GCC) 8.1.0 kernel signature: 7d3db63e791628925e35028f36bc4d079afacf45 all runs: OK # git bisect good 5513d08d29511c263c00933c00dd7a82fffda3c9 Bisecting: 1 revision left to test after this (roughly 1 step) [c84bed440e4e11a973e8c0254d0dfaccfca41fb0] ip_gre: erspan device should keep dst testing commit c84bed440e4e11a973e8c0254d0dfaccfca41fb0 with gcc (GCC) 8.1.0 kernel signature: 59217d0bbf1d1e829a441f296cdd1db1a0702fab all runs: crashed: possible deadlock in sch_direct_xmit # git bisect bad c84bed440e4e11a973e8c0254d0dfaccfca41fb0 Bisecting: 0 revisions left to test after this (roughly 0 steps) [c122fda271717f4fc618e0a31e833941fd5f1efd] ip_gre: set tunnel hlen properly in erspan_tunnel_init testing commit c122fda271717f4fc618e0a31e833941fd5f1efd with gcc (GCC) 8.1.0 kernel signature: d3d73fe2fc9b099401e4828237a85135fde499c9 all runs: OK # git bisect good c122fda271717f4fc618e0a31e833941fd5f1efd c84bed440e4e11a973e8c0254d0dfaccfca41fb0 is the first bad commit commit c84bed440e4e11a973e8c0254d0dfaccfca41fb0 Author: Xin Long Date: Sun Oct 1 22:00:56 2017 +0800 ip_gre: erspan device should keep dst The patch 'ip_gre: ipgre_tap device should keep dst' fixed the issue ipgre_tap dev mtu couldn't be updated in tx path. The same fix is needed for erspan as well. Fixes: 84e54fe0a5ea ("gre: introduce native tunnel support for ERSPAN") Signed-off-by: Xin Long Signed-off-by: David S. Miller net/ipv4/ip_gre.c | 1 + 1 file changed, 1 insertion(+) kernel signature: 59217d0bbf1d1e829a441f296cdd1db1a0702fab previous signature: d3d73fe2fc9b099401e4828237a85135fde499c9 revisions tested: 17, total time: 3h55m25.643258483s (build: 1h2m24.959067719s, test: 2h48m14.230726893s) first bad commit: c84bed440e4e11a973e8c0254d0dfaccfca41fb0 ip_gre: erspan device should keep dst cc: ["davem@davemloft.net" "kuznet@ms2.inr.ac.ru" "linux-kernel@vger.kernel.org" "lucien.xin@gmail.com" "netdev@vger.kernel.org" "yoshfuji@linux-ipv6.org"] crash: possible deadlock in sch_direct_xmit ============================================ WARNING: possible recursive locking detected 4.14.0-rc1-syzkaller #0 Not tainted -------------------------------------------- syz-executor860/19792 is trying to acquire lock: (_xmit_ETHER#2){+.-.}, at: [] spin_lock include/linux/spinlock.h:316 [inline] (_xmit_ETHER#2){+.-.}, at: [] __netif_tx_lock include/linux/netdevice.h:3500 [inline] (_xmit_ETHER#2){+.-.}, at: [] sch_direct_xmit+0x21d/0x5c0 net/sched/sch_generic.c:184 but task is already holding lock: (_xmit_ETHER#2){+.-.}, at: [] spin_lock include/linux/spinlock.h:316 [inline] (_xmit_ETHER#2){+.-.}, at: [] __netif_tx_lock include/linux/netdevice.h:3500 [inline] (_xmit_ETHER#2){+.-.}, at: [] sch_direct_xmit+0x21d/0x5c0 net/sched/sch_generic.c:184 other info that might help us debug this: Possible unsafe locking scenario: CPU0 ---- lock(_xmit_ETHER#2); lock(_xmit_ETHER#2); *** DEADLOCK *** May be due to missing lock nesting notation 9 locks held by syz-executor860/19792: #0: (rcu_read_lock){....}, at: [] arch_static_branch arch/x86/include/asm/jump_label.h:35 [inline] #0: (rcu_read_lock){....}, at: [] static_key_false include/linux/jump_label.h:141 [inline] #0: (rcu_read_lock){....}, at: [] netif_receive_skb_internal+0x91/0x5a0 net/core/dev.c:4513 #1: (k-slock-AF_INET){+...}, at: [] spin_trylock include/linux/spinlock.h:326 [inline] #1: (k-slock-AF_INET){+...}, at: [] icmp_xmit_lock net/ipv4/icmp.c:219 [inline] #1: (k-slock-AF_INET){+...}, at: [] icmp_send+0x4e5/0x2250 net/ipv4/icmp.c:668 #2: (rcu_read_lock_bh){....}, at: [] lwtunnel_xmit_redirect include/net/lwtunnel.h:91 [inline] #2: (rcu_read_lock_bh){....}, at: [] ip_finish_output2+0x280/0x1400 net/ipv4/ip_output.c:213 #3: (rcu_read_lock_bh){....}, at: [] __dev_queue_xmit+0x280/0x2a80 net/core/dev.c:3410 #4: (dev->qdisc_running_key ?: &qdisc_running_key){+...}, at: [] dev_queue_xmit+0xb/0x10 net/core/dev.c:3509 #5: (_xmit_ETHER#2){+.-.}, at: [] spin_lock include/linux/spinlock.h:316 [inline] #5: (_xmit_ETHER#2){+.-.}, at: [] __netif_tx_lock include/linux/netdevice.h:3500 [inline] #5: (_xmit_ETHER#2){+.-.}, at: [] sch_direct_xmit+0x21d/0x5c0 net/sched/sch_generic.c:184 #6: (rcu_read_lock_bh){....}, at: [] lwtunnel_xmit_redirect include/net/lwtunnel.h:91 [inline] #6: (rcu_read_lock_bh){....}, at: [] ip_finish_output2+0x280/0x1400 net/ipv4/ip_output.c:213 #7: (rcu_read_lock_bh){....}, at: [] __dev_queue_xmit+0x280/0x2a80 net/core/dev.c:3410 #8: (dev->qdisc_running_key ?: &qdisc_running_key){+...}, at: [] dev_queue_xmit+0xb/0x10 net/core/dev.c:3509 stack backtrace: CPU: 1 PID: 19792 Comm: syz-executor860 Not tainted 4.14.0-rc1-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:16 [inline] dump_stack+0x145/0x1f0 lib/dump_stack.c:52 print_deadlock_bug kernel/locking/lockdep.c:1797 [inline] check_deadlock kernel/locking/lockdep.c:1844 [inline] validate_chain kernel/locking/lockdep.c:2453 [inline] __lock_acquire.cold.78+0x251/0x554 kernel/locking/lockdep.c:3498 lock_acquire+0x1dc/0x520 kernel/locking/lockdep.c:4002 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline] _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:151 spin_lock include/linux/spinlock.h:316 [inline] __netif_tx_lock include/linux/netdevice.h:3500 [inline] sch_direct_xmit+0x21d/0x5c0 net/sched/sch_generic.c:184 __dev_xmit_skb net/core/dev.c:3177 [inline] __dev_queue_xmit+0x1dd9/0x2a80 net/core/dev.c:3444 dev_queue_xmit+0xb/0x10 net/core/dev.c:3509 neigh_resolve_output+0x54d/0x9c0 net/core/neighbour.c:1350 neigh_output include/net/neighbour.h:481 [inline] ip_finish_output2+0x855/0x1400 net/ipv4/ip_output.c:229 ip_finish_output+0x713/0xe90 net/ipv4/ip_output.c:317 NF_HOOK_COND include/linux/netfilter.h:238 [inline] ip_mc_output+0x265/0x11f0 net/ipv4/ip_output.c:390 dst_output include/net/dst.h:458 [inline] ip_local_out+0x78/0x130 net/ipv4/ip_output.c:124 iptunnel_xmit+0x4fc/0x860 net/ipv4/ip_tunnel_core.c:91 ip_tunnel_xmit+0x1204/0x3e04 net/ipv4/ip_tunnel.c:786 __gre_xmit+0x494/0xa00 net/ipv4/ip_gre.c:436 erspan_xmit+0x44f/0x12c0 net/ipv4/ip_gre.c:742 __netdev_start_xmit include/linux/netdevice.h:4000 [inline] netdev_start_xmit include/linux/netdevice.h:4009 [inline] xmit_one net/core/dev.c:2979 [inline] dev_hard_start_xmit+0x22e/0xa00 net/core/dev.c:2995 sch_direct_xmit+0x2ab/0x5c0 net/sched/sch_generic.c:186 __dev_xmit_skb net/core/dev.c:3177 [inline] __dev_queue_xmit+0x1dd9/0x2a80 net/core/dev.c:3444 dev_queue_xmit+0xb/0x10 net/core/dev.c:3509 neigh_resolve_output+0x54d/0x9c0 net/core/neighbour.c:1350 neigh_output include/net/neighbour.h:481 [inline] ip_finish_output2+0x855/0x1400 net/ipv4/ip_output.c:229 ip_finish_output+0x713/0xe90 net/ipv4/ip_output.c:317 NF_HOOK_COND include/linux/netfilter.h:238 [inline] ip_mc_output+0x265/0x11f0 net/ipv4/ip_output.c:390 dst_output include/net/dst.h:458 [inline] ip_local_out+0x78/0x130 net/ipv4/ip_output.c:124 ip_send_skb+0x36/0xa0 net/ipv4/ip_output.c:1414 ip_push_pending_frames+0x4d/0x70 net/ipv4/ip_output.c:1434 icmp_push_reply+0x321/0x590 net/ipv4/icmp.c:394 icmp_send+0x1455/0x2250 net/ipv4/icmp.c:741 ip_options_compile+0xca1/0x2510 net/ipv4/ip_options.c:471 ip_rcv_options net/ipv4/ip_input.c:284 [inline] ip_rcv_finish+0x6f2/0x2510 net/ipv4/ip_input.c:365 NF_HOOK include/linux/netfilter.h:249 [inline] ip_rcv+0xd4c/0x19d8 net/ipv4/ip_input.c:493 __netif_receive_skb_core+0x2094/0x35b0 net/core/dev.c:4428 __netif_receive_skb+0x1f/0x1a0 net/core/dev.c:4466 netif_receive_skb_internal+0xdc/0x5a0 net/core/dev.c:4539 netif_receive_skb+0x95/0x320 net/core/dev.c:4563 tun_rx_batched.isra.46+0x418/0x880 drivers/net/tun.c:1218 tun_get_user+0x1307/0x21b0 drivers/net/tun.c:1555 tun_chr_write_iter+0xd1/0x1a0 drivers/net/tun.c:1581 call_write_iter include/linux/fs.h:1770 [inline] do_iter_readv_writev+0x4a3/0xaa0 fs/read_write.c:673 do_iter_write+0x130/0x530 fs/read_write.c:952 vfs_writev+0x16b/0x320 fs/read_write.c:997 do_writev+0xf3/0x340 fs/read_write.c:1032 SYSC_writev fs/read_write.c:1105 [inline] SyS_writev+0xb/0x10 fs/read_write.c:1102 entry_SYSCALL_64_fastpath+0x23/0xc2 RIP: 0033:0x4450a0 RSP: 002b:00007ffd0b7e4008 EFLAGS: 00000246 ORIG_RAX: 0000000000000014 RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00000000004450a0 RDX: 0000000000000001 RSI: 00007ffd0b7e4040 RDI: 0000000000000003 RBP: 0000000000000086 R08: 0000000000000000 R09: 0000000000000023