bisecting cause commit starting from 5ded5871030eb75017639148da0a58931dfbfc25 building syzkaller on f42dee6d5e501a061cdbb807672361369bf28492 testing commit 5ded5871030eb75017639148da0a58931dfbfc25 with gcc (GCC) 8.1.0 kernel signature: a408248830fd00f95633ea7da028e8cb64982fb2 run #0: boot failed: create image operation failed: &{Code:QUOTA_EXCEEDED Location: Message:Quota 'CPUS' exceeded. Limit: 500.0 in region us-central1. ForceSendFields:[] NullFields:[]}. run #1: crashed: INFO: task hung in rtnl_lock run #2: crashed: INFO: task hung in rtnl_lock run #3: crashed: INFO: task hung in rtnl_lock run #4: crashed: INFO: task hung in rtnl_lock run #5: crashed: INFO: task hung in rtnl_lock run #6: crashed: INFO: task hung in rtnl_lock run #7: crashed: INFO: task hung in rtnl_lock run #8: crashed: INFO: task hung in rtnl_lock run #9: crashed: INFO: task hung in rtnl_lock testing release v4.20 testing commit 8fe28cb58bcb235034b64cbbb7550a8a43fd88be with gcc (GCC) 8.1.0 kernel signature: 8f9b97fa90432d24c6628ccefa55cbd10682ea84 all runs: crashed: INFO: task hung in rtnl_lock testing release v4.19 testing commit 84df9525b0c27f3ebc2ebb1864fa62a97fdedb7d with gcc (GCC) 8.1.0 kernel signature: 8cdef83160327dc5341792c6adb33aa312e6aa06 all runs: crashed: INFO: task hung in rtnl_lock testing release v4.18 testing commit 94710cac0ef4ee177a63b5227664b38c95bbf703 with gcc (GCC) 8.1.0 kernel signature: 88a18ec1f7fff908e76d46894d717010bf8cdf43 all runs: crashed: INFO: task hung in rtnl_lock testing release v4.17 testing commit 29dcea88779c856c7dc92040a0c01233263101d4 with gcc (GCC) 8.1.0 kernel signature: b274960ce192b21f5b15e0d2f421763aed3698af all runs: crashed: INFO: task hung in rtnl_lock testing release v4.16 testing commit 0adb32858b0bddf4ada5f364a84ed60b196dbcda with gcc (GCC) 8.1.0 kernel signature: 5c1fcd136711ecf9daa7624a170c793a3407b909 all runs: crashed: INFO: task hung in rtnl_lock testing release v4.15 testing commit d8a5b80568a9cb66810e75b182018e9edb68e8ff with gcc (GCC) 8.1.0 kernel signature: 4a97a522d1a4481e44b21581ef0c99657635d923 all runs: crashed: INFO: task hung in rtnl_lock testing release v4.14 testing commit bebc6082da0a9f5d47a1ea2edc099bf671058bd4 with gcc (GCC) 8.1.0 kernel signature: e444ce476feecb4b4f34ccc41307da080f342f6e run #0: crashed: INFO: task hung in rtnl_lock run #1: crashed: INFO: task hung in rtnl_lock run #2: crashed: INFO: task hung in rtnetlink_rcv_msg run #3: crashed: INFO: task hung in rtnl_lock run #4: crashed: INFO: task hung in rtnl_lock run #5: crashed: INFO: task hung in rtnl_lock run #6: crashed: INFO: task hung in rtnl_lock run #7: crashed: INFO: task hung in rtnl_lock run #8: crashed: INFO: task hung in rtnl_lock run #9: basic kernel testing failed: failed to copy test binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/jobs/linux/workdir/image/key" "/tmp/syz-executor705616791" "root@10.128.1.58:./syz-executor705616791"]: exit status 1 Connection timed out during banner exchange lost connection testing release v4.13 testing commit 569dbb88e80deb68974ef6fdd6a13edb9d686261 with gcc (GCC) 8.1.0 kernel signature: 3930e6e8872cf6f8b9aaf03b59573ef03772fd7a all runs: crashed: INFO: task hung in rtnl_lock testing release v4.12 testing commit 6f7da290413ba713f0cdd9ff1a2a9bb129ef4f6c with gcc (GCC) 8.1.0 kernel signature: 8b1079cf13bd1e9458d601a475604e2a6308863b run #0: crashed: INFO: task hung in rtnl_lock run #1: crashed: INFO: task hung in rtnl_lock run #2: crashed: INFO: task hung in rtnl_lock run #3: crashed: INFO: task hung in rtnl_lock run #4: crashed: INFO: task hung in rtnl_lock run #5: crashed: INFO: task hung in rtnl_lock run #6: crashed: INFO: task hung in rtnl_lock run #7: crashed: INFO: task hung in rtnl_lock run #8: crashed: INFO: task hung in rtnl_lock run #9: crashed: INFO: task hung in rtnetlink_rcv testing release v4.11 testing commit a351e9b9fc24e982ec2f0e76379a49826036da12 with gcc (GCC) 7.3.0 kernel signature: a49962cdfa75c862ab310a749e85333e09405f79 all runs: crashed: INFO: task hung in rtnl_lock testing release v4.10 testing commit c470abd4fde40ea6a0846a2beab642a578c0b8cd with gcc (GCC) 5.5.0 kernel signature: 7c5b3218333f405790eeeb724ab88400445dd9bc run #0: crashed: INFO: task hung in rtnl_lock run #1: crashed: INFO: task hung in rtnl_lock run #2: crashed: INFO: task hung in rtnl_lock run #3: crashed: INFO: task hung in rtnl_lock run #4: crashed: INFO: task hung in rtnl_lock run #5: crashed: INFO: task hung in rtnl_lock run #6: crashed: INFO: task hung in rtnl_lock run #7: crashed: INFO: task hung in rtnl_lock run #8: basic kernel testing failed: failed to copy test binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/jobs/linux/workdir/image/key" "/tmp/syz-executor149715678" "root@10.128.1.53:./syz-executor149715678"]: exit status 1 Connection timed out during banner exchange lost connection run #9: basic kernel testing failed: failed to copy test binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/jobs/linux/workdir/image/key" "/tmp/syz-executor449911973" "root@10.128.0.252:./syz-executor449911973"]: exit status 1 Connection timed out during banner exchange lost connection testing release v4.9 testing commit 69973b830859bc6529a7a0468ba0d80ee5117826 with gcc (GCC) 5.5.0 kernel signature: 6994d353d5d4b1f6e279680eaaa38f2658be1e3c run #0: crashed: INFO: task hung in rtnl_lock run #1: crashed: INFO: task hung in rtnl_lock run #2: crashed: INFO: task hung in rtnl_lock run #3: crashed: INFO: task hung in rtnl_lock run #4: crashed: INFO: task hung in rtnl_lock run #5: crashed: INFO: task hung in rtnl_lock run #6: crashed: INFO: task hung in rtnl_lock run #7: crashed: INFO: task hung in rtnl_lock run #8: crashed: INFO: task hung in rtnl_lock run #9: OK testing release v4.8 testing commit c8d2bc9bc39ebea8437fd974fdbc21847bb897a3 with gcc (GCC) 5.5.0 kernel signature: 4422a8101e6442ccd249286b5a49d4da79a8b205 all runs: crashed: INFO: task hung in rtnl_lock testing release v4.7 testing commit 523d939ef98fd712632d93a5a2b588e477a7565e with gcc (GCC) 5.5.0 kernel signature: c207e7a36be4f70576f665e652a3b6cf55031ef4 all runs: crashed: INFO: task hung in rtnl_lock testing release v4.6 testing commit 2dcd0af568b0cf583645c8a317dd12e344b1c72a with gcc (GCC) 5.5.0 kernel signature: 64fbe97a05a6a5048204293109754eb23bc3ca30 all runs: crashed: INFO: task hung in rtnl_lock testing release v4.5 testing commit b562e44f507e863c6792946e4e1b1449fbbac85d with gcc (GCC) 5.5.0 kernel signature: e33e5011d37ac393f8027c5e587889bf646fae5b run #0: crashed: INFO: task hung in rtnetlink_rcv run #1: crashed: INFO: task hung in rtnl_lock run #2: crashed: INFO: task hung in rtnl_lock run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK testing release v4.4 testing commit afd2ff9b7e1b367172f18ba7f693dfb62bdcb2dc with gcc (GCC) 5.5.0 kernel signature: 0c1c6d6b8dc8061bdca4b99312d924f595777dc2 run #0: boot failed: create image operation failed: &{Code:QUOTA_EXCEEDED Location: Message:Quota 'CPUS' exceeded. Limit: 500.0 in region us-central1. ForceSendFields:[] NullFields:[]}. run #1: crashed: INFO: task hung in rtnl_lock run #2: crashed: INFO: task hung in rtnetlink_rcv run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK testing release v4.3 testing commit 6a13feb9c82803e2b815eca72fa7a9f5561d7861 with gcc (GCC) 5.5.0 kernel signature: 3a9f739bb7a9b470f8451275719663ff86237a5b run #0: boot failed: create image operation failed: &{Code:QUOTA_EXCEEDED Location: Message:Quota 'CPUS' exceeded. Limit: 500.0 in region us-central1. ForceSendFields:[] NullFields:[]}. run #1: crashed: INFO: task hung in rtnl_lock run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK testing release v4.2 testing commit 64291f7db5bd8150a74ad2036f1037e6a0428df2 with gcc (GCC) 5.5.0 kernel signature: 36b9056d4882fc7395e05ce524dce2463fe5fbf4 run #0: crashed: INFO: task hung in rtnetlink_rcv run #1: OK run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK testing release v4.1 testing commit b953c0d234bc72e8489d3bf51a276c5c4ec85345 with gcc (GCC) 5.5.0 kernel signature: 28c1bf8be3d9811064e0bd61a06e65829ec8e50f all runs: crashed: INFO: task hung in rtnl_lock revisions tested: 21, total time: 4h27m5.318819858s (build: 1h30m19.461805727s, test: 2h50m34.491017627s) the crash already happened on the oldest tested release commit msg: Linux 4.1 crash: INFO: task hung in rtnl_lock bridge0: port 1(bridge_slave_0) entered forwarding state bridge0: port 2(bridge_slave_1) entered forwarding state bridge0: port 1(bridge_slave_0) entered forwarding state bridge0: port 2(bridge_slave_1) entered forwarding state INFO: task kworker/1:1:627 blocked for more than 140 seconds. Not tainted 4.1.0-syzkaller #0 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kworker/1:1 D ffff88012a8f7c28 13192 627 2 0x00000000 Workqueue: ipv6_addrconf addrconf_dad_work ffff88012a8f7c28 000000002a8f7c28 ffff88012a8f0650 ffff880100000000 ffff88012a8f8000 ffffffff83397688 ffff8800b39aea00 ffff88012a8f0650 0000000000000286 ffff88012a8f7c48 ffffffff82642472 ffffffff83397680 Call Trace: [] schedule+0x32/0x80 kernel/sched/core.c:2826 [] schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:2858 [] __mutex_lock_common kernel/locking/mutex.c:578 [inline] [] mutex_lock_nested+0x195/0x610 kernel/locking/mutex.c:617 [] rtnl_lock+0x12/0x20 net/core/rtnetlink.c:70 [] addrconf_dad_work+0x28/0x330 net/ipv6/addrconf.c:3501 [] process_one_work+0x214/0x8d0 kernel/workqueue.c:2025 [] worker_thread+0x4b/0x470 kernel/workqueue.c:2157 [] kthread+0xea/0x100 drivers/block/aoe/aoecmd.c:1312 [] ret_from_fork+0x42/0x70 arch/x86/kernel/entry_64.S:639 3 locks held by kworker/1:1/627: #0: ("%s"("ipv6_addrconf")){.+.+..}, at: [] set_work_data kernel/workqueue.c:606 [inline] #0: ("%s"("ipv6_addrconf")){.+.+..}, at: [] set_work_pool_and_clear_pending kernel/workqueue.c:634 [inline] #0: ("%s"("ipv6_addrconf")){.+.+..}, at: [] process_one_work+0x177/0x8d0 kernel/workqueue.c:2018 #1: ((&(&ifa->dad_work)->work)){+.+...}, at: [] set_work_data kernel/workqueue.c:606 [inline] #1: ((&(&ifa->dad_work)->work)){+.+...}, at: [] set_work_pool_and_clear_pending kernel/workqueue.c:634 [inline] #1: ((&(&ifa->dad_work)->work)){+.+...}, at: [] process_one_work+0x177/0x8d0 kernel/workqueue.c:2018 #2: (rtnl_mutex){+.+.+.}, at: [] rtnl_lock+0x12/0x20 net/core/rtnetlink.c:70 sending NMI to all CPUs: NMI backtrace for cpu 0 CPU: 0 PID: 5691 Comm: syz-executor.3 Not tainted 4.1.0-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 task: ffff8800b642e890 ti: ffff8800aefbc000 task.ti: ffff8800aefbc000 RIP: 0010:[] [] mark_held_locks+0x69/0xc0 kernel/locking/lockdep.c:2533 RSP: 0018:ffff8800aefbf8d8 EFLAGS: 00000097 RAX: 0000000000000004 RBX: 0000000000000000 RCX: 0000000000000002 RDX: 0000000000000006 RSI: ffff8800b642f0f8 RDI: ffff8800b642e890 RBP: ffff8800aefbf908 R08: 0000000000000001 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000007 R13: ffff8800b642e890 R14: 0000000000000006 R15: ffff8800b642f0f8 FS: 00007f1397880700(0000) GS:ffff88012c000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000002962978 CR3: 00000000b39ac000 CR4: 00000000001407f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Stack: ffff8800b39a6000 ffff8800b642e890 ffffffff812daeed 0000000000000286 ffff88012bc00700 0000000000000000 ffff8800aefbf928 ffffffff811c4a35 ffff88012c01ea90 ffff8800aff68000 ffff8800aefbf938 ffffffff811c4aed Call Trace: [] __trace_hardirqs_on_caller kernel/locking/lockdep.c:2565 [inline] [] trace_hardirqs_on_caller+0x155/0x200 kernel/locking/lockdep.c:2612 [] trace_hardirqs_on+0xd/0x10 kernel/locking/lockdep.c:2619 [] kfree+0x1ad/0x4b0 mm/slab.c:3586 [] skb_free_head+0x19/0x60 net/core/skbuff.c:617 [] pskb_expand_head+0xd8/0x260 net/core/skbuff.c:1214 [] netlink_trim+0x91/0xd0 net/netlink/af_netlink.c:1745 [] netlink_unicast+0x39/0x2e0 net/netlink/af_netlink.c:1779 [] rtnetlink_send+0x4d/0x80 net/core/rtnetlink.c:629 [] tcf_add_notify net/sched/act_api.c:920 [inline] [] tcf_action_add net/sched/act_api.c:941 [inline] [] tc_ctl_action+0x176/0x240 net/sched/act_api.c:978 [] rtnetlink_rcv_msg+0x83/0x230 net/core/rtnetlink.c:3250 [] netlink_rcv_skb+0xa9/0xd0 net/netlink/af_netlink.c:2843 [] rtnetlink_rcv+0x29/0x40 net/core/rtnetlink.c:3256 [] netlink_unicast_kernel net/netlink/af_netlink.c:1763 [inline] [] netlink_unicast+0x1ca/0x2e0 net/netlink/af_netlink.c:1789 [] netlink_sendmsg+0x310/0x3d0 net/netlink/af_netlink.c:2353 [] sock_sendmsg_nosec net/socket.c:613 [inline] [] sock_sendmsg+0x35/0x40 net/socket.c:623 [] ___sys_sendmsg+0x2c3/0x2d0 net/socket.c:1955 [] __sys_sendmsg+0x3d/0x80 net/socket.c:1989 [] SYSC_sendmsg net/socket.c:2000 [inline] [] SyS_sendmsg+0xd/0x20 net/socket.c:1996 [] system_call_fastpath+0x16/0x7a Code: 66 90 83 c3 01 41 39 9d 60 08 00 00 7e 4c 48 63 c3 44 89 f2 48 8d 04 80 49 8d 34 c7 0f b6 46 22 a8 03 41 0f 45 d4 83 fa 0c 77 27 04 74 d3 4c 89 ef e8 3b f8 ff ff 85 c0 75 c7 48 83 c4 08 5b NMI backtrace for cpu 1 CPU: 1 PID: 867 Comm: khungtaskd Not tainted 4.1.0-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 task: ffff88012adde990 ti: ffff88012a88c000 task.ti: ffff88012a88c000 RIP: 0010:[] [] native_write_msr_safe+0xa/0x10 arch/x86/include/asm/msr.h:95 RSP: 0018:ffff88012a88fd08 EFLAGS: 00000082 RAX: 0000000000000400 RBX: 0000000000000001 RCX: 0000000000000830 RDX: 0000000000000001 RSI: 0000000000000400 RDI: 0000000000000830 RBP: ffff88012a88fd08 R08: 0000000000000000 R09: 0000000000000003 R10: ffff88012adde990 R11: 0000000000000001 R12: ffffffff8341a8c8 R13: 0000000000080000 R14: 0000000000000001 R15: 000000000000a120 FS: 0000000000000000(0000) GS:ffff88012c100000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007fff1c556fe8 CR3: 0000000128c17000 CR4: 00000000001407e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Stack: ffff88012a88fd68 ffffffff810c517f ffff88012a88fd78 0000000000000296 000000020000000a 0000000000000002 ffff88012a88fd88 0000000000000040 000000000000d3c0 0000000000000001 ffff88012a8f0650 000000000000008c Call Trace: [] paravirt_write_msr arch/x86/include/asm/paravirt.h:133 [inline] [] native_x2apic_icr_write arch/x86/include/asm/apic.h:168 [inline] [] __x2apic_send_IPI_dest arch/x86/include/asm/x2apic.h:26 [inline] [] __x2apic_send_IPI_mask+0x10f/0x1a0 arch/x86/kernel/apic/x2apic_phys.c:52 [] x2apic_send_IPI_mask+0xe/0x10 arch/x86/kernel/apic/x2apic_cluster.c:79 [] arch_trigger_all_cpu_backtrace+0x33d/0x350 arch/x86/kernel/apic/hw_nmi.c:89 [] trigger_all_cpu_backtrace include/linux/nmi.h:43 [inline] [] check_hung_task kernel/hung_task.c:125 [inline] [] check_hung_uninterruptible_tasks kernel/hung_task.c:182 [inline] [] watchdog+0x47e/0x6c0 kernel/hung_task.c:238 [] kthread+0xea/0x100 drivers/block/aoe/aoecmd.c:1312 [] ret_from_fork+0x42/0x70 arch/x86/kernel/entry_64.S:639 Code: 00 55 89 f9 48 89 e5 0f 32 45 31 c0 48 89 d7 44 89 06 89 c6 5d 48 c1 e7 20 48 89 f8 48 09 f0 c3 90 55 89 f0 89 f9 48 89 e5 0f 30 <31> c0 5d c3 66 90 55 89 f9 48 89 e5 0f 33 48 89 d7 89 c1 5d 48