bisecting cause commit starting from 64c0133eb88a3b0c11c42580a520fe78b71b3932 building syzkaller on f42dee6d5e501a061cdbb807672361369bf28492 testing commit 64c0133eb88a3b0c11c42580a520fe78b71b3932 with gcc (GCC) 8.1.0 run #0: crashed: INFO: task hung in switchdev_deferred_process_work run #1: crashed: INFO: task hung in linkwatch_event run #2: crashed: INFO: task hung in addrconf_dad_work run #3: crashed: INFO: task hung in addrconf_dad_work run #4: crashed: INFO: task hung in addrconf_dad_work run #5: crashed: INFO: task hung in addrconf_dad_work run #6: crashed: INFO: task hung in switchdev_deferred_process_work run #7: crashed: INFO: task hung in addrconf_dad_work run #8: crashed: INFO: task hung in switchdev_deferred_process_work run #9: crashed: INFO: task hung in addrconf_dad_work testing release v4.20 testing commit 8fe28cb58bcb235034b64cbbb7550a8a43fd88be with gcc (GCC) 8.1.0 run #0: crashed: INFO: task hung in addrconf_dad_work run #1: crashed: INFO: task hung in addrconf_dad_work run #2: crashed: INFO: task hung in switchdev_deferred_process_work run #3: crashed: INFO: task hung in switchdev_deferred_process_work run #4: crashed: INFO: task hung in linkwatch_event run #5: crashed: INFO: task hung in addrconf_dad_work run #6: crashed: INFO: task hung in linkwatch_event run #7: crashed: INFO: task hung in addrconf_dad_work run #8: crashed: INFO: task hung in linkwatch_event run #9: crashed: INFO: task hung in addrconf_dad_work testing release v4.19 testing commit 84df9525b0c27f3ebc2ebb1864fa62a97fdedb7d with gcc (GCC) 8.1.0 run #0: crashed: INFO: task hung in addrconf_dad_work run #1: crashed: INFO: task hung in switchdev_deferred_process_work run #2: crashed: INFO: task hung in addrconf_dad_work run #3: crashed: INFO: task hung in addrconf_dad_work run #4: crashed: INFO: task hung in addrconf_dad_work run #5: crashed: INFO: task hung in addrconf_dad_work run #6: crashed: INFO: task hung in addrconf_dad_work run #7: crashed: INFO: task hung in addrconf_dad_work run #8: crashed: INFO: task hung in linkwatch_event run #9: crashed: INFO: task hung in addrconf_dad_work testing release v4.18 testing commit 94710cac0ef4ee177a63b5227664b38c95bbf703 with gcc (GCC) 8.1.0 run #0: crashed: INFO: task hung in addrconf_dad_work run #1: crashed: INFO: task hung in addrconf_dad_work run #2: crashed: INFO: task hung in addrconf_dad_work run #3: crashed: INFO: task hung in addrconf_dad_work run #4: crashed: INFO: task hung in addrconf_dad_work run #5: crashed: INFO: task hung in addrconf_dad_work run #6: crashed: INFO: task hung in addrconf_dad_work run #7: crashed: INFO: task hung in linkwatch_event run #8: crashed: INFO: task hung in addrconf_dad_work run #9: crashed: INFO: task hung in addrconf_dad_work testing release v4.17 testing commit 29dcea88779c856c7dc92040a0c01233263101d4 with gcc (GCC) 8.1.0 run #0: crashed: INFO: task hung in switchdev_deferred_process_work run #1: crashed: INFO: task hung in linkwatch_event run #2: crashed: INFO: task hung in linkwatch_event run #3: crashed: INFO: task hung in addrconf_dad_work run #4: crashed: INFO: task hung in switchdev_deferred_process_work run #5: crashed: INFO: task hung in switchdev_deferred_process_work run #6: crashed: INFO: task hung in switchdev_deferred_process_work run #7: crashed: INFO: task hung in switchdev_deferred_process_work run #8: crashed: INFO: task hung in addrconf_dad_work run #9: crashed: INFO: task hung in addrconf_dad_work testing release v4.16 testing commit 0adb32858b0bddf4ada5f364a84ed60b196dbcda with gcc (GCC) 8.1.0 run #0: crashed: INFO: task hung in reg_check_chans_work run #1: crashed: INFO: task hung in linkwatch_event run #2: crashed: INFO: task hung in linkwatch_event run #3: crashed: INFO: task hung in switchdev_deferred_process_work run #4: crashed: INFO: task hung in addrconf_dad_work run #5: crashed: INFO: task hung in addrconf_dad_work run #6: crashed: INFO: task hung in addrconf_dad_work run #7: crashed: INFO: task hung in addrconf_dad_work run #8: crashed: INFO: task hung in addrconf_dad_work run #9: crashed: INFO: task hung in linkwatch_event testing release v4.15 testing commit d8a5b80568a9cb66810e75b182018e9edb68e8ff with gcc (GCC) 8.1.0 run #0: crashed: INFO: task hung in addrconf_dad_work run #1: crashed: INFO: task hung in addrconf_dad_work run #2: crashed: INFO: task hung in reg_check_chans_work run #3: crashed: INFO: task hung in addrconf_dad_work run #4: crashed: INFO: task hung in addrconf_dad_work run #5: crashed: INFO: task hung in addrconf_dad_work run #6: crashed: INFO: task hung in addrconf_dad_work run #7: crashed: INFO: task hung in addrconf_dad_work run #8: crashed: INFO: task hung in addrconf_dad_work run #9: crashed: INFO: task hung in bond_netdev_notify_work testing release v4.14 testing commit bebc6082da0a9f5d47a1ea2edc099bf671058bd4 with gcc (GCC) 8.1.0 run #0: crashed: INFO: task hung in addrconf_dad_work run #1: crashed: INFO: task hung in switchdev_deferred_process_work run #2: crashed: INFO: task hung in addrconf_dad_work run #3: crashed: INFO: task hung in addrconf_dad_work run #4: crashed: INFO: task hung in switchdev_deferred_process_work run #5: crashed: INFO: task hung in switchdev_deferred_process_work run #6: crashed: INFO: task hung in addrconf_dad_work run #7: crashed: INFO: task hung in addrconf_dad_work run #8: crashed: INFO: task hung in addrconf_dad_work run #9: crashed: INFO: task hung in addrconf_verify_work testing release v4.13 testing commit 569dbb88e80deb68974ef6fdd6a13edb9d686261 with gcc (GCC) 8.1.0 all runs: crashed: INFO: task hung in addrconf_dad_work testing release v4.12 testing commit 6f7da290413ba713f0cdd9ff1a2a9bb129ef4f6c with gcc (GCC) 8.1.0 all runs: crashed: INFO: task hung in addrconf_dad_work testing release v4.11 testing commit a351e9b9fc24e982ec2f0e76379a49826036da12 with gcc (GCC) 7.3.0 run #0: crashed: INFO: task hung in addrconf_dad_work run #1: crashed: INFO: task hung in addrconf_dad_work run #2: crashed: INFO: task hung in addrconf_dad_work run #3: crashed: INFO: task hung in addrconf_dad_work run #4: crashed: INFO: task hung in addrconf_dad_work run #5: crashed: INFO: task hung in addrconf_dad_work run #6: crashed: INFO: task hung in addrconf_dad_work run #7: crashed: INFO: task hung in addrconf_dad_work run #8: OK run #9: OK testing release v4.10 testing commit c470abd4fde40ea6a0846a2beab642a578c0b8cd with gcc (GCC) 5.5.0 run #0: crashed: INFO: task hung in addrconf_dad_work run #1: crashed: INFO: task hung in addrconf_dad_work run #2: crashed: INFO: task hung in addrconf_dad_work run #3: crashed: INFO: task hung in addrconf_dad_work run #4: crashed: INFO: task hung in reg_check_chans_work run #5: crashed: INFO: task hung in addrconf_dad_work run #6: crashed: INFO: task hung in addrconf_dad_work run #7: basic kernel testing failed: failed to copy test binary to VM: failed to run ["scp" "-P" "43548" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/usr/local/google/home/dvyukov/syzkaller/ci-bisect2/jobs/linux/workdir/image/key" "/tmp/syz-executor030933081" "root@localhost:/syz-executor030933081"]: exit status 1 ssh: connect to host localhost port 43548: Connection timed out lost connection run #8: basic kernel testing failed: failed to copy test binary to VM: failed to run ["scp" "-P" "31350" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/usr/local/google/home/dvyukov/syzkaller/ci-bisect2/jobs/linux/workdir/image/key" "/tmp/syz-executor634218995" "root@localhost:/syz-executor634218995"]: exit status 1 ssh: connect to host localhost port 31350: Connection timed out lost connection run #9: OK testing release v4.9 testing commit 69973b830859bc6529a7a0468ba0d80ee5117826 with gcc (GCC) 5.5.0 run #0: crashed: INFO: task hung in addrconf_dad_work run #1: crashed: INFO: task hung in addrconf_dad_work run #2: crashed: INFO: task hung in addrconf_dad_work run #3: crashed: INFO: task hung in addrconf_dad_work run #4: crashed: INFO: task hung in addrconf_dad_work run #5: crashed: INFO: task hung in addrconf_dad_work run #6: crashed: INFO: task hung in addrconf_dad_work run #7: crashed: INFO: task hung in addrconf_dad_work run #8: crashed: INFO: task hung in reg_check_chans_work run #9: OK testing release v4.8 testing commit c8d2bc9bc39ebea8437fd974fdbc21847bb897a3 with gcc (GCC) 5.5.0 run #0: crashed: INFO: task hung in addrconf_dad_work run #1: crashed: INFO: task hung in addrconf_dad_work run #2: crashed: INFO: task hung in linkwatch_event run #3: crashed: INFO: task hung in addrconf_dad_work run #4: crashed: INFO: task hung in addrconf_dad_work run #5: crashed: INFO: task hung in linkwatch_event run #6: crashed: INFO: task hung in linkwatch_event run #7: crashed: INFO: task hung in addrconf_dad_work run #8: crashed: INFO: task hung in addrconf_dad_work run #9: crashed: INFO: task hung in addrconf_dad_work testing release v4.7 testing commit 523d939ef98fd712632d93a5a2b588e477a7565e with gcc (GCC) 5.5.0 run #0: crashed: INFO: task hung in linkwatch_event run #1: crashed: INFO: task hung in addrconf_dad_work run #2: crashed: INFO: task hung in addrconf_dad_work run #3: crashed: INFO: task hung in addrconf_dad_work run #4: crashed: INFO: task hung in addrconf_dad_work run #5: crashed: INFO: task hung in addrconf_dad_work run #6: crashed: INFO: task hung in linkwatch_event run #7: crashed: INFO: task hung in addrconf_dad_work run #8: crashed: INFO: task hung in addrconf_dad_work run #9: crashed: INFO: task hung in addrconf_dad_work testing release v4.6 testing commit 2dcd0af568b0cf583645c8a317dd12e344b1c72a with gcc (GCC) 5.5.0 run #0: crashed: INFO: task hung in addrconf_dad_work run #1: crashed: INFO: task hung in switchdev_deferred_process_work run #2: crashed: INFO: task hung in addrconf_verify_work run #3: crashed: INFO: task hung in addrconf_verify_work run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK testing release v4.5 testing commit b562e44f507e863c6792946e4e1b1449fbbac85d with gcc (GCC) 5.5.0 run #0: crashed: INFO: task hung in addrconf_dad_work run #1: crashed: INFO: task hung in addrconf_verify_work run #2: crashed: INFO: task hung in addrconf_verify_work run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK testing release v4.4 testing commit afd2ff9b7e1b367172f18ba7f693dfb62bdcb2dc with gcc (GCC) 5.5.0 run #0: crashed: INFO: task hung in addrconf_verify_work run #1: OK run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK testing release v4.3 testing commit 6a13feb9c82803e2b815eca72fa7a9f5561d7861 with gcc (GCC) 5.5.0 run #0: crashed: INFO: task hung in addrconf_verify_work run #1: OK run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK testing release v4.2 testing commit 64291f7db5bd8150a74ad2036f1037e6a0428df2 with gcc (GCC) 5.5.0 run #0: crashed: INFO: task hung in addrconf_dad_work run #1: crashed: INFO: task hung in addrconf_dad_work run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK testing release v4.1 testing commit b953c0d234bc72e8489d3bf51a276c5c4ec85345 with gcc (GCC) 5.5.0 run #0: crashed: INFO: task hung in rtnetlink_rcv run #1: crashed: INFO: task hung in reg_check_chans_work run #2: crashed: INFO: task hung in linkwatch_event run #3: crashed: INFO: task hung in addrconf_dad_work run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK revisions tested: 21, total time: 3h43m4.528541669s (build: 1h7m4.436819593s, test: 2h32m1.082295299s) the crash already happened on the oldest tested release crash: INFO: task hung in addrconf_dad_work bridge0: port 1(bridge_slave_0) entered forwarding state bridge0: port 2(bridge_slave_1) entered forwarding state bridge0: port 1(bridge_slave_0) entered forwarding state bridge0: port 2(bridge_slave_1) entered forwarding state bridge0: port 2(bridge_slave_1) entered forwarding state INFO: task kworker/0:2:2176 blocked for more than 140 seconds. Not tainted 4.1.0 #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kworker/0:2 D ffff88003cc4fc28 12984 2176 2 0x00000000 Workqueue: ipv6_addrconf addrconf_dad_work ffff88003cc4fc28 000000003cc4fc28 ffff88003de3c750 ffff880000000000 ffff88003cc50000 ffffffff83397748 ffff880039485e00 ffff88003de3c750 0000000000000286 ffff88003cc4fc48 ffffffff8264ba72 ffffffff83397740 Call Trace: [] schedule+0x32/0x80 kernel/sched/core.c:2826 [] schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:2858 [] __mutex_lock_common kernel/locking/mutex.c:578 [inline] [] mutex_lock_nested+0x195/0x610 kernel/locking/mutex.c:617 [] rtnl_lock+0x12/0x20 net/core/rtnetlink.c:70 [] addrconf_dad_work+0x28/0x330 net/ipv6/addrconf.c:3501 [] process_one_work+0x214/0x8d0 kernel/workqueue.c:2025 [] worker_thread+0x4b/0x470 kernel/workqueue.c:2157 [] kthread+0xea/0x100 drivers/block/aoe/aoecmd.c:1312 [] ret_from_fork+0x42/0x70 arch/x86/kernel/entry_64.S:639 3 locks held by kworker/0:2/2176: #0: ("%s"("ipv6_addrconf")){.+.+..}, at: [] set_work_data kernel/workqueue.c:606 [inline] #0: ("%s"("ipv6_addrconf")){.+.+..}, at: [] set_work_pool_and_clear_pending kernel/workqueue.c:634 [inline] #0: ("%s"("ipv6_addrconf")){.+.+..}, at: [] process_one_work+0x177/0x8d0 kernel/workqueue.c:2018 #1: ((&(&ifa->dad_work)->work)){+.+...}, at: [] set_work_data kernel/workqueue.c:606 [inline] #1: ((&(&ifa->dad_work)->work)){+.+...}, at: [] set_work_pool_and_clear_pending kernel/workqueue.c:634 [inline] #1: ((&(&ifa->dad_work)->work)){+.+...}, at: [] process_one_work+0x177/0x8d0 kernel/workqueue.c:2018 #2: (rtnl_mutex){+.+.+.}, at: [] rtnl_lock+0x12/0x20 net/core/rtnetlink.c:70 sending NMI to all CPUs: NMI backtrace for cpu 0 CPU: 0 PID: 6256 Comm: syz-executor.2 Not tainted 4.1.0 #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014 task: ffff880078e68450 ti: ffff88007faf4000 task.ti: ffff88007faf4000 RIP: 0010:[] [] lookup_chain_cache kernel/locking/lockdep.c:2036 [inline] RIP: 0010:[] [] validate_chain kernel/locking/lockdep.c:2115 [inline] RIP: 0010:[] [] __lock_acquire+0xc43/0x1c70 kernel/locking/lockdep.c:3205 RSP: 0018:ffff88007faf7748 EFLAGS: 00000002 RAX: 0000000000000000 RBX: 0000000000156000 RCX: 0000000000000000 RDX: ffffffff8361ea48 RSI: 0000000000000001 RDI: ffff880078e68450 RBP: ffff88007faf7808 R08: ffff880078e68cb8 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff83d465f0 R13: ffffffff83622408 R14: ffff880078e68cb8 R15: ffff880078e68450 FS: 00007f93c57b0700(0000) GS:ffff88003ec00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007fffa5b2aff0 CR3: 00000000780d0000 CR4: 00000000003407f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Stack: ffff88007faf77c8 ffff880078e68450 ffffffff83628f28 ffffffff83d56600 ffff880078e68ce0 ffffffff00000000 000000000007fff0 ffffffff83d465f0 ffff88007faf7808 0000000000000092 ffffffff83622408 ffffffff843af4c8 Call Trace: [] lock_acquire+0xe6/0x310 kernel/locking/lockdep.c:3623 [] __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:112 [inline] [] _raw_spin_lock_irqsave+0x62/0x90 kernel/locking/spinlock.c:159 [] __debug_check_no_obj_freed lib/debugobjects.c:688 [inline] [] debug_check_no_obj_freed+0x92/0x250 lib/debugobjects.c:726 [] kfree+0xd0/0x4b0 mm/slab.c:3584 [] skb_free_head+0x19/0x60 net/core/skbuff.c:617 [] pskb_expand_head+0xd8/0x260 net/core/skbuff.c:1214 [] netlink_trim+0x91/0xd0 net/netlink/af_netlink.c:1745 [] netlink_unicast+0x39/0x2e0 net/netlink/af_netlink.c:1779 [] rtnetlink_send+0x4d/0x80 net/core/rtnetlink.c:629 [] tcf_add_notify net/sched/act_api.c:920 [inline] [] tcf_action_add net/sched/act_api.c:941 [inline] [] tc_ctl_action+0x176/0x240 net/sched/act_api.c:978 [] rtnetlink_rcv_msg+0x83/0x230 net/core/rtnetlink.c:3250 [] netlink_rcv_skb+0xa9/0xd0 net/netlink/af_netlink.c:2843 [] rtnetlink_rcv+0x29/0x40 net/core/rtnetlink.c:3256 [] netlink_unicast_kernel net/netlink/af_netlink.c:1763 [inline] [] netlink_unicast+0x1ca/0x2e0 net/netlink/af_netlink.c:1789 [] netlink_sendmsg+0x310/0x3d0 net/netlink/af_netlink.c:2353 [] sock_sendmsg_nosec net/socket.c:613 [inline] [] sock_sendmsg+0x35/0x40 net/socket.c:623 [] ___sys_sendmsg+0x2c3/0x2d0 net/socket.c:1955 [] __sys_sendmsg+0x3d/0x80 net/socket.c:1989 [] SYSC_sendmsg net/socket.c:2000 [inline] [] SyS_sendmsg+0xd/0x20 net/socket.c:1996 [] system_call_fastpath+0x16/0x7a Code: ff ff 44 89 a5 68 ff ff ff 4c 8b a5 78 ff ff ff 4d 89 c6 4c 89 ad 60 ff ff ff 48 8b 40 08 48 89 45 90 4c 8b 6d 90 e8 6d cc 01 00 <49> 8d 45 f8 4d 39 ec 0f 84 3f 02 00 00 49 3b 5d 10 75 dc e9 11 NMI backtrace for cpu 1 CPU: 1 PID: 871 Comm: khungtaskd Not tainted 4.1.0 #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014 task: ffff88003cc76190 ti: ffff88003ccec000 task.ti: ffff88003ccec000 RIP: 0010:[] [] native_write_msr_safe+0xa/0x10 arch/x86/include/asm/msr.h:95 RSP: 0018:ffff88003ccefd08 EFLAGS: 00000082 RAX: 0000000000000400 RBX: 0000000000000001 RCX: 0000000000000830 RDX: 0000000000000001 RSI: 0000000000000400 RDI: 0000000000000830 RBP: ffff88003ccefd08 R08: 0000000000000000 R09: 0000000000000003 R10: ffff88003cc76190 R11: 0000000000000001 R12: ffffffff8341ab08 R13: 0000000000080000 R14: 0000000000000001 R15: 000000000000a120 FS: 0000000000000000(0000) GS:ffff88007fc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000000025cc914 CR3: 000000003aad1000 CR4: 00000000003407e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Stack: ffff88003ccefd68 ffffffff810c517f ffff88003ccefd78 0000000000000296 000000020000000a 0000000000000002 ffff88003ccefd88 0000000000000040 000000000000d3c0 0000000000000001 ffff88003de3c750 000000000000008c Call Trace: [] paravirt_write_msr arch/x86/include/asm/paravirt.h:133 [inline] [] native_x2apic_icr_write arch/x86/include/asm/apic.h:168 [inline] [] __x2apic_send_IPI_dest arch/x86/include/asm/x2apic.h:26 [inline] [] __x2apic_send_IPI_mask+0x10f/0x1a0 arch/x86/kernel/apic/x2apic_phys.c:52 [] x2apic_send_IPI_mask+0xe/0x10 arch/x86/kernel/apic/x2apic_cluster.c:79 [] arch_trigger_all_cpu_backtrace+0x33d/0x350 arch/x86/kernel/apic/hw_nmi.c:89 [] trigger_all_cpu_backtrace include/linux/nmi.h:43 [inline] [] check_hung_task kernel/hung_task.c:125 [inline] [] check_hung_uninterruptible_tasks kernel/hung_task.c:182 [inline] [] watchdog+0x47e/0x6c0 kernel/hung_task.c:238 [] kthread+0xea/0x100 drivers/block/aoe/aoecmd.c:1312 [] ret_from_fork+0x42/0x70 arch/x86/kernel/entry_64.S:639 Code: 00 55 89 f9 48 89 e5 0f 32 45 31 c0 48 89 d7 44 89 06 89 c6 5d 48 c1 e7 20 48 89 f8 48 09 f0 c3 90 55 89 f0 89 f9 48 89 e5 0f 30 <31> c0 5d c3 66 90 55 89 f9 48 89 e5 0f 33 48 89 d7 89 c1 5d 48