bisecting cause commit starting from 46cf053efec6a3a5f343fead837777efe8252a46 building syzkaller on be5c2c81971442d623dd1b265dabf4644ceeb35b testing commit 46cf053efec6a3a5f343fead837777efe8252a46 with gcc (GCC) 8.1.0 kernel signature: d879af3139245f80c5ce309aa359ca864809327e all runs: crashed: INFO: task hung in htable_put testing release v5.4 testing commit 219d54332a09e8d8741c1e1982f5eae56099de85 with gcc (GCC) 8.1.0 kernel signature: f222aba8ecab9600b9e1ceb6cda43eb2579f4344 run #0: basic kernel testing failed: failed to copy test binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/jobs/linux/workdir/image/key" "/tmp/syz-executor415541708" "root@10.128.0.222:./syz-executor415541708"]: exit status 1 ssh: connect to host 10.128.0.222 port 22: Connection timed out lost connection run #1: crashed: INFO: task hung in htable_put run #2: crashed: INFO: task hung in htable_put run #3: crashed: INFO: task hung in htable_put run #4: crashed: INFO: task hung in htable_put run #5: crashed: INFO: task hung in htable_put run #6: crashed: INFO: task hung in htable_put run #7: crashed: INFO: task hung in htable_put run #8: crashed: INFO: task hung in htable_put run #9: crashed: INFO: task hung in htable_put testing release v5.3 testing commit 4d856f72c10ecb060868ed10ff1b1453943fc6c8 with gcc (GCC) 8.1.0 kernel signature: 4fc194f2b6daccc3581f1883a1f190f314f93473 run #0: crashed: INFO: task hung in htable_put run #1: crashed: INFO: task hung in htable_put run #2: crashed: INFO: task hung in htable_put run #3: crashed: INFO: task hung in htable_put run #4: crashed: INFO: task hung in htable_put run #5: basic kernel testing failed: failed to copy test binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/jobs/linux/workdir/image/key" "/tmp/syz-executor335098590" "root@10.128.10.37:./syz-executor335098590"]: exit status 1 Connection timed out during banner exchange lost connection run #6: crashed: INFO: task hung in htable_put run #7: crashed: INFO: task hung in htable_put run #8: crashed: INFO: task hung in htable_put run #9: crashed: INFO: task hung in htable_put testing release v5.2 testing commit 0ecfebd2b52404ae0c54a878c872bb93363ada36 with gcc (GCC) 8.1.0 kernel signature: 503dc565f1606333bf1b5a49551c5b4b67df4d6f run #0: basic kernel testing failed: failed to copy test binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/jobs/linux/workdir/image/key" "/tmp/syz-executor499843008" "root@10.128.10.36:./syz-executor499843008"]: exit status 1 Connection timed out during banner exchange lost connection run #1: crashed: INFO: task hung in htable_put run #2: crashed: INFO: task hung in htable_put run #3: crashed: INFO: task hung in htable_put run #4: crashed: INFO: task hung in htable_put run #5: crashed: INFO: task hung in htable_put run #6: crashed: INFO: task hung in htable_put run #7: crashed: INFO: task hung in drain_all_pages run #8: crashed: INFO: task hung in htable_put run #9: crashed: INFO: task hung in htable_put testing release v5.1 testing commit e93c9c99a629c61837d5a7fc2120cd2b6c70dbdd with gcc (GCC) 8.1.0 kernel signature: 80aaef235d715e0dbfce51a69e2b21c6a2785280 run #0: crashed: INFO: task hung in htable_put run #1: crashed: INFO: task hung in htable_put run #2: crashed: INFO: task hung in drain_all_pages run #3: crashed: INFO: task hung in htable_put run #4: crashed: INFO: task hung in htable_put run #5: crashed: INFO: task hung in htable_put run #6: crashed: INFO: task hung in htable_put run #7: crashed: INFO: task hung in htable_put run #8: crashed: INFO: task hung in synchronize_rcu run #9: crashed: INFO: task hung in htable_put testing release v5.0 testing commit 1c163f4c7b3f621efff9b28a47abb36f7378d783 with gcc (GCC) 8.1.0 kernel signature: efb4c0f4c5a919fa7f1f1d4283be2770081a43b7 all runs: crashed: INFO: task hung in htable_put testing release v4.20 testing commit 8fe28cb58bcb235034b64cbbb7550a8a43fd88be with gcc (GCC) 8.1.0 kernel signature: f4fe92c91e568c3eab4601c7111054fa4c32d956 all runs: crashed: INFO: task hung in htable_put testing release v4.19 testing commit 84df9525b0c27f3ebc2ebb1864fa62a97fdedb7d with gcc (GCC) 8.1.0 kernel signature: a3d4ad7903ea80aea5b5d18d7009634ad207fa61 run #0: basic kernel testing failed: failed to copy test binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/jobs/linux/workdir/image/key" "/tmp/syz-executor005990851" "root@10.128.0.169:./syz-executor005990851"]: exit status 1 Connection timed out during banner exchange lost connection run #1: crashed: kernel panic: System is deadlocked on memory run #2: crashed: INFO: task hung in htable_put run #3: crashed: INFO: task hung in htable_put run #4: crashed: INFO: task hung in htable_put run #5: crashed: INFO: task hung in htable_put run #6: crashed: INFO: task hung in htable_put run #7: crashed: INFO: task hung in htable_put run #8: crashed: INFO: task hung in htable_put run #9: crashed: INFO: task hung in htable_put testing release v4.18 testing commit 94710cac0ef4ee177a63b5227664b38c95bbf703 with gcc (GCC) 8.1.0 kernel signature: da2d84173f6922d6a750bf3a120b125e0ecf6436 run #0: crashed: kernel panic: Out of memory and no killable processes... run #1: crashed: kernel panic: Out of memory and no killable processes... run #2: crashed: BUG: workqueue lockup run #3: crashed: BUG: workqueue lockup run #4: crashed: BUG: workqueue lockup run #5: crashed: BUG: workqueue lockup run #6: crashed: BUG: workqueue lockup run #7: crashed: BUG: workqueue lockup run #8: crashed: INFO: task hung in corrupted run #9: crashed: INFO: task hung in corrupted testing release v4.17 testing commit 29dcea88779c856c7dc92040a0c01233263101d4 with gcc (GCC) 8.1.0 kernel signature: 69da59c62a6696aaf28ce7cb48d8f6c9872f53fe run #0: crashed: BUG: workqueue lockup run #1: crashed: INFO: task hung in corrupted run #2: crashed: INFO: task hung in corrupted run #3: crashed: BUG: workqueue lockup run #4: crashed: BUG: workqueue lockup run #5: crashed: BUG: workqueue lockup run #6: crashed: INFO: task hung in corrupted run #7: crashed: BUG: workqueue lockup run #8: crashed: BUG: workqueue lockup run #9: crashed: BUG: workqueue lockup testing release v4.16 testing commit 0adb32858b0bddf4ada5f364a84ed60b196dbcda with gcc (GCC) 8.1.0 kernel signature: a7ae7bf14b8551961aa6bbcde6826b01ce18f13b run #0: basic kernel testing failed: failed to copy test binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/jobs/linux/workdir/image/key" "/tmp/syz-executor609450407" "root@10.128.0.232:./syz-executor609450407"]: exit status 1 Connection timed out during banner exchange lost connection run #1: basic kernel testing failed: failed to copy test binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/jobs/linux/workdir/image/key" "/tmp/syz-executor155639002" "root@10.128.0.125:./syz-executor155639002"]: exit status 1 ssh: connect to host 10.128.0.125 port 22: Connection timed out lost connection run #2: crashed: INFO: task hung in hashlimit_mt_check_common run #3: crashed: INFO: task hung in hashlimit_mt_check_common run #4: crashed: BUG: workqueue lockup run #5: crashed: BUG: workqueue lockup run #6: crashed: INFO: task hung in htable_put run #7: crashed: BUG: workqueue lockup run #8: crashed: BUG: workqueue lockup run #9: crashed: BUG: workqueue lockup testing release v4.15 testing commit d8a5b80568a9cb66810e75b182018e9edb68e8ff with gcc (GCC) 8.1.0 kernel signature: 4990621de95a89ebc2ff48ebcbc5a44091049dc3 run #0: basic kernel testing failed: failed to copy test binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/jobs/linux/workdir/image/key" "/tmp/syz-executor397738268" "root@10.128.0.141:./syz-executor397738268"]: exit status 1 ssh_exchange_identification: read: Connection reset by peer lost connection run #1: crashed: INFO: task hung in hashlimit_mt_check_common run #2: crashed: INFO: task hung in hashlimit_mt_check_common run #3: crashed: INFO: task hung in hashlimit_mt_check_common run #4: crashed: BUG: workqueue lockup run #5: crashed: BUG: workqueue lockup run #6: crashed: BUG: workqueue lockup run #7: crashed: INFO: task hung in hashlimit_mt_check_common run #8: crashed: BUG: workqueue lockup run #9: crashed: BUG: workqueue lockup testing release v4.14 testing commit bebc6082da0a9f5d47a1ea2edc099bf671058bd4 with gcc (GCC) 8.1.0 kernel signature: 33d65294fb6ddea43bedc74e3a8fd7a9d8b9371d run #0: crashed: INFO: task hung in hashlimit_mt_check_common run #1: crashed: BUG: workqueue lockup run #2: crashed: BUG: workqueue lockup run #3: crashed: BUG: workqueue lockup run #4: crashed: BUG: workqueue lockup run #5: crashed: BUG: workqueue lockup run #6: crashed: BUG: workqueue lockup run #7: crashed: BUG: workqueue lockup run #8: crashed: INFO: task hung in hashlimit_mt_check_common run #9: crashed: BUG: workqueue lockup testing release v4.13 testing commit 569dbb88e80deb68974ef6fdd6a13edb9d686261 with gcc (GCC) 8.1.0 kernel signature: db469d1ed4ef4eef56ab7d2469e52a5268334c47 all runs: OK # git bisect start bebc6082da0a9f5d47a1ea2edc099bf671058bd4 569dbb88e80deb68974ef6fdd6a13edb9d686261 Bisecting: 7300 revisions left to test after this (roughly 13 steps) [15d8ffc96464f6571ecf22043c45fad659f11bdd] Merge tag 'mmc-v4.14' of git://git.kernel.org/pub/scm/linux/kernel/git/ulfh/mmc testing commit 15d8ffc96464f6571ecf22043c45fad659f11bdd with gcc (GCC) 8.1.0 kernel signature: 82c15a6884c87dde93411653cc8660f0d3f44a2f run #0: basic kernel testing failed: failed to copy test binary to VM: failed to run ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/jobs/linux/workdir/image/key" "/syzkaller/jobs/linux/gopath/src/github.com/google/syzkaller/bin/linux_amd64/syz-executor" "root@10.128.10.45:./syz-executor"]: exit status 1 ssh: connect to host 10.128.10.45 port 22: Connection timed out lost connection run #1: basic kernel testing failed: failed to copy test binary to VM: timedout ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/jobs/linux/workdir/image/key" "/syzkaller/jobs/linux/gopath/src/github.com/google/syzkaller/bin/linux_amd64/syz-executor" "root@10.128.1.11:./syz-executor"] Warning: Permanently added '10.128.1.11' (ECDSA) to the list of known hosts. run #2: basic kernel testing failed: failed to copy test binary to VM: timedout ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/jobs/linux/workdir/image/key" "/syzkaller/jobs/linux/gopath/src/github.com/google/syzkaller/bin/linux_amd64/syz-executor" "root@10.128.0.59:./syz-executor"] Warning: Permanently added '10.128.0.59' (ECDSA) to the list of known hosts. run #3: basic kernel testing failed: failed to copy test binary to VM: timedout ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/jobs/linux/workdir/image/key" "/syzkaller/jobs/linux/gopath/src/github.com/google/syzkaller/bin/linux_amd64/syz-execprog" "root@10.128.10.43:./syz-execprog"] Warning: Permanently added '10.128.10.43' (ECDSA) to the list of known hosts. run #4: basic kernel testing failed: failed to copy test binary to VM: timedout ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/jobs/linux/workdir/image/key" "/syzkaller/jobs/linux/gopath/src/github.com/google/syzkaller/bin/linux_amd64/syz-executor" "root@10.128.0.56:./syz-executor"] Warning: Permanently added '10.128.0.56' (ECDSA) to the list of known hosts. run #5: basic kernel testing failed: failed to copy test binary to VM: timedout ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/jobs/linux/workdir/image/key" "/syzkaller/jobs/linux/gopath/src/github.com/google/syzkaller/bin/linux_amd64/syz-execprog" "root@10.128.0.252:./syz-execprog"] Warning: Permanently added '10.128.0.252' (ECDSA) to the list of known hosts. run #6: basic kernel testing failed: failed to copy test binary to VM: timedout ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/jobs/linux/workdir/image/key" "/syzkaller/jobs/linux/gopath/src/github.com/google/syzkaller/bin/linux_amd64/syz-executor" "root@10.128.10.14:./syz-executor"] run #7: basic kernel testing failed: failed to copy test binary to VM: timedout ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/jobs/linux/workdir/image/key" "/syzkaller/jobs/linux/gopath/src/github.com/google/syzkaller/bin/linux_amd64/syz-executor" "root@10.128.0.76:./syz-executor"] Warning: Permanently added '10.128.0.76' (ECDSA) to the list of known hosts. run #8: basic kernel testing failed: failed to copy test binary to VM: timedout ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/jobs/linux/workdir/image/key" "/syzkaller/jobs/linux/gopath/src/github.com/google/syzkaller/bin/linux_amd64/syz-executor" "root@10.128.0.144:./syz-executor"] run #9: basic kernel testing failed: failed to copy test binary to VM: timedout ["scp" "-P" "22" "-F" "/dev/null" "-o" "UserKnownHostsFile=/dev/null" "-o" "BatchMode=yes" "-o" "IdentitiesOnly=yes" "-o" "StrictHostKeyChecking=no" "-o" "ConnectTimeout=10" "-i" "/syzkaller/jobs/linux/workdir/image/key" "/syzkaller/jobs/linux/gopath/src/github.com/google/syzkaller/bin/linux_amd64/syz-execprog" "root@10.128.1.25:./syz-execprog"] Warning: Permanently added '10.128.1.25' (ECDSA) to the list of known hosts. # git bisect skip 15d8ffc96464f6571ecf22043c45fad659f11bdd Bisecting: 7300 revisions left to test after this (roughly 13 steps) [ace743214ea205c7d433562c5fa24e33bdfda7ab] net/mlx5e: Fix erroneous freeing of encap header buffer testing commit ace743214ea205c7d433562c5fa24e33bdfda7ab with gcc (GCC) 8.1.0 kernel signature: 9dba380907694cf5764bf0b840c3ba635d13e99a all runs: OK # git bisect good ace743214ea205c7d433562c5fa24e33bdfda7ab Bisecting: 976 revisions left to test after this (roughly 10 steps) [8d5f4b07174976c55a5f5d6967777373c6826944] stmmac: Don't access tx_q->dirty_tx before netif_tx_lock testing commit 8d5f4b07174976c55a5f5d6967777373c6826944 with gcc (GCC) 8.1.0 kernel signature: 06283439f3f3c18478db0e7449c8d7c1f07225d8 all runs: OK # git bisect good 8d5f4b07174976c55a5f5d6967777373c6826944 Bisecting: 488 revisions left to test after this (roughly 9 steps) [567825502730b6bc108f926bdb1cf2de7ae436ca] Merge tag 'pm-4.14-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm testing commit 567825502730b6bc108f926bdb1cf2de7ae436ca with gcc (GCC) 8.1.0 kernel signature: f67a5af57b1cf05761f24d14a9389e2c4c8e347d run #0: crashed: BUG: workqueue lockup run #1: crashed: BUG: workqueue lockup run #2: crashed: BUG: workqueue lockup run #3: crashed: BUG: workqueue lockup run #4: crashed: BUG: workqueue lockup run #5: crashed: BUG: workqueue lockup run #6: crashed: BUG: workqueue lockup run #7: crashed: BUG: workqueue lockup run #8: crashed: BUG: workqueue lockup run #9: crashed: INFO: task hung in corrupted # git bisect bad 567825502730b6bc108f926bdb1cf2de7ae436ca Bisecting: 243 revisions left to test after this (roughly 8 steps) [e7d6a32f0d1505695d4417caf93e4a64c06ffe9f] Merge tag 'usb-serial-4.14-rc6' of git://git.kernel.org/pub/scm/linux/kernel/git/johan/usb-serial into usb-linus testing commit e7d6a32f0d1505695d4417caf93e4a64c06ffe9f with gcc (GCC) 8.1.0 kernel signature: f1e1e7ffcaed48f79870f8cd687cbe37658977e7 run #0: crashed: BUG: workqueue lockup run #1: crashed: BUG: workqueue lockup run #2: crashed: BUG: workqueue lockup run #3: crashed: BUG: workqueue lockup run #4: crashed: BUG: workqueue lockup run #5: crashed: INFO: task hung in synchronize_rcu run #6: crashed: BUG: workqueue lockup run #7: crashed: BUG: workqueue lockup run #8: crashed: BUG: workqueue lockup run #9: crashed: BUG: workqueue lockup # git bisect bad e7d6a32f0d1505695d4417caf93e4a64c06ffe9f Bisecting: 130 revisions left to test after this (roughly 7 steps) [06d97c58f65cc747573a3fa8569cca0169e5096e] Merge branch 'akpm' (patches from Andrew) testing commit 06d97c58f65cc747573a3fa8569cca0169e5096e with gcc (GCC) 8.1.0 kernel signature: 5091a5205f6eca950081c268ebae383df4323aa1 run #0: crashed: BUG: workqueue lockup run #1: crashed: BUG: workqueue lockup run #2: crashed: BUG: workqueue lockup run #3: crashed: INFO: task hung in corrupted run #4: crashed: BUG: workqueue lockup run #5: crashed: INFO: task hung in hashlimit_mt_check_common run #6: crashed: BUG: workqueue lockup run #7: crashed: INFO: task hung in hashlimit_mt_check_common run #8: crashed: BUG: workqueue lockup run #9: crashed: INFO: task hung in corrupted # git bisect bad 06d97c58f65cc747573a3fa8569cca0169e5096e Bisecting: 57 revisions left to test after this (roughly 6 steps) [73a752cce2d4e9f704b097e63a79c68d71f7992d] Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 testing commit 73a752cce2d4e9f704b097e63a79c68d71f7992d with gcc (GCC) 8.1.0 kernel signature: 89ac2e208ccc882102cc7404f32d3d9e7065b9c4 all runs: OK # git bisect good 73a752cce2d4e9f704b097e63a79c68d71f7992d Bisecting: 29 revisions left to test after this (roughly 5 steps) [e18e884445aacb319a7b607a5813b02bcaeca834] Merge tag 'powerpc-4.14-5' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux testing commit e18e884445aacb319a7b607a5813b02bcaeca834 with gcc (GCC) 8.1.0 kernel signature: 23b6acce74a71a598b23eb628bda89b62d880eea all runs: OK # git bisect good e18e884445aacb319a7b607a5813b02bcaeca834 Bisecting: 14 revisions left to test after this (roughly 4 steps) [064f0e9302af4f4ab5e9dca03a5a77d6bebfd35e] mm: only display online cpus of the numa node testing commit 064f0e9302af4f4ab5e9dca03a5a77d6bebfd35e with gcc (GCC) 8.1.0 kernel signature: 2d260b155bcc167d7ff561438148e5e80bd2d69b all runs: OK # git bisect good 064f0e9302af4f4ab5e9dca03a5a77d6bebfd35e Bisecting: 6 revisions left to test after this (roughly 3 steps) [7e86600606cef21beec725039d70377fb364f881] fs/binfmt_misc.c: node could be NULL when evicting inode testing commit 7e86600606cef21beec725039d70377fb364f881 with gcc (GCC) 8.1.0 kernel signature: 4dd8a5c5536b1a7b50f05f10b57e54932dd218d7 run #0: crashed: INFO: task hung in hashlimit_mt_check_common run #1: crashed: BUG: workqueue lockup run #2: crashed: BUG: workqueue lockup run #3: crashed: INFO: task hung in corrupted run #4: crashed: BUG: workqueue lockup run #5: crashed: INFO: task hung in hashlimit_mt_check_common run #6: crashed: BUG: workqueue lockup run #7: crashed: BUG: workqueue lockup run #8: crashed: BUG: workqueue lockup run #9: crashed: BUG: workqueue lockup # git bisect bad 7e86600606cef21beec725039d70377fb364f881 Bisecting: 3 revisions left to test after this (roughly 2 steps) [b8c8a338f75e052d9fa2fed851259320af412e3f] Revert "vmalloc: back off when the current task is killed" testing commit b8c8a338f75e052d9fa2fed851259320af412e3f with gcc (GCC) 8.1.0 kernel signature: 43da7d7cac484c5c6975c81f5f355a103e693628 run #0: crashed: BUG: workqueue lockup run #1: crashed: BUG: workqueue lockup run #2: crashed: BUG: workqueue lockup run #3: crashed: INFO: task hung in htable_put run #4: crashed: BUG: workqueue lockup run #5: crashed: INFO: task hung in hashlimit_mt_check_common run #6: crashed: BUG: workqueue lockup run #7: crashed: BUG: workqueue lockup run #8: crashed: INFO: task hung in corrupted run #9: crashed: INFO: task hung in hashlimit_mt_check_common # git bisect bad b8c8a338f75e052d9fa2fed851259320af412e3f Bisecting: 1 revision left to test after this (roughly 1 step) [51962a9d437f0d580c04cd2c4abc2bd417200da2] scripts/kallsyms.c: ignore symbol type 'n' testing commit 51962a9d437f0d580c04cd2c4abc2bd417200da2 with gcc (GCC) 8.1.0 kernel signature: dcf588facf8f41c34c02f8159059a91693e28d18 all runs: OK # git bisect good 51962a9d437f0d580c04cd2c4abc2bd417200da2 Bisecting: 0 revisions left to test after this (roughly 0 steps) [ef4650144e76ae361fe4b8c9a0afcd53074cd520] mm/cma.c: take __GFP_NOWARN into account in cma_alloc() testing commit ef4650144e76ae361fe4b8c9a0afcd53074cd520 with gcc (GCC) 8.1.0 kernel signature: fae22c71c72fdfd9e642ebdc7d97457d20e5476e all runs: OK # git bisect good ef4650144e76ae361fe4b8c9a0afcd53074cd520 b8c8a338f75e052d9fa2fed851259320af412e3f is the first bad commit commit b8c8a338f75e052d9fa2fed851259320af412e3f Author: Johannes Weiner Date: Fri Oct 13 15:58:05 2017 -0700 Revert "vmalloc: back off when the current task is killed" This reverts commits 5d17a73a2ebe ("vmalloc: back off when the current task is killed") and 171012f56127 ("mm: don't warn when vmalloc() fails due to a fatal signal"). Commit 5d17a73a2ebe ("vmalloc: back off when the current task is killed") made all vmalloc allocations from a signal-killed task fail. We have seen crashes in the tty driver from this, where a killed task exiting tries to switch back to N_TTY, fails n_tty_open because of the vmalloc failing, and later crashes when dereferencing tty->disc_data. Arguably, relying on a vmalloc() call to succeed in order to properly exit a task is not the most robust way of doing things. There will be a follow-up patch to the tty code to fall back to the N_NULL ldisc. But the justification to make that vmalloc() call fail like this isn't convincing, either. The patch mentions an OOM victim exhausting the memory reserves and thus deadlocking the machine. But the OOM killer is only one, improbable source of fatal signals. It doesn't make sense to fail allocations preemptively with plenty of memory in most cases. The patch doesn't mention real-life instances where vmalloc sites would exhaust memory, which makes it sound more like a theoretical issue to begin with. But just in case, the OOM access to memory reserves has been restricted on the allocator side in cd04ae1e2dc8 ("mm, oom: do not rely on TIF_MEMDIE for memory reserves access"), which should take care of any theoretical concerns on that front. Revert this patch, and the follow-up that suppresses the allocation warnings when we fail the allocations due to a signal. Link: http://lkml.kernel.org/r/20171004185906.GB2136@cmpxchg.org Fixes: 171012f56127 ("mm: don't warn when vmalloc() fails due to a fatal signal") Signed-off-by: Johannes Weiner Acked-by: Vlastimil Babka Acked-by: Michal Hocko Cc: Alan Cox Cc: Christoph Hellwig Cc: Dmitry Vyukov Cc: Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds mm/vmalloc.c | 6 ------ 1 file changed, 6 deletions(-) culprit signature: 43da7d7cac484c5c6975c81f5f355a103e693628 parent signature: fae22c71c72fdfd9e642ebdc7d97457d20e5476e revisions tested: 27, total time: 6h0m58.53178405s (build: 2h28m7.188409308s, test: 3h30m13.479862886s) first bad commit: b8c8a338f75e052d9fa2fed851259320af412e3f Revert "vmalloc: back off when the current task is killed" cc: ["akpm@linux-foundation.org" "hannes@cmpxchg.org" "mhocko@suse.com" "torvalds@linux-foundation.org" "vbabka@suse.cz"] crash: INFO: task hung in hashlimit_mt_check_common IPv6: ADDRCONF(NETDEV_CHANGE): team0: link becomes ready IPv6: ADDRCONF(NETDEV_CHANGE): hsr_slave_1: link becomes ready 8021q: adding VLAN 0 to HW filter on device batadv0 8021q: adding VLAN 0 to HW filter on device batadv0 INFO: task syz-executor.0:7972 blocked for more than 140 seconds. Not tainted 4.14.0-rc4-syzkaller #0 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. syz-executor.0 D25080 7972 7893 0x00000004 Call Trace: context_switch kernel/sched/core.c:2790 [inline] __schedule+0x821/0x1fe0 kernel/sched/core.c:3366 schedule+0xf6/0x440 kernel/sched/core.c:3425 schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:3483 __mutex_lock_common kernel/locking/mutex.c:833 [inline] __mutex_lock+0xe0e/0x19c0 kernel/locking/mutex.c:893 mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908 hashlimit_mt_check_common.isra.10+0x296/0x12b0 net/netfilter/xt_hashlimit.c:897 hashlimit_mt_check_v2+0x48e/0x6f0 net/netfilter/xt_hashlimit.c:944 xt_check_match+0x200/0x5d0 net/netfilter/x_tables.c:465 check_match net/ipv6/netfilter/ip6_tables.c:498 [inline] find_check_match net/ipv6/netfilter/ip6_tables.c:515 [inline] find_check_entry.isra.8+0x3fc/0xe90 net/ipv6/netfilter/ip6_tables.c:566 translate_table+0xc5c/0x1bd0 net/ipv4/netfilter/ip_tables.c:731 do_replace net/ipv4/netfilter/ip_tables.c:1130 [inline] do_ipt_set_ctl+0x2d7/0x52d net/ipv4/netfilter/ip_tables.c:1664 nf_sockopt net/netfilter/nf_sockopt.c:105 [inline] nf_setsockopt+0x5c/0xb0 net/netfilter/nf_sockopt.c:114 ip_setsockopt+0x6b/0xa0 net/ipv4/ip_sockglue.c:1255 udp_setsockopt+0x16/0x30 net/ipv4/udp.c:2412 sock_common_setsockopt+0x73/0xf0 net/core/sock.c:2965 SYSC_setsockopt net/socket.c:1852 [inline] SyS_setsockopt+0x167/0x320 net/socket.c:1831 entry_SYSCALL_64_fastpath+0x23/0xc2 RIP: 0033:0x45a919 RSP: 002b:00007f82108dfc78 EFLAGS: 00000246 ORIG_RAX: 0000000000000036 RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 000000000045a919 RDX: 0000000000000040 RSI: 0000000000000000 RDI: 0000000000000005 RBP: 0000000000000082 R08: 0000000000000001 R09: 0000000000000000 R10: 0000000020000500 R11: 0000000000000246 R12: 0000000000000000 R13: 00007ffffb2f62ef R14: 00007f82108e09c0 R15: 000000000075bf2c Showing all locks held in the system: 2 locks held by khungtaskd/1036: #0: (rcu_read_lock){....}, at: [] check_hung_uninterruptible_tasks kernel/hung_task.c:175 [inline] #0: (rcu_read_lock){....}, at: [] watchdog+0x19d/0xba0 kernel/hung_task.c:249 #1: (tasklist_lock){.+.+}, at: [] debug_show_all_locks+0xde/0x340 kernel/locking/lockdep.c:4552 1 lock held by rsyslogd/7653: #0: (&f->f_pos_lock){+.+.}, at: [] __fdget_pos+0x14d/0x1b0 fs/file.c:767 2 locks held by getty/7775: #0: (&tty->ldisc_sem){++++}, at: [] ldsem_down_read+0x2d/0x40 drivers/tty/tty_ldsem.c:367 #1: (&ldata->atomic_read_lock){+.+.}, at: [] n_tty_read+0x2ec/0x1b00 drivers/tty/n_tty.c:2133 2 locks held by getty/7776: #0: (&tty->ldisc_sem){++++}, at: [] ldsem_down_read+0x2d/0x40 drivers/tty/tty_ldsem.c:367 #1: (&ldata->atomic_read_lock){+.+.}, at: [] n_tty_read+0x2ec/0x1b00 drivers/tty/n_tty.c:2133 2 locks held by getty/7777: #0: (&tty->ldisc_sem){++++}, at: [] ldsem_down_read+0x2d/0x40 drivers/tty/tty_ldsem.c:367 #1: (&ldata->atomic_read_lock){+.+.}, at: [] n_tty_read+0x2ec/0x1b00 drivers/tty/n_tty.c:2133 2 locks held by getty/7778: #0: (&tty->ldisc_sem){++++}, at: [] ldsem_down_read+0x2d/0x40 drivers/tty/tty_ldsem.c:367 #1: (&ldata->atomic_read_lock){+.+.}, at: [] n_tty_read+0x2ec/0x1b00 drivers/tty/n_tty.c:2133 2 locks held by getty/7779: #0: (&tty->ldisc_sem){++++}, at: [] ldsem_down_read+0x2d/0x40 drivers/tty/tty_ldsem.c:367 #1: (&ldata->atomic_read_lock){+.+.}, at: [] n_tty_read+0x2ec/0x1b00 drivers/tty/n_tty.c:2133 2 locks held by getty/7780: #0: (&tty->ldisc_sem){++++}, at: [] ldsem_down_read+0x2d/0x40 drivers/tty/tty_ldsem.c:367 #1: (&ldata->atomic_read_lock){+.+.}, at: [] n_tty_read+0x2ec/0x1b00 drivers/tty/n_tty.c:2133 2 locks held by getty/7781: #0: (&tty->ldisc_sem){++++}, at: [] ldsem_down_read+0x2d/0x40 drivers/tty/tty_ldsem.c:367 #1: (&ldata->atomic_read_lock){+.+.}, at: [] n_tty_read+0x2ec/0x1b00 drivers/tty/n_tty.c:2133 2 locks held by syz-executor.0/7972: #0: (sk_lock-AF_INET){+.+.}, at: [] lock_sock include/net/sock.h:1459 [inline] #0: (sk_lock-AF_INET){+.+.}, at: [] ip_setsockopt+0x56/0xa0 net/ipv4/ip_sockglue.c:1254 #1: (hashlimit_mutex){+.+.}, at: [] hashlimit_mt_check_common.isra.10+0x296/0x12b0 net/netfilter/xt_hashlimit.c:897 2 locks held by syz-executor.0/8015: #0: (sk_lock-AF_INET){+.+.}, at: [] lock_sock include/net/sock.h:1459 [inline] #0: (sk_lock-AF_INET){+.+.}, at: [] ip_setsockopt+0x56/0xa0 net/ipv4/ip_sockglue.c:1254 #1: (hashlimit_mutex){+.+.}, at: [] hashlimit_mt_check_common.isra.10+0x296/0x12b0 net/netfilter/xt_hashlimit.c:897 2 locks held by syz-executor.5/7981: #0: (sk_lock-AF_INET){+.+.}, at: [] lock_sock include/net/sock.h:1459 [inline] #0: (sk_lock-AF_INET){+.+.}, at: [] ip_setsockopt+0x56/0xa0 net/ipv4/ip_sockglue.c:1254 #1: (hashlimit_mutex){+.+.}, at: [] hashlimit_mt_check_common.isra.10+0x296/0x12b0 net/netfilter/xt_hashlimit.c:897 2 locks held by syz-executor.5/8019: #0: (sk_lock-AF_INET){+.+.}, at: [] lock_sock include/net/sock.h:1459 [inline] #0: (sk_lock-AF_INET){+.+.}, at: [] ip_setsockopt+0x56/0xa0 net/ipv4/ip_sockglue.c:1254 #1: (hashlimit_mutex){+.+.}, at: [] hashlimit_mt_check_common.isra.10+0x296/0x12b0 net/netfilter/xt_hashlimit.c:897 2 locks held by syz-executor.4/8023: #0: (sk_lock-AF_INET){+.+.}, at: [] lock_sock include/net/sock.h:1459 [inline] #0: (sk_lock-AF_INET){+.+.}, at: [] ip_setsockopt+0x56/0xa0 net/ipv4/ip_sockglue.c:1254 #1: (hashlimit_mutex){+.+.}, at: [] hashlimit_mt_check_common.isra.10+0x296/0x12b0 net/netfilter/xt_hashlimit.c:897 2 locks held by syz-executor.4/8032: #0: (sk_lock-AF_INET){+.+.}, at: [] lock_sock include/net/sock.h:1459 [inline] #0: (sk_lock-AF_INET){+.+.}, at: [] ip_setsockopt+0x56/0xa0 net/ipv4/ip_sockglue.c:1254 #1: (hashlimit_mutex){+.+.}, at: [] hashlimit_mt_check_common.isra.10+0x296/0x12b0 net/netfilter/xt_hashlimit.c:897 2 locks held by syz-executor.3/8028: #0: (sk_lock-AF_INET){+.+.}, at: [] lock_sock include/net/sock.h:1459 [inline] #0: (sk_lock-AF_INET){+.+.}, at: [] ip_setsockopt+0x56/0xa0 net/ipv4/ip_sockglue.c:1254 #1: (hashlimit_mutex){+.+.}, at: [] hashlimit_mt_check_common.isra.10+0x296/0x12b0 net/netfilter/xt_hashlimit.c:897 2 locks held by syz-executor.1/8031: #0: (sk_lock-AF_INET){+.+.}, at: [] lock_sock include/net/sock.h:1459 [inline] #0: (sk_lock-AF_INET){+.+.}, at: [] ip_setsockopt+0x56/0xa0 net/ipv4/ip_sockglue.c:1254 #1: (hashlimit_mutex){+.+.}, at: [] hashlimit_mt_check_common.isra.10+0x296/0x12b0 net/netfilter/xt_hashlimit.c:897 ============================================= NMI backtrace for cpu 1 CPU: 1 PID: 1036 Comm: khungtaskd Not tainted 4.14.0-rc4-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:16 [inline] dump_stack+0x145/0x1e1 lib/dump_stack.c:52 nmi_cpu_backtrace.cold.5+0x13/0xb2 lib/nmi_backtrace.c:102 nmi_trigger_cpumask_backtrace+0xf4/0x118 lib/nmi_backtrace.c:61 arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:37 trigger_all_cpu_backtrace include/linux/nmi.h:137 [inline] check_hung_task kernel/hung_task.c:132 [inline] check_hung_uninterruptible_tasks kernel/hung_task.c:190 [inline] watchdog+0x74d/0xba0 kernel/hung_task.c:249 kthread+0x376/0x440 kernel/kthread.c:231 ret_from_fork+0x2a/0x40 arch/x86/entry/entry_64.S:431 Sending NMI from CPU 1 to CPUs 0: NMI backtrace for cpu 0 CPU: 0 PID: 7963 Comm: syz-executor.2 Not tainted 4.14.0-rc4-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 task: ffff8800a892c400 task.stack: ffff8800a8930000 RIP: 0010:trace_hardirqs_off_caller+0x198/0x2b0 kernel/locking/lockdep.c:2971 RSP: 0018:ffff8800a8937740 EFLAGS: 00000807 RAX: dffffc0000000000 RBX: ffff8800a892c400 RCX: 0000000000000000 RDX: 1ffff10015125993 RSI: 0000000000000000 RDI: ffff8800a892cc9c RBP: ffff8800a8937758 R08: 1ffff10015126ecc R09: ffff8800a892c400 R10: e3abaf4c495a6d20 R11: 1ffff10015125b2b R12: 0000000024ebda55 R13: ffffffff813fcb7c R14: ffffffff8618f780 R15: 0000000000000000 FS: 00007f4fc330b700(0000) GS:ffff88012c000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f30de94a020 CR3: 0000000128efb000 CR4: 00000000001406f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: trace_hardirqs_off+0xd/0x10 kernel/locking/lockdep.c:2980 __local_bh_enable_ip+0x5c/0x170 kernel/softirq.c:163 __raw_spin_unlock_bh include/linux/spinlock_api_smp.h:176 [inline] _raw_spin_unlock_bh+0x30/0x40 kernel/locking/spinlock.c:207 spin_unlock_bh include/linux/spinlock.h:361 [inline] htable_selective_cleanup+0x231/0x3a0 net/netfilter/xt_hashlimit.c:380 htable_destroy net/netfilter/xt_hashlimit.c:415 [inline] htable_put+0x1c0/0x310 net/netfilter/xt_hashlimit.c:442 hashlimit_mt_destroy_v2+0x51/0x70 net/netfilter/xt_hashlimit.c:963 cleanup_match+0x13b/0x210 net/ipv6/netfilter/ip6_tables.c:487 find_check_entry.isra.8+0x4fd/0xe90 net/ipv6/netfilter/ip6_tables.c:589 translate_table+0xc5c/0x1bd0 net/ipv4/netfilter/ip_tables.c:731 do_replace net/ipv4/netfilter/ip_tables.c:1130 [inline] do_ipt_set_ctl+0x2d7/0x52d net/ipv4/netfilter/ip_tables.c:1664 nf_sockopt net/netfilter/nf_sockopt.c:105 [inline] nf_setsockopt+0x5c/0xb0 net/netfilter/nf_sockopt.c:114 ip_setsockopt+0x6b/0xa0 net/ipv4/ip_sockglue.c:1255 udp_setsockopt+0x16/0x30 net/ipv4/udp.c:2412 sock_common_setsockopt+0x73/0xf0 net/core/sock.c:2965 SYSC_setsockopt net/socket.c:1852 [inline] SyS_setsockopt+0x167/0x320 net/socket.c:1831 entry_SYSCALL_64_fastpath+0x23/0xc2 RIP: 0033:0x45a919 RSP: 002b:00007f4fc330ac78 EFLAGS: 00000246 ORIG_RAX: 0000000000000036 RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 000000000045a919 RDX: 0000000000000040 RSI: 0000000000000000 RDI: 0000000000000005 RBP: 0000000000000082 R08: 0000000000000001 R09: 0000000000000000 R10: 0000000020000500 R11: 0000000000000246 R12: 0000000000000000 R13: 00007ffd1479102f R14: 00007f4fc330b9c0 R15: 000000000075bfd4 Code: 00 00 8b 83 80 08 00 00 48 8d bb 9c 08 00 00 48 89 fa 48 c1 ea 03 44 8d 60 01 48 b8 00 00 00 00 00 fc ff df 44 89 a3 80 08 00 00 <0f> b6 14 02 48 89 f8 83 e0 07 83 c0 03 38 d0 7c 08 84 d2 0f 85