ci starts bisection 2023-05-05 10:45:43.463581018 +0000 UTC m=+37303.810786138 bisecting cause commit starting from 3c4aa44343777844e425c28f1427127f3e55826f building syzkaller on 518a39a63148f6aee9c82e5b6b1c20889a21f698 ensuring issue is reproducible on original commit 3c4aa44343777844e425c28f1427127f3e55826f testing commit 3c4aa44343777844e425c28f1427127f3e55826f gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: ddd4ee3e76c54430cc81b3aff3a38b1580eb390d14edc83f619c16c875a54302 run #0: crashed: WARNING in shmem_evict_inode run #1: crashed: WARNING in shmem_evict_inode run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK reproducer seems to be flaky testing release v6.3 testing commit 457391b0380335d5e9a5babdec90ac53928b23b4 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 0a59ac0e6ebbe2ef9e9c0e9ed72f8a5e7e536b6ea9426fe23a7641faf7fb2fff all runs: OK # git bisect start 3c4aa44343777844e425c28f1427127f3e55826f 457391b0380335d5e9a5babdec90ac53928b23b4 Bisecting: 7448 revisions left to test after this (roughly 13 steps) [b68ee1c6131c540a62ecd443be89c406401df091] Merge tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi testing commit b68ee1c6131c540a62ecd443be89c406401df091 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 3f5cf1fd57ab2250c0dcf4ce23e515b79743d24c68c7a01d1c1bec3b5e9ceef2 all runs: OK # git bisect good b68ee1c6131c540a62ecd443be89c406401df091 Bisecting: 3720 revisions left to test after this (roughly 12 steps) [b39667abcdcc754e32a0eb0df9cf49d45333d4ae] Merge tag 'tty-6.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty testing commit b39667abcdcc754e32a0eb0df9cf49d45333d4ae gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 0613672fb6ef216a63f633780d3aae9f7a0ae27afce04384ca83dc63f45b4ee2 all runs: OK # git bisect good b39667abcdcc754e32a0eb0df9cf49d45333d4ae Bisecting: 1848 revisions left to test after this (roughly 11 steps) [1e098dec61ba342c8cebbfdf0fcdcd9ce54f7fa1] Merge tag 'ntfs3_for_6.4' of https://github.com/Paragon-Software-Group/linux-ntfs3 testing commit 1e098dec61ba342c8cebbfdf0fcdcd9ce54f7fa1 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 7f5c24ebe9d88e204c523c9030d8c6b7c5472923444039e8fb56adf4751f1bfe run #0: crashed: WARNING in shmem_evict_inode run #1: crashed: WARNING in shmem_evict_inode run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK # git bisect bad 1e098dec61ba342c8cebbfdf0fcdcd9ce54f7fa1 Bisecting: 1042 revisions left to test after this (roughly 10 steps) [91ec4b0d11fe115581ce2835300558802ce55e6c] Merge tag 'mips_6.4' of git://git.kernel.org/pub/scm/linux/kernel/git/mips/linux testing commit 91ec4b0d11fe115581ce2835300558802ce55e6c gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: e250519d812775b3784c9414750deac5d099ed014ebcf9b64ea2f0dba0b421cb all runs: OK # git bisect good 91ec4b0d11fe115581ce2835300558802ce55e6c Bisecting: 524 revisions left to test after this (roughly 9 steps) [7c339778f908875772c17f2e04ed731aac772881] Merge tag 'perf-core-2023-04-27' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip testing commit 7c339778f908875772c17f2e04ed731aac772881 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 5d10502f5ebb405846b2993d001d1ae6bc5d7f39a0e9243b29af34c120a77f2e run #0: crashed: WARNING in shmem_evict_inode run #1: OK run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK run #10: OK run #11: crashed: WARNING in shmem_evict_inode run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK # git bisect bad 7c339778f908875772c17f2e04ed731aac772881 Bisecting: 258 revisions left to test after this (roughly 8 steps) [0169fd518a8934d8d723659752b07589ecc9f692] userfaultfd: convert mfill_atomic_hugetlb() to use a folio testing commit 0169fd518a8934d8d723659752b07589ecc9f692 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: df52e70cb5869dda82dc99c5992e6ba6d1ae3dfd5ca6841160c0670ff42d313e all runs: crashed: INFO: rcu detected stall in corrupted # git bisect bad 0169fd518a8934d8d723659752b07589ecc9f692 Bisecting: 129 revisions left to test after this (roughly 7 steps) [53d36a56d8c494554e816300ebc0f7c23274b3ae] mm: prefer fault_around_pages to fault_around_bytes testing commit 53d36a56d8c494554e816300ebc0f7c23274b3ae gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: cace20ebd2f604828146684698943e3954520b7fc4e1e4541360fae87cd1bb4f all runs: OK # git bisect good 53d36a56d8c494554e816300ebc0f7c23274b3ae Bisecting: 64 revisions left to test after this (roughly 6 steps) [ef6a22b70f6d90449a5c797b8968a682824e2011] sched/numa: apply the scan delay to every new vma testing commit ef6a22b70f6d90449a5c797b8968a682824e2011 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: e06689efdeead5b0db03635151133c5eea2ffa6f54a4165264e7778b25fce178 all runs: OK # git bisect good ef6a22b70f6d90449a5c797b8968a682824e2011 Bisecting: 32 revisions left to test after this (roughly 5 steps) [27d9a0fdb53f05c93ed9c674b870c8add451697e] kmemleak-test: fix kmemleak_test.c build logic testing commit 27d9a0fdb53f05c93ed9c674b870c8add451697e gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: a0765754f4fa1fedd240774972239945bb3e949bdf7638cb34140e6a5d7a0253 all runs: OK # git bisect good 27d9a0fdb53f05c93ed9c674b870c8add451697e Bisecting: 16 revisions left to test after this (roughly 4 steps) [12904d953364e3bd21789a45137bf90df7cc78ee] mm/khugepaged: recover from poisoned file-backed memory testing commit 12904d953364e3bd21789a45137bf90df7cc78ee gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 85d6ce1d14c9baa5a5416a0abd2fe068148da7f15d546e3e46a45c2d3aed51dc run #0: OK run #1: crashed: WARNING in shmem_evict_inode run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK # git bisect bad 12904d953364e3bd21789a45137bf90df7cc78ee Bisecting: 7 revisions left to test after this (roughly 3 steps) [a2174e95cce5e65ee49c60368434aaae944ff1af] memcg: do not flush stats in irq context testing commit a2174e95cce5e65ee49c60368434aaae944ff1af gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: eaf378b36f8db70d16e892820708e8fd30abd269e3f22d078b2eae05473bd00b all runs: OK # git bisect good a2174e95cce5e65ee49c60368434aaae944ff1af Bisecting: 3 revisions left to test after this (roughly 2 steps) [0d856cfedd6bc0cb8e19c1e70d400e79b655cfdd] vmscan: memcg: sleep when flushing stats during reclaim testing commit 0d856cfedd6bc0cb8e19c1e70d400e79b655cfdd gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: d1b6d1c08e4235d08515143373279e32619935afd2d008025d8cdf287a139c18 all runs: OK # git bisect good 0d856cfedd6bc0cb8e19c1e70d400e79b655cfdd Bisecting: 1 revision left to test after this (roughly 1 step) [98c76c9f1ef7599b39bfd4bd99b8a760d4a8cd3b] mm/khugepaged: recover from poisoned anonymous memory testing commit 98c76c9f1ef7599b39bfd4bd99b8a760d4a8cd3b gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 1258ec6fd5e4736e83b1748110d0b8bb2e5f3cd5a887d5f9e20c7bc172b5839a all runs: OK # git bisect good 98c76c9f1ef7599b39bfd4bd99b8a760d4a8cd3b Bisecting: 0 revisions left to test after this (roughly 0 steps) [6efc7afb5cc98488410d44695685d003d832534d] mm/hwpoison: introduce copy_mc_highpage testing commit 6efc7afb5cc98488410d44695685d003d832534d gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: aa11139bfddd1f522429a14734ca39ed95939aeeed99fbea5274087c4a819bec all runs: OK # git bisect good 6efc7afb5cc98488410d44695685d003d832534d 12904d953364e3bd21789a45137bf90df7cc78ee is the first bad commit commit 12904d953364e3bd21789a45137bf90df7cc78ee Author: Jiaqi Yan Date: Wed Mar 29 08:11:21 2023 -0700 mm/khugepaged: recover from poisoned file-backed memory Make collapse_file roll back when copying pages failed. More concretely: - extract copying operations into a separate loop - postpone the updates for nr_none until both scanning and copying succeeded - postpone joining small xarray entries until both scanning and copying succeeded - postpone the update operations to NR_XXX_THPS until both scanning and copying succeeded - for non-SHMEM file, roll back filemap_nr_thps_inc if scan succeeded but copying failed Tested manually: 0. Enable khugepaged on system under test. Mount tmpfs at /mnt/ramdisk. 1. Start a two-thread application. Each thread allocates a chunk of non-huge memory buffer from /mnt/ramdisk. 2. Pick 4 random buffer address (2 in each thread) and inject uncorrectable memory errors at physical addresses. 3. Signal both threads to make their memory buffer collapsible, i.e. calling madvise(MADV_HUGEPAGE). 4. Wait and then check kernel log: khugepaged is able to recover from poisoned pages by skipping them. 5. Signal both threads to inspect their buffer contents and make sure no data corruption. Link: https://lkml.kernel.org/r/20230329151121.949896-4-jiaqiyan@google.com Signed-off-by: Jiaqi Yan Reviewed-by: Yang Shi Acked-by: Hugh Dickins Cc: David Stevens Cc: Kefeng Wang Cc: Kirill A. Shutemov Cc: "Kirill A. Shutemov" Cc: Miaohe Lin Cc: Naoya Horiguchi Cc: Oscar Salvador Cc: Tong Tiangen Cc: Tony Luck Signed-off-by: Andrew Morton mm/khugepaged.c | 87 +++++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 56 insertions(+), 31 deletions(-) culprit signature: 85d6ce1d14c9baa5a5416a0abd2fe068148da7f15d546e3e46a45c2d3aed51dc parent signature: aa11139bfddd1f522429a14734ca39ed95939aeeed99fbea5274087c4a819bec Reproducer flagged being flaky revisions tested: 16, total time: 6h4m41.590944671s (build: 3h38m42.547539091s, test: 2h23m18.590994608s) first bad commit: 12904d953364e3bd21789a45137bf90df7cc78ee mm/khugepaged: recover from poisoned file-backed memory recipients (to): ["akpm@linux-foundation.org" "hughd@google.com" "jiaqiyan@google.com" "shy828301@gmail.com"] recipients (cc): [] crash: WARNING in shmem_evict_inode ------------[ cut here ]------------ WARNING: CPU: 1 PID: 11993 at mm/shmem.c:1186 shmem_evict_inode+0x765/0xa30 mm/shmem.c:1175 Modules linked in: CPU: 1 PID: 11993 Comm: syz-executor.3 Not tainted 6.3.0-rc4-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 04/14/2023 RIP: 0010:shmem_evict_inode+0x765/0xa30 mm/shmem.c:1186 Code: b6 06 41 38 c5 7c 08 84 c0 0f 85 c7 00 00 00 41 8b 44 24 e0 85 c0 75 c2 48 8b 7c 24 20 4c 89 fe e8 70 51 b3 ff e9 5a fe ff ff <0f> 0b e9 9c f9 ff ff 48 b8 00 00 00 00 00 fc ff df 4c 89 fa 48 c1 RSP: 0018:ffffc900055efb68 EFLAGS: 00010282 RAX: dffffc0000000000 RBX: ffff888073dbccf8 RCX: ffffffff81d20c84 RDX: 1ffff1100e7b79d3 RSI: ffff888073dbcd08 RDI: ffff888073dbce98 RBP: ffffc900055efc60 R08: 0000000000000001 R09: ffffc900055efaff R10: fffff52000abdf5f R11: 0000000000000000 R12: ffff888073dbcdc8 R13: ffff88801627f800 R14: ffff888073dbcdf8 R15: ffff888073dbce18 FS: 0000000000000000(0000) GS:ffff8880b9b00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007ffea06d4fa8 CR3: 000000001faaf000 CR4: 00000000003506e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: evict+0x296/0x5d0 fs/inode.c:665 __dentry_kill+0x315/0x560 fs/dcache.c:607 __fput+0x2f6/0x9a0 fs/file_table.c:329 task_work_run+0x12f/0x220 kernel/task_work.c:179 exit_task_work include/linux/task_work.h:38 [inline] do_exit+0x92c/0x2480 kernel/exit.c:869 do_group_exit+0xb4/0x250 kernel/exit.c:1019 __do_sys_exit_group kernel/exit.c:1030 [inline] __se_sys_exit_group kernel/exit.c:1028 [inline] __x64_sys_exit_group+0x39/0x40 kernel/exit.c:1028 do_syscall_x64 arch/x86/entry/common.c:50 [inline] do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80 entry_SYSCALL_64_after_hwframe+0x63/0xcd RIP: 0033:0x7f6c9248c169 Code: Unable to access opcode bytes at 0x7f6c9248c13f. RSP: 002b:00007ffedacaa528 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7 RAX: ffffffffffffffda RBX: 000000000000001e RCX: 00007f6c9248c169 RDX: 00007f6c9243e01b RSI: ffffffffffffffb8 RDI: 0000000000000000 RBP: 0000000000000000 R08: 00007ffedace4080 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000001 R13: 0000000000000000 R14: 0000000000000001 R15: 00007ffedacaa610