ci2 starts bisection 2023-05-13 07:06:13.222202365 +0000 UTC m=+103.126353302 bisecting fixing commit since 1b929c02afd37871d5afb9d498426f83432e71c2 building syzkaller on 9da18ae8fa827d046ef8da48cc23c97418553c23 ensuring issue is reproducible on original commit 1b929c02afd37871d5afb9d498426f83432e71c2 testing commit 1b929c02afd37871d5afb9d498426f83432e71c2 gcc compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 518a5ffbaaf1490dc225f221e8dc2f07b5e62303a07b8b620978ab7d4dc3395d all runs: crashed: KASAN: slab-out-of-bounds Read in xfs_btree_lookup_get_block testing current HEAD 9a48d604672220545d209e9996c2a1edbb5637f6 testing commit 9a48d604672220545d209e9996c2a1edbb5637f6 gcc compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 99401f2f98e2fa078c21eda3ad77075e505c249d1a4fe123a4438a975ae43c71 all runs: OK # git bisect start 9a48d604672220545d209e9996c2a1edbb5637f6 1b929c02afd37871d5afb9d498426f83432e71c2 Bisecting: 15951 revisions left to test after this (roughly 14 steps) [49be4fb28109b86a8ffe117415c306389a394cb2] Merge tag 'perf-tools-fixes-for-v6.3-1-2023-03-09' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux testing commit 49be4fb28109b86a8ffe117415c306389a394cb2 gcc compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 84ba4b04079b2a9b03a4db370764749eef2ae684ad6095ba458e2f95cda18987 all runs: crashed: KASAN: slab-out-of-bounds Read in xfs_btree_lookup_get_block # git bisect good 49be4fb28109b86a8ffe117415c306389a394cb2 Bisecting: 8363 revisions left to test after this (roughly 13 steps) [b68ee1c6131c540a62ecd443be89c406401df091] Merge tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi testing commit b68ee1c6131c540a62ecd443be89c406401df091 gcc compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 3b63d49e98c7a4036d1603dbbbbd684ef5055b513b2d05b28efdd994a1a1f784 all runs: crashed: KASAN: slab-out-of-bounds Read in xfs_btree_lookup_get_block # git bisect good b68ee1c6131c540a62ecd443be89c406401df091 Bisecting: 4121 revisions left to test after this (roughly 12 steps) [cec24b8b6bb841a19b5c5555b600a511a8988100] Merge tag 'char-misc-6.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc testing commit cec24b8b6bb841a19b5c5555b600a511a8988100 gcc compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 8d3ca634cae7433f1f346a14f60ddf404bd98ff1713f5da20a0784d6c8d3e1c6 all runs: crashed: KASAN: slab-out-of-bounds Read in xfs_btree_lookup_get_block # git bisect good cec24b8b6bb841a19b5c5555b600a511a8988100 Bisecting: 2042 revisions left to test after this (roughly 11 steps) [7acc1372113083fa281ba426021801e2402caca1] Merge tag 'cxl-for-6.4' of git://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl testing commit 7acc1372113083fa281ba426021801e2402caca1 gcc compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 23c230f4f02f0a3c752d97c51414bfa2bc38ef39df0f24e671abfe4fe5c68768 all runs: OK # git bisect bad 7acc1372113083fa281ba426021801e2402caca1 Bisecting: 1006 revisions left to test after this (roughly 10 steps) [70cc1b5307e8ee3076fdf2ecbeb89eb973aa0ff7] Merge tag 'powerpc-6.4-1' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux testing commit 70cc1b5307e8ee3076fdf2ecbeb89eb973aa0ff7 gcc compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 3b7abb71b7d8759ff1bcb454a9e88f782beeeabf837adce5722e39fa140ff13f all runs: crashed: KASAN: slab-out-of-bounds Read in xfs_btree_lookup_get_block # git bisect good 70cc1b5307e8ee3076fdf2ecbeb89eb973aa0ff7 Bisecting: 575 revisions left to test after this (roughly 9 steps) [af3877265dd88d7e333f94fb37bc09554544adca] Merge tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma testing commit af3877265dd88d7e333f94fb37bc09554544adca gcc compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: a99ce07c9b8c6373fc3d9e2842964172ec2d6a7fbacca70d27c5845c3f0a9bc2 all runs: OK # git bisect bad af3877265dd88d7e333f94fb37bc09554544adca Bisecting: 175 revisions left to test after this (roughly 8 steps) [56c455b38dba47ae9cb48d71b2a106d769d1a694] Merge tag 'xfs-6.4-merge-1' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux testing commit 56c455b38dba47ae9cb48d71b2a106d769d1a694 gcc compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: d5d5aaa394962a0daeb65f374c6fd5b1872fbc163728b1a3590e69ff457ebc04 all runs: OK # git bisect bad 56c455b38dba47ae9cb48d71b2a106d769d1a694 Bisecting: 127 revisions left to test after this (roughly 7 steps) [71deb8a5658c592ccad5ededb2ceffef6fcbba5f] xfs: Extend table marker on deprecated mount options table testing commit 71deb8a5658c592ccad5ededb2ceffef6fcbba5f gcc compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 7cb38392fa810712c9ed4c5bb5a8c1edec00a74c26431bacbc3eb94c179e8b9a all runs: OK # git bisect bad 71deb8a5658c592ccad5ededb2ceffef6fcbba5f Bisecting: 63 revisions left to test after this (roughly 6 steps) [e7cef2fe444b18e246d95d2a9156b37506590d61] Merge tag 'scrub-detect-refcount-gaps-6.4_2023-04-11' of git://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux into guilt/xfs-for-next testing commit e7cef2fe444b18e246d95d2a9156b37506590d61 gcc compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 433f78b78c166eab2d3df709cd199c1ef5b653304c1dc85349d0bf61dc4a2c15 all runs: OK # git bisect bad e7cef2fe444b18e246d95d2a9156b37506590d61 Bisecting: 31 revisions left to test after this (roughly 5 steps) [793f5c2cca10d0f33d38563c142ee00942c3e21e] Merge tag 'scrub-fix-legalese-6.4_2023-04-11' of git://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux into guilt/xfs-for-next testing commit 793f5c2cca10d0f33d38563c142ee00942c3e21e gcc compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: e92dfae994113ce9d064ea97f73c40c7d2a927ce4e53e9247c9c9a3e6e13882b all runs: OK # git bisect bad 793f5c2cca10d0f33d38563c142ee00942c3e21e Bisecting: 15 revisions left to test after this (roughly 4 steps) [774a99b47b588bf0bd9f65d3b241d5bba0b2fcb0] xfs: give xfs_bmap_intent its own perag reference testing commit 774a99b47b588bf0bd9f65d3b241d5bba0b2fcb0 gcc compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 1ceab9fa0c85fda24b21eda6365111bd8ba9962d89fe7939d5e153ff7b418d61 all runs: crashed: KASAN: slab-out-of-bounds Read in xfs_btree_lookup_get_block # git bisect good 774a99b47b588bf0bd9f65d3b241d5bba0b2fcb0 Bisecting: 8 revisions left to test after this (roughly 3 steps) [ecc73f8a58c7844b04186726f8699ba97cec2ef9] xfs: update copyright years for scrub/ files testing commit ecc73f8a58c7844b04186726f8699ba97cec2ef9 gcc compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 71f09a72ea4071a60f75153d7d0e7f84144618378fdd60c5b2a6ab61aaf47e3b all runs: crashed: KASAN: slab-out-of-bounds Read in xfs_btree_lookup_get_block # git bisect good ecc73f8a58c7844b04186726f8699ba97cec2ef9 Bisecting: 4 revisions left to test after this (roughly 2 steps) [22ed903eee23a5b174e240f1cdfa9acf393a5210] xfs: verify buffer contents when we skip log replay testing commit 22ed903eee23a5b174e240f1cdfa9acf393a5210 gcc compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: b2ecf13ee7a7a4a76781352ad5c22e6c3d8f11ef1cf1b612092b1819775a128e all runs: OK # git bisect bad 22ed903eee23a5b174e240f1cdfa9acf393a5210 Bisecting: 1 revision left to test after this (roughly 1 step) [4b827b3f305d1fcf837265f1e12acc22ee84327c] xfs: remove WARN when dquot cache insertion fails testing commit 4b827b3f305d1fcf837265f1e12acc22ee84327c gcc compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: ca8d55a757cf101ebd2d8c0389adcbc1d15a148ab239ba9e26816e8a9d1f4810 all runs: crashed: KASAN: slab-out-of-bounds Read in xfs_btree_lookup_get_block # git bisect good 4b827b3f305d1fcf837265f1e12acc22ee84327c Bisecting: 0 revisions left to test after this (roughly 0 steps) [c95356ca884885db702670e24933ee7f2b9f1754] xfs: _{attr,data}_map_shared should take ILOCK_EXCL until iread_extents is completely done testing commit c95356ca884885db702670e24933ee7f2b9f1754 gcc compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: f065722428431a320ca46bdcd1bee66c94c7611ef9215592f4ebb7603eb181e9 all runs: crashed: KASAN: slab-out-of-bounds Read in xfs_btree_lookup_get_block # git bisect good c95356ca884885db702670e24933ee7f2b9f1754 22ed903eee23a5b174e240f1cdfa9acf393a5210 is the first bad commit commit 22ed903eee23a5b174e240f1cdfa9acf393a5210 Author: Darrick J. Wong Date: Wed Apr 12 15:49:23 2023 +1000 xfs: verify buffer contents when we skip log replay syzbot detected a crash during log recovery: XFS (loop0): Mounting V5 Filesystem bfdc47fc-10d8-4eed-a562-11a831b3f791 XFS (loop0): Torn write (CRC failure) detected at log block 0x180. Truncating head block from 0x200. XFS (loop0): Starting recovery (logdev: internal) ================================================================== BUG: KASAN: slab-out-of-bounds in xfs_btree_lookup_get_block+0x15c/0x6d0 fs/xfs/libxfs/xfs_btree.c:1813 Read of size 8 at addr ffff88807e89f258 by task syz-executor132/5074 CPU: 0 PID: 5074 Comm: syz-executor132 Not tainted 6.2.0-rc1-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/26/2022 Call Trace: __dump_stack lib/dump_stack.c:88 [inline] dump_stack_lvl+0x1b1/0x290 lib/dump_stack.c:106 print_address_description+0x74/0x340 mm/kasan/report.c:306 print_report+0x107/0x1f0 mm/kasan/report.c:417 kasan_report+0xcd/0x100 mm/kasan/report.c:517 xfs_btree_lookup_get_block+0x15c/0x6d0 fs/xfs/libxfs/xfs_btree.c:1813 xfs_btree_lookup+0x346/0x12c0 fs/xfs/libxfs/xfs_btree.c:1913 xfs_btree_simple_query_range+0xde/0x6a0 fs/xfs/libxfs/xfs_btree.c:4713 xfs_btree_query_range+0x2db/0x380 fs/xfs/libxfs/xfs_btree.c:4953 xfs_refcount_recover_cow_leftovers+0x2d1/0xa60 fs/xfs/libxfs/xfs_refcount.c:1946 xfs_reflink_recover_cow+0xab/0x1b0 fs/xfs/xfs_reflink.c:930 xlog_recover_finish+0x824/0x920 fs/xfs/xfs_log_recover.c:3493 xfs_log_mount_finish+0x1ec/0x3d0 fs/xfs/xfs_log.c:829 xfs_mountfs+0x146a/0x1ef0 fs/xfs/xfs_mount.c:933 xfs_fs_fill_super+0xf95/0x11f0 fs/xfs/xfs_super.c:1666 get_tree_bdev+0x400/0x620 fs/super.c:1282 vfs_get_tree+0x88/0x270 fs/super.c:1489 do_new_mount+0x289/0xad0 fs/namespace.c:3145 do_mount fs/namespace.c:3488 [inline] __do_sys_mount fs/namespace.c:3697 [inline] __se_sys_mount+0x2d3/0x3c0 fs/namespace.c:3674 do_syscall_x64 arch/x86/entry/common.c:50 [inline] do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80 entry_SYSCALL_64_after_hwframe+0x63/0xcd RIP: 0033:0x7f89fa3f4aca Code: 83 c4 08 5b 5d c3 66 2e 0f 1f 84 00 00 00 00 00 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 c0 ff ff ff f7 d8 64 89 01 48 RSP: 002b:00007fffd5fb5ef8 EFLAGS: 00000206 ORIG_RAX: 00000000000000a5 RAX: ffffffffffffffda RBX: 00646975756f6e2c RCX: 00007f89fa3f4aca RDX: 0000000020000100 RSI: 0000000020009640 RDI: 00007fffd5fb5f10 RBP: 00007fffd5fb5f10 R08: 00007fffd5fb5f50 R09: 000000000000970d R10: 0000000000200800 R11: 0000000000000206 R12: 0000000000000004 R13: 0000555556c6b2c0 R14: 0000000000200800 R15: 00007fffd5fb5f50 The fuzzed image contains an AGF with an obviously garbage agf_refcount_level value of 32, and a dirty log with a buffer log item for that AGF. The ondisk AGF has a higher LSN than the recovered log item. xlog_recover_buf_commit_pass2 reads the buffer, compares the LSNs, and decides to skip replay because the ondisk buffer appears to be newer. Unfortunately, the ondisk buffer is corrupt, but recovery just read the buffer with no buffer ops specified: error = xfs_buf_read(mp->m_ddev_targp, buf_f->blf_blkno, buf_f->blf_len, buf_flags, &bp, NULL); Skipping the buffer leaves its contents in memory unverified. This sets us up for a kernel crash because xfs_refcount_recover_cow_leftovers reads the buffer (which is still around in XBF_DONE state, so no read verification) and creates a refcountbt cursor of height 32. This is impossible so we run off the end of the cursor object and crash. Fix this by invoking the verifier on all skipped buffers and aborting log recovery if the ondisk buffer is corrupt. It might be smarter to force replay the log item atop the buffer and then see if it'll pass the write verifier (like ext4 does) but for now let's go with the conservative option where we stop immediately. Link: https://syzkaller.appspot.com/bug?extid=7e9494b8b399902e994e Signed-off-by: Darrick J. Wong Reviewed-by: Dave Chinner Signed-off-by: Dave Chinner fs/xfs/xfs_buf_item_recover.c | 10 ++++++++++ 1 file changed, 10 insertions(+) culprit signature: b2ecf13ee7a7a4a76781352ad5c22e6c3d8f11ef1cf1b612092b1819775a128e parent signature: f065722428431a320ca46bdcd1bee66c94c7611ef9215592f4ebb7603eb181e9 revisions tested: 17, total time: 10h22m14.671341071s (build: 8h32m34.587603099s, test: 1h43m1.045048365s) first good commit: 22ed903eee23a5b174e240f1cdfa9acf393a5210 xfs: verify buffer contents when we skip log replay recipients (to): ["david@fromorbit.com" "dchinner@redhat.com" "djwong@kernel.org"] recipients (cc): []