bisecting cause commit starting from bf8d1cd4386535004c4afe7f03d37f9864c9940e building syzkaller on af6b8ef8f18c45343685f0ee7af9dd3a4b2b5d95 testing commit bf8d1cd4386535004c4afe7f03d37f9864c9940e with gcc (GCC) 8.1.0 kernel signature: 3caf7f345f951e18f726f09b895bd7f0a11bd457 run #0: crashed: INFO: task hung in __generic_file_fsync run #1: crashed: INFO: task hung in __generic_file_fsync run #2: crashed: INFO: task hung in __generic_file_fsync run #3: crashed: INFO: task hung in __generic_file_fsync run #4: crashed: INFO: task hung in __generic_file_fsync run #5: crashed: INFO: task hung in truncate_inode_pages_range run #6: crashed: INFO: task hung in __generic_file_fsync run #7: crashed: INFO: task hung in __generic_file_fsync run #8: crashed: INFO: task hung in __generic_file_fsync run #9: crashed: INFO: task hung in __generic_file_fsync testing release v5.4 testing commit 219d54332a09e8d8741c1e1982f5eae56099de85 with gcc (GCC) 8.1.0 kernel signature: ddd84cdf61efb64d5b07fd767f66d9cce201ffab run #0: crashed: INFO: task hung in __generic_file_fsync run #1: crashed: INFO: task hung in truncate_inode_pages_range run #2: crashed: INFO: task hung in __generic_file_fsync run #3: crashed: INFO: task hung in __generic_file_fsync run #4: crashed: INFO: task hung in truncate_inode_pages_range run #5: crashed: INFO: task hung in __generic_file_fsync run #6: crashed: INFO: task hung in __generic_file_fsync run #7: crashed: INFO: task hung in truncate_inode_pages_range run #8: crashed: INFO: task hung in process_measurement run #9: crashed: INFO: task hung in __generic_file_fsync testing release v5.3 testing commit 4d856f72c10ecb060868ed10ff1b1453943fc6c8 with gcc (GCC) 8.1.0 kernel signature: 5341c51122e91219665e578bd5da07e2eef80156 run #0: crashed: INFO: task hung in __generic_file_fsync run #1: crashed: INFO: task hung in __generic_file_fsync run #2: crashed: INFO: task hung in __generic_file_fsync run #3: crashed: INFO: task hung in __generic_file_fsync run #4: crashed: INFO: task hung in __generic_file_fsync run #5: crashed: INFO: task hung in __generic_file_fsync run #6: crashed: INFO: task hung in truncate_inode_pages_range run #7: crashed: INFO: task hung in __generic_file_fsync run #8: crashed: INFO: task hung in __generic_file_fsync run #9: crashed: INFO: task hung in __generic_file_fsync testing release v5.2 testing commit 0ecfebd2b52404ae0c54a878c872bb93363ada36 with gcc (GCC) 8.1.0 kernel signature: 5c69b5e6dffb8de105dc6152a96e78b02778213e run #0: crashed: INFO: task hung in __generic_file_fsync run #1: crashed: INFO: task hung in __generic_file_fsync run #2: crashed: INFO: task hung in truncate_inode_pages_range run #3: crashed: INFO: task hung in __generic_file_fsync run #4: crashed: INFO: task hung in __generic_file_fsync run #5: crashed: INFO: task hung in __generic_file_fsync run #6: crashed: INFO: task hung in __generic_file_fsync run #7: crashed: INFO: task hung in __generic_file_fsync run #8: crashed: INFO: task hung in __generic_file_fsync run #9: crashed: INFO: task hung in __generic_file_fsync testing release v5.1 testing commit e93c9c99a629c61837d5a7fc2120cd2b6c70dbdd with gcc (GCC) 8.1.0 kernel signature: af62bb26a89ce8ef8298d91df417820800fb5b73 run #0: crashed: INFO: task hung in __generic_file_fsync run #1: crashed: INFO: task hung in __generic_file_fsync run #2: crashed: INFO: task hung in process_measurement run #3: crashed: INFO: task hung in __generic_file_fsync run #4: crashed: INFO: task hung in __generic_file_fsync run #5: crashed: INFO: task hung in __generic_file_fsync run #6: crashed: INFO: task hung in __generic_file_fsync run #7: crashed: INFO: task hung in __generic_file_fsync run #8: crashed: INFO: task hung in __generic_file_fsync run #9: crashed: INFO: task hung in __generic_file_fsync testing release v5.0 testing commit 1c163f4c7b3f621efff9b28a47abb36f7378d783 with gcc (GCC) 8.1.0 kernel signature: 7e7fb9232683af8248b029908a37371cb7da1b4c all runs: OK # git bisect start e93c9c99a629c61837d5a7fc2120cd2b6c70dbdd 1c163f4c7b3f621efff9b28a47abb36f7378d783 Bisecting: 7074 revisions left to test after this (roughly 13 steps) [b5dd0c658c31b469ccff1b637e5124851e7a4a1c] Merge branch 'akpm' (patches from Andrew) testing commit b5dd0c658c31b469ccff1b637e5124851e7a4a1c with gcc (GCC) 8.1.0 kernel signature: fe32c68db42d6bf727f4a950b494a43adb2bc960 all runs: OK # git bisect good b5dd0c658c31b469ccff1b637e5124851e7a4a1c Bisecting: 3573 revisions left to test after this (roughly 12 steps) [4f0237062ca70c8e34e16e518aee4b84c30d1832] Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/dtor/input testing commit 4f0237062ca70c8e34e16e518aee4b84c30d1832 with gcc (GCC) 8.1.0 kernel signature: 50d137d09a802456083ff5804ed2aa1975990657 run #0: crashed: INFO: task hung in __generic_file_fsync run #1: crashed: INFO: task hung in __generic_file_fsync run #2: crashed: INFO: task hung in __generic_file_fsync run #3: crashed: INFO: task hung in __generic_file_fsync run #4: crashed: INFO: task hung in __generic_file_fsync run #5: crashed: INFO: task hung in process_measurement run #6: crashed: INFO: task hung in __generic_file_fsync run #7: crashed: INFO: task hung in __generic_file_fsync run #8: crashed: INFO: task hung in __generic_file_fsync run #9: crashed: INFO: task hung in __generic_file_fsync # git bisect bad 4f0237062ca70c8e34e16e518aee4b84c30d1832 Bisecting: 1707 revisions left to test after this (roughly 11 steps) [80201fe175cbf7f3e372f53eba0a881a702ad926] Merge tag 'for-5.1/block-20190302' of git://git.kernel.dk/linux-block testing commit 80201fe175cbf7f3e372f53eba0a881a702ad926 with gcc (GCC) 8.1.0 kernel signature: 0ae4c89a1fa1f6848911f1ed8c33b3389a24d0ce run #0: crashed: INFO: task hung in __generic_file_fsync run #1: crashed: INFO: task hung in __generic_file_fsync run #2: crashed: INFO: task hung in __generic_file_fsync run #3: crashed: INFO: task hung in truncate_inode_pages_range run #4: crashed: INFO: task hung in __generic_file_fsync run #5: crashed: INFO: task hung in __generic_file_fsync run #6: crashed: INFO: task hung in __generic_file_fsync run #7: crashed: INFO: task hung in __generic_file_fsync run #8: crashed: INFO: task hung in __generic_file_fsync run #9: crashed: INFO: task hung in __generic_file_fsync # git bisect bad 80201fe175cbf7f3e372f53eba0a881a702ad926 Bisecting: 888 revisions left to test after this (roughly 10 steps) [5ea3998d56346975c2701df18fb5b6e3ab5c8d9e] Merge tag 'drm-intel-next-2019-02-07' of git://anongit.freedesktop.org/drm/drm-intel into drm-next testing commit 5ea3998d56346975c2701df18fb5b6e3ab5c8d9e with gcc (GCC) 8.1.0 kernel signature: 950557c49ec449415354d7cf6fbbd73edc86fc2f all runs: OK # git bisect good 5ea3998d56346975c2701df18fb5b6e3ab5c8d9e Bisecting: 467 revisions left to test after this (roughly 9 steps) [1cabd3e0bd88d7ba9854cbb9213ef40eccad603b] Merge tag 'for-v5.1' of git://git.kernel.org/pub/scm/linux/kernel/git/sre/linux-power-supply testing commit 1cabd3e0bd88d7ba9854cbb9213ef40eccad603b with gcc (GCC) 8.1.0 kernel signature: ff868ebaa56ec8607fe0acae1996b28a03e7c1ea all runs: OK # git bisect good 1cabd3e0bd88d7ba9854cbb9213ef40eccad603b Bisecting: 246 revisions left to test after this (roughly 8 steps) [cf2e8c544cd3b33e9e403b7b72404c221bf888d1] Merge tag 'mfd-next-5.1' of git://git.kernel.org/pub/scm/linux/kernel/git/lee/mfd testing commit cf2e8c544cd3b33e9e403b7b72404c221bf888d1 with gcc (GCC) 8.1.0 kernel signature: 1fcbbb64e60e6c0ca7c12681b713e19c20b97af8 all runs: OK # git bisect good cf2e8c544cd3b33e9e403b7b72404c221bf888d1 Bisecting: 121 revisions left to test after this (roughly 7 steps) [4221b807d1f73c03d22543416d303b60a5d1ef31] Merge tag 'for-5.1/libata-20190301' of git://git.kernel.dk/linux-block testing commit 4221b807d1f73c03d22543416d303b60a5d1ef31 with gcc (GCC) 8.1.0 kernel signature: c971ba51632358254129c26c5b4f85b25c9f95b0 all runs: OK # git bisect good 4221b807d1f73c03d22543416d303b60a5d1ef31 Bisecting: 60 revisions left to test after this (roughly 6 steps) [19d62f6d00972f957c94aba0975c14490cfed385] block: remove bvec_iter_rewind() testing commit 19d62f6d00972f957c94aba0975c14490cfed385 with gcc (GCC) 8.1.0 kernel signature: 515ee6c487e50f5d8aa7a2789ee39caf3ccfc5be all runs: OK # git bisect good 19d62f6d00972f957c94aba0975c14490cfed385 Bisecting: 30 revisions left to test after this (roughly 5 steps) [5d8762d5684ab997c7ccf2457c8beec7ef972ceb] nvme-rdma: convert to SPDX identifiers testing commit 5d8762d5684ab997c7ccf2457c8beec7ef972ceb with gcc (GCC) 8.1.0 kernel signature: bccb748a0f2c7e42cf1f58bb81e7e9d53f522219 run #0: crashed: kernel BUG at mm/filemap.c:LINE! run #1: crashed: kernel BUG at mm/filemap.c:LINE! run #2: crashed: kernel BUG at mm/filemap.c:LINE! run #3: crashed: kernel BUG at mm/filemap.c:LINE! run #4: crashed: kernel BUG at mm/filemap.c:LINE! run #5: crashed: kernel BUG at mm/filemap.c:LINE! run #6: OK run #7: OK run #8: OK run #9: OK # git bisect bad 5d8762d5684ab997c7ccf2457c8beec7ef972ceb Bisecting: 14 revisions left to test after this (roughly 4 steps) [56d18f62f556b80105e38e7975975cf7465aae3e] block: kill BLK_MQ_F_SG_MERGE testing commit 56d18f62f556b80105e38e7975975cf7465aae3e with gcc (GCC) 8.1.0 kernel signature: 9ebda5dd7dd6f7f6280fa30aa5cc5e07fd05b131 run #0: crashed: kernel BUG at mm/filemap.c:LINE! run #1: crashed: kernel BUG at mm/filemap.c:LINE! run #2: crashed: kernel BUG at mm/filemap.c:LINE! run #3: crashed: kernel BUG at mm/filemap.c:LINE! run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK # git bisect bad 56d18f62f556b80105e38e7975975cf7465aae3e Bisecting: 7 revisions left to test after this (roughly 3 steps) [c3a7ce738009912f9d237bdabf4a20038522de10] btrfs: use mp_bvec_last_segment to get bio's last page testing commit c3a7ce738009912f9d237bdabf4a20038522de10 with gcc (GCC) 8.1.0 kernel signature: ad4981929539a146bb32fdc10a7a516dae733481 all runs: OK # git bisect good c3a7ce738009912f9d237bdabf4a20038522de10 Bisecting: 3 revisions left to test after this (roughly 2 steps) [07173c3ec276cbb18dc0e0687d37d310e98a1480] block: enable multipage bvecs testing commit 07173c3ec276cbb18dc0e0687d37d310e98a1480 with gcc (GCC) 8.1.0 kernel signature: b3708922b36ae1d6189eb8cbd52385e783e1efe9 run #0: crashed: kernel BUG at mm/filemap.c:LINE! run #1: crashed: kernel BUG at mm/filemap.c:LINE! run #2: crashed: kernel BUG at mm/filemap.c:LINE! run #3: crashed: kernel BUG at mm/filemap.c:LINE! run #4: crashed: kernel BUG at mm/filemap.c:LINE! run #5: crashed: kernel BUG at mm/filemap.c:LINE! run #6: crashed: kernel BUG at mm/filemap.c:LINE! run #7: crashed: kernel BUG at mm/filemap.c:LINE! run #8: OK run #9: OK # git bisect bad 07173c3ec276cbb18dc0e0687d37d310e98a1480 Bisecting: 1 revision left to test after this (roughly 1 step) [2e1f4f4d2481d8bf111904c3e45fc0c4c94bf76e] bcache: avoid to use bio_for_each_segment_all() in bch_bio_alloc_pages() testing commit 2e1f4f4d2481d8bf111904c3e45fc0c4c94bf76e with gcc (GCC) 8.1.0 kernel signature: 11d29f9de559c8a88227638e265b1a35980dd134 all runs: OK # git bisect good 2e1f4f4d2481d8bf111904c3e45fc0c4c94bf76e Bisecting: 0 revisions left to test after this (roughly 0 steps) [6dc4f100c175dd0511ae8674786e7c9006cdfbfa] block: allow bio_for_each_segment_all() to iterate over multi-page bvec testing commit 6dc4f100c175dd0511ae8674786e7c9006cdfbfa with gcc (GCC) 8.1.0 kernel signature: b4c32b81b6c444e622ccdac82ba70345320f1ba4 run #0: crashed: kernel BUG at mm/filemap.c:LINE! run #1: crashed: kernel BUG at mm/filemap.c:LINE! run #2: crashed: kernel BUG at mm/filemap.c:LINE! run #3: crashed: kernel BUG at mm/filemap.c:LINE! run #4: crashed: kernel BUG at mm/filemap.c:LINE! run #5: crashed: kernel BUG at mm/filemap.c:LINE! run #6: OK run #7: OK run #8: OK run #9: OK # git bisect bad 6dc4f100c175dd0511ae8674786e7c9006cdfbfa 6dc4f100c175dd0511ae8674786e7c9006cdfbfa is the first bad commit commit 6dc4f100c175dd0511ae8674786e7c9006cdfbfa Author: Ming Lei Date: Fri Feb 15 19:13:19 2019 +0800 block: allow bio_for_each_segment_all() to iterate over multi-page bvec This patch introduces one extra iterator variable to bio_for_each_segment_all(), then we can allow bio_for_each_segment_all() to iterate over multi-page bvec. Given it is just one mechannical & simple change on all bio_for_each_segment_all() users, this patch does tree-wide change in one single patch, so that we can avoid to use a temporary helper for this conversion. Reviewed-by: Omar Sandoval Reviewed-by: Christoph Hellwig Signed-off-by: Ming Lei Signed-off-by: Jens Axboe block/bio.c | 27 ++++++++++++++++++--------- block/bounce.c | 6 ++++-- drivers/md/bcache/btree.c | 3 ++- drivers/md/dm-crypt.c | 3 ++- drivers/md/raid1.c | 3 ++- drivers/staging/erofs/data.c | 3 ++- drivers/staging/erofs/unzip_vle.c | 3 ++- fs/block_dev.c | 6 ++++-- fs/btrfs/compression.c | 3 ++- fs/btrfs/disk-io.c | 3 ++- fs/btrfs/extent_io.c | 9 ++++++--- fs/btrfs/inode.c | 6 ++++-- fs/btrfs/raid56.c | 3 ++- fs/crypto/bio.c | 3 ++- fs/direct-io.c | 4 +++- fs/exofs/ore.c | 3 ++- fs/exofs/ore_raid.c | 3 ++- fs/ext4/page-io.c | 3 ++- fs/ext4/readpage.c | 3 ++- fs/f2fs/data.c | 9 ++++++--- fs/gfs2/lops.c | 9 ++++++--- fs/gfs2/meta_io.c | 3 ++- fs/iomap.c | 6 ++++-- fs/mpage.c | 3 ++- fs/xfs/xfs_aops.c | 5 +++-- include/linux/bio.h | 11 +++++++++-- include/linux/bvec.h | 30 ++++++++++++++++++++++++++++++ 27 files changed, 127 insertions(+), 46 deletions(-) culprit signature: b4c32b81b6c444e622ccdac82ba70345320f1ba4 parent signature: 11d29f9de559c8a88227638e265b1a35980dd134 revisions tested: 20, total time: 5h0m1.94951937s (build: 1h51m34.146963584s, test: 3h6m57.088806172s) first bad commit: 6dc4f100c175dd0511ae8674786e7c9006cdfbfa block: allow bio_for_each_segment_all() to iterate over multi-page bvec cc: ["axboe@kernel.dk" "hch@lst.de" "ming.lei@redhat.com" "osandov@fb.com"] crash: kernel BUG at mm/filemap.c:LINE! attempt to access beyond end of device loop5: rw=1, want=707, limit=624 attempt to access beyond end of device ------------[ cut here ]------------ loop1: rw=1, want=707, limit=624 kernel BUG at mm/filemap.c:1283! invalid opcode: 0000 [#1] PREEMPT SMP KASAN CPU: 0 PID: 9 Comm: ksoftirqd/0 Not tainted 5.0.0-rc4-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 RIP: 0010:end_page_writeback+0x2bd/0x300 mm/filemap.c:1283 Code: fe ff ff 4c 89 ef e8 a2 53 19 00 e9 00 fe ff ff 48 89 df e8 95 53 19 00 e9 01 ff ff ff 4c 89 e7 e8 88 53 19 00 e9 37 fe ff ff <0f> 0b 4c 89 ef e8 79 53 19 00 e9 51 fe ff ff 4c 89 e7 e8 6c 53 19 RSP: 0018:ffff8880a98c7ac8 EFLAGS: 00010246 RAX: 0000000000000000 RBX: ffffea0001d628c0 RCX: 1ffff11015317156 RDX: 1ffffffff10a453e RSI: 1ffff11015317158 RDI: 0000000000000282 RBP: ffff8880a98c7ae0 R08: 0000000000000000 R09: 0000000000000001 R10: ffffed1015d45bcf R11: ffff8880aea2de7b R12: ffffea0001d628c8 R13: ffffea0001d628c0 R14: dffffc0000000000 R15: ffff888090ea08c0 FS: 0000000000000000(0000) GS:ffff8880aea00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f1d32920000 CR3: 0000000093d0a000 CR4: 00000000001406f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: page_endio+0x244/0x4c0 mm/filemap.c:1313 mpage_end_io+0x14f/0x2d0 fs/mpage.c:55 bio_endio+0x459/0x830 block/bio.c:1802 req_bio_endio block/blk-core.c:196 [inline] blk_update_request+0x248/0x9e0 block/blk-core.c:1454 blk_mq_end_request+0x54/0x4e0 block/blk-mq.c:562 lo_complete_rq+0x1af/0x270 drivers/block/loop.c:485 blk_done_softirq+0x2c6/0x490 block/blk-softirq.c:37 __do_softirq+0x260/0x958 kernel/softirq.c:292 run_ksoftirqd+0x94/0x100 kernel/softirq.c:654 smpboot_thread_fn+0x55f/0x860 kernel/smpboot.c:164 kthread+0x324/0x3e0 kernel/kthread.c:246 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352 Modules linked in: ---[ end trace 545ef6405c308b44 ]--- RIP: 0010:end_page_writeback+0x2bd/0x300 mm/filemap.c:1283 Code: fe ff ff 4c 89 ef e8 a2 53 19 00 e9 00 fe ff ff 48 89 df e8 95 53 19 00 e9 01 ff ff ff 4c 89 e7 e8 88 53 19 00 e9 37 fe ff ff <0f> 0b 4c 89 ef e8 79 53 19 00 e9 51 fe ff ff 4c 89 e7 e8 6c 53 19 RSP: 0018:ffff8880a98c7ac8 EFLAGS: 00010246 RAX: 0000000000000000 RBX: ffffea0001d628c0 RCX: 1ffff11015317156 RDX: 1ffffffff10a453e RSI: 1ffff11015317158 RDI: 0000000000000282 RBP: ffff8880a98c7ae0 R08: 0000000000000000 R09: 0000000000000001 R10: ffffed1015d45bcf R11: ffff8880aea2de7b R12: ffffea0001d628c8 R13: ffffea0001d628c0 R14: dffffc0000000000 R15: ffff888090ea08c0 FS: 0000000000000000(0000) GS:ffff8880aea00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f1d32920000 CR3: 0000000008471000 CR4: 00000000001406f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400