ci starts bisection 2023-03-01 22:20:02.119937709 +0000 UTC m=+43528.491190555 bisecting cause commit starting from 1716a175592aff9549a0c07aac8f9cadd03003f5 building syzkaller on f8902b5747fbe3d5b860bd782eec63fc9c7da6e7 ensuring issue is reproducible on original commit 1716a175592aff9549a0c07aac8f9cadd03003f5 testing commit 1716a175592aff9549a0c07aac8f9cadd03003f5 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 90195ca6b16c56e8093ea1f493d8d2762199e52bfdda97d0a33bca3d81e11746 all runs: crashed: possible deadlock in collapse_file testing release v6.2 testing commit c9c3395d5e3dcc6daee66c6908354d47bf98cb0c gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: ab0499845c45ebc5999dc24b23472c6420c626d04fde28cfbdbe64f48bf8722b all runs: OK # git bisect start 1716a175592aff9549a0c07aac8f9cadd03003f5 c9c3395d5e3dcc6daee66c6908354d47bf98cb0c Bisecting: 7554 revisions left to test after this (roughly 13 steps) [307e14c039063f0c9bd7a18a7add8f940580dcc9] Merge tag '6.3-rc-smb3-client-fixes' of git://git.samba.org/sfrench/cifs-2.6 testing commit 307e14c039063f0c9bd7a18a7add8f940580dcc9 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 0871c94db60b38381b6ad6417ee3bd154d8700ac54d5b0170ea7eac9fb3afe30 all runs: OK # git bisect good 307e14c039063f0c9bd7a18a7add8f940580dcc9 Bisecting: 3746 revisions left to test after this (roughly 12 steps) [90ddb3f03418cce0d83c415c0c1d470cf524ba46] Merge tag 'pci-v6.3-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/pci/pci testing commit 90ddb3f03418cce0d83c415c0c1d470cf524ba46 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 3d97f6c0f4d4fa97b31230e1951bf5e3acc61c7b83a7e61e44b9ddcfdbcedefb run #0: OK run #1: OK run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: boot failed: can't ssh into the instance # git bisect good 90ddb3f03418cce0d83c415c0c1d470cf524ba46 Bisecting: 1881 revisions left to test after this (roughly 11 steps) [cc38a46de76e15d20bea5768e99af17b65a9caeb] Merge tag 'rpmsg-v6.3' of git://git.kernel.org/pub/scm/linux/kernel/git/remoteproc/linux testing commit cc38a46de76e15d20bea5768e99af17b65a9caeb gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: c5809318eaea8b512171aef9e4eb36cc58750ea360cca0efcbdfa688752f70ac all runs: OK # git bisect good cc38a46de76e15d20bea5768e99af17b65a9caeb Bisecting: 941 revisions left to test after this (roughly 10 steps) [8770739123e236a33d695939f219bba58014e58d] Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/krzk/linux.git testing commit 8770739123e236a33d695939f219bba58014e58d gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 80c141a9776c60ac25989a15bf766b4bfafccb94d4b7ccbe176a9c766a1e8f13 all runs: crashed: possible deadlock in retract_page_tables # git bisect bad 8770739123e236a33d695939f219bba58014e58d Bisecting: 444 revisions left to test after this (roughly 9 steps) [103830683cfc8f43b15158b0a48014b6d6e83633] Merge tag 'f2fs-for-6.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs testing commit 103830683cfc8f43b15158b0a48014b6d6e83633 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: f6d8caba32ad81494bbc8c4d54bdd54e3afcf90978876200981ecdf5a10c0d24 all runs: OK # git bisect good 103830683cfc8f43b15158b0a48014b6d6e83633 Bisecting: 256 revisions left to test after this (roughly 8 steps) [654fac3d4940765f25891cd66cc7d5fbaf471f98] Merge branch 'mm-nonmm-stable' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm testing commit 654fac3d4940765f25891cd66cc7d5fbaf471f98 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 5ad662a6e794527e42d69ef12dfa2a1849cc17316196d125c4dfe0e7027ae138 all runs: OK # git bisect good 654fac3d4940765f25891cd66cc7d5fbaf471f98 Bisecting: 110 revisions left to test after this (roughly 7 steps) [049de297d1f9dcaa528802d6f1bcc31312d6781c] Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc.git testing commit 049de297d1f9dcaa528802d6f1bcc31312d6781c gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 62fecdc7760d58217c1e9ee99768eecdb0594749a9ca9d2f272fcadf306d9b41 all runs: crashed: possible deadlock in retract_page_tables # git bisect bad 049de297d1f9dcaa528802d6f1bcc31312d6781c Bisecting: 72 revisions left to test after this (roughly 6 steps) [3a9b6c5bbe06d15b305e3669b8d6483ee7ef20b9] mm: prevent userfaults to be handled under per-vma lock testing commit 3a9b6c5bbe06d15b305e3669b8d6483ee7ef20b9 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: df9a483921b20cd5f06430cc8cb37664a4592a58c03b97ac268b7a83621f1e34 all runs: OK # git bisect good 3a9b6c5bbe06d15b305e3669b8d6483ee7ef20b9 Bisecting: 36 revisions left to test after this (roughly 5 steps) [491a4ce54443854bc2ca60b09107dc7c9e97886a] Merge branch 'soc/drivers' into for-next testing commit 491a4ce54443854bc2ca60b09107dc7c9e97886a gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: ad738b58acd463efb4e16f85d53f42cac7acdd2fefcc96cae39f3324315f54ea all runs: OK # git bisect good 491a4ce54443854bc2ca60b09107dc7c9e97886a Bisecting: 17 revisions left to test after this (roughly 4 steps) [0e2f689a39052bbe82d4e81126596e1d31abddd0] Merge branch 'compiler-attributes' of https://github.com/ojeda/linux.git testing commit 0e2f689a39052bbe82d4e81126596e1d31abddd0 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: ef1a1a7fb27827b2db868a03da133dcdb133eff37a706e7d4263f9b3c94ba58c all runs: crashed: possible deadlock in retract_page_tables # git bisect bad 0e2f689a39052bbe82d4e81126596e1d31abddd0 Bisecting: 9 revisions left to test after this (roughly 3 steps) [68eb89322a466968c445517391d82033b89bfdf7] scripts/gdb: support getting current task struct in UML testing commit 68eb89322a466968c445517391d82033b89bfdf7 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 6928552367c2c33a7d21f4782900b10bd50bff10d134c9983f0dabbb52cf7ba9 all runs: OK # git bisect good 68eb89322a466968c445517391d82033b89bfdf7 Bisecting: 4 revisions left to test after this (roughly 2 steps) [e93b519012226f2283c8dcfa2c2238bf5a66a22c] mm/mmap: free vm_area_struct without call_rcu in exit_mmap testing commit e93b519012226f2283c8dcfa2c2238bf5a66a22c gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 9d6982abe98278087261602d36aba46760ebd5769887de8a542f417a06e6ebb3 all runs: crashed: possible deadlock in retract_page_tables # git bisect bad e93b519012226f2283c8dcfa2c2238bf5a66a22c Bisecting: 2 revisions left to test after this (roughly 1 step) [3d7cb67369a08d4933713290acf458990a50b6f9] x86/mm: try VMA lock-based page fault handling first testing commit 3d7cb67369a08d4933713290acf458990a50b6f9 gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 0915335f367de07252fb6a9b9346b2ba6aeeec44dd2f996128b2448466a82d75 all runs: crashed: possible deadlock in retract_page_tables # git bisect bad 3d7cb67369a08d4933713290acf458990a50b6f9 Bisecting: 0 revisions left to test after this (roughly 0 steps) [36306b229dfd5488c7c15c3795cc3bae611e5d2f] mm: introduce per-VMA lock statistics testing commit 36306b229dfd5488c7c15c3795cc3bae611e5d2f gcc compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: caa574d5505b2a7184fc3a76b5a147db0bb235f0053ca25ae2c748a70f6c61ac all runs: OK # git bisect good 36306b229dfd5488c7c15c3795cc3bae611e5d2f 3d7cb67369a08d4933713290acf458990a50b6f9 is the first bad commit commit 3d7cb67369a08d4933713290acf458990a50b6f9 Author: Suren Baghdasaryan Date: Mon Feb 27 09:36:28 2023 -0800 x86/mm: try VMA lock-based page fault handling first Attempt VMA lock-based page fault handling first, and fall back to the existing mmap_lock-based handling if that fails. Link: https://lkml.kernel.org/r/20230227173632.3292573-30-surenb@google.com Signed-off-by: Suren Baghdasaryan Signed-off-by: Andrew Morton arch/x86/Kconfig | 1 + arch/x86/mm/fault.c | 36 ++++++++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+) culprit signature: 0915335f367de07252fb6a9b9346b2ba6aeeec44dd2f996128b2448466a82d75 parent signature: caa574d5505b2a7184fc3a76b5a147db0bb235f0053ca25ae2c748a70f6c61ac revisions tested: 16, total time: 5h43m22.238573904s (build: 3h47m8.255969875s, test: 1h53m43.216598355s) first bad commit: 3d7cb67369a08d4933713290acf458990a50b6f9 x86/mm: try VMA lock-based page fault handling first recipients (to): ["akpm@linux-foundation.org" "surenb@google.com"] recipients (cc): [] crash: possible deadlock in retract_page_tables ====================================================== WARNING: possible circular locking dependency detected 6.2.0-syzkaller #0 Not tainted ------------------------------------------------------ syz-executor.0/5575 is trying to acquire lock: ffff88807703daa8 (&vma->lock){++++}-{3:3}, at: vma_start_write include/linux/mm.h:682 [inline] ffff88807703daa8 (&vma->lock){++++}-{3:3}, at: retract_page_tables+0x59d/0x8d0 mm/khugepaged.c:1826 but task is already holding lock: ffff888077f4efa8 (&mapping->i_mmap_rwsem){++++}-{3:3}, at: i_mmap_lock_write include/linux/fs.h:468 [inline] ffff888077f4efa8 (&mapping->i_mmap_rwsem){++++}-{3:3}, at: retract_page_tables+0x98/0x8d0 mm/khugepaged.c:1745 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #2 (&mapping->i_mmap_rwsem){++++}-{3:3}: down_write+0x92/0x200 kernel/locking/rwsem.c:1573 i_mmap_lock_write include/linux/fs.h:468 [inline] dma_resv_lockdep+0x242/0x500 drivers/dma-buf/dma-resv.c:760 do_one_initcall+0xf8/0x580 init/main.c:1306 do_initcall_level init/main.c:1379 [inline] do_initcalls init/main.c:1395 [inline] do_basic_setup init/main.c:1414 [inline] kernel_init_freeable+0x5e4/0xb50 init/main.c:1634 kernel_init+0x1a/0x1c0 init/main.c:1522 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308 -> #1 (fs_reclaim){+.+.}-{0:0}: __fs_reclaim_acquire mm/page_alloc.c:4647 [inline] fs_reclaim_acquire+0x11d/0x160 mm/page_alloc.c:4661 might_alloc include/linux/sched/mm.h:299 [inline] prepare_alloc_pages+0x159/0x570 mm/page_alloc.c:5293 __alloc_pages+0x149/0x5c0 mm/page_alloc.c:5511 __folio_alloc+0x16/0x40 mm/page_alloc.c:5554 vma_alloc_folio+0x11a/0x690 mm/mempolicy.c:2244 do_anonymous_page mm/memory.c:4062 [inline] handle_pte_fault mm/memory.c:4917 [inline] __handle_mm_fault+0x10b1/0x2f20 mm/memory.c:5061 handle_mm_fault+0x245/0x7a0 mm/memory.c:5207 do_user_addr_fault+0x1bf/0xc00 arch/x86/mm/fault.c:1349 handle_page_fault arch/x86/mm/fault.c:1534 [inline] exc_page_fault+0x5e/0xc0 arch/x86/mm/fault.c:1590 asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:570 -> #0 (&vma->lock){++++}-{3:3}: check_prev_add kernel/locking/lockdep.c:3098 [inline] check_prevs_add kernel/locking/lockdep.c:3217 [inline] validate_chain kernel/locking/lockdep.c:3832 [inline] __lock_acquire+0x2ec7/0x5d40 kernel/locking/lockdep.c:5056 lock_acquire kernel/locking/lockdep.c:5669 [inline] lock_acquire+0x1e3/0x670 kernel/locking/lockdep.c:5634 down_write+0x92/0x200 kernel/locking/rwsem.c:1573 vma_start_write include/linux/mm.h:682 [inline] retract_page_tables+0x59d/0x8d0 mm/khugepaged.c:1826 collapse_file+0x197e/0x3bd0 mm/khugepaged.c:2204 hpage_collapse_scan_file+0x965/0x1050 mm/khugepaged.c:2358 madvise_collapse+0x457/0xa70 mm/khugepaged.c:2818 madvise_vma_behavior+0x531/0x1910 mm/madvise.c:1086 madvise_walk_vmas+0x155/0x230 mm/madvise.c:1260 do_madvise.part.0+0x21a/0x370 mm/madvise.c:1439 do_madvise mm/madvise.c:1452 [inline] __do_sys_madvise mm/madvise.c:1452 [inline] __se_sys_madvise mm/madvise.c:1450 [inline] __x64_sys_madvise+0xd0/0x130 mm/madvise.c:1450 do_syscall_x64 arch/x86/entry/common.c:50 [inline] do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80 entry_SYSCALL_64_after_hwframe+0x63/0xcd other info that might help us debug this: Chain exists of: &vma->lock --> fs_reclaim --> &mapping->i_mmap_rwsem Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(&mapping->i_mmap_rwsem); lock(fs_reclaim); lock(&mapping->i_mmap_rwsem); lock(&vma->lock); *** DEADLOCK *** 2 locks held by syz-executor.0/5575: #0: ffff888077f4efa8 (&mapping->i_mmap_rwsem){++++}-{3:3}, at: i_mmap_lock_write include/linux/fs.h:468 [inline] #0: ffff888077f4efa8 (&mapping->i_mmap_rwsem){++++}-{3:3}, at: retract_page_tables+0x98/0x8d0 mm/khugepaged.c:1745 #1: ffff888027de8ad8 (&mm->mmap_lock){++++}-{3:3}, at: mmap_write_trylock include/linux/mmap_lock.h:120 [inline] #1: ffff888027de8ad8 (&mm->mmap_lock){++++}-{3:3}, at: retract_page_tables+0x381/0x8d0 mm/khugepaged.c:1797 stack backtrace: CPU: 0 PID: 5575 Comm: syz-executor.0 Not tainted 6.2.0-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 02/16/2023 Call Trace: __dump_stack lib/dump_stack.c:88 [inline] dump_stack_lvl+0x64/0xb0 lib/dump_stack.c:106 check_noncircular+0x25f/0x2e0 kernel/locking/lockdep.c:2178 check_prev_add kernel/locking/lockdep.c:3098 [inline] check_prevs_add kernel/locking/lockdep.c:3217 [inline] validate_chain kernel/locking/lockdep.c:3832 [inline] __lock_acquire+0x2ec7/0x5d40 kernel/locking/lockdep.c:5056 lock_acquire kernel/locking/lockdep.c:5669 [inline] lock_acquire+0x1e3/0x670 kernel/locking/lockdep.c:5634 down_write+0x92/0x200 kernel/locking/rwsem.c:1573 vma_start_write include/linux/mm.h:682 [inline] retract_page_tables+0x59d/0x8d0 mm/khugepaged.c:1826 collapse_file+0x197e/0x3bd0 mm/khugepaged.c:2204 hpage_collapse_scan_file+0x965/0x1050 mm/khugepaged.c:2358 madvise_collapse+0x457/0xa70 mm/khugepaged.c:2818 madvise_vma_behavior+0x531/0x1910 mm/madvise.c:1086 madvise_walk_vmas+0x155/0x230 mm/madvise.c:1260 do_madvise.part.0+0x21a/0x370 mm/madvise.c:1439 do_madvise mm/madvise.c:1452 [inline] __do_sys_madvise mm/madvise.c:1452 [inline] __se_sys_madvise mm/madvise.c:1450 [inline] __x64_sys_madvise+0xd0/0x130 mm/madvise.c:1450 do_syscall_x64 arch/x86/entry/common.c:50 [inline] do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80 entry_SYSCALL_64_after_hwframe+0x63/0xcd RIP: 0033:0x7f19bb08c0f9 Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 f1 19 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48 RSP: 002b:00007f19bbd0e168 EFLAGS: 00000246 ORIG_RAX: 000000000000001c RAX: ffffffffffffffda RBX: 00007f19bb1abf80 RCX: 00007f19bb08c0f9 RDX: 0000000000000019 RSI: 0000000000600003 RDI: 0000000020000000 RBP: 00007f19bb0e7ae9 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000 R13: 00007ffd1c0b36cf R14: 00007f19bbd0e300 R15: 0000000000022000