syzbot

 sign-in | mailing list | source | docs
🐞 Open [717] 🐞 Fixed [181] 🐞 Invalid [640] 📈 Kernel Health 📈 Bug Lifetimes 📈 Fuzzing 📈 Crashes 💬 Send us feedback

possible deadlock in ext4_file_write_iter (2)

Status: upstream: reported syz repro on 2020/10/24 08:36
Reported-by: syzbot+735bf1ddeace671aa13b@syzkaller.appspotmail.com
First crash: 1280d, last: 535d
Fix bisection the fix commit could be any of (bisect log):
  fe4c89150d9b include/trace/events/writeback.h: fix -Wstringop-truncation warnings
  d24cf6d0d72a memcg: fix a crash in wb_workfn when a device disappears
   
▶ ▼ Similar bugs (4)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.14 possible deadlock in ext4_file_write_iter 3 1440d 1492d 0/1 auto-closed as invalid on 2020/09/14 06:49
linux-4.19 possible deadlock in ext4_file_write_iter (2) 1 1001d 1001d 0/1 auto-closed as invalid on 2021/11/26 16:39
linux-4.19 possible deadlock in ext4_file_write_iter (3) hfsplus syz error 13 456d 853d 0/1 upstream: reported syz repro on 2021/12/24 22:56
linux-4.19 possible deadlock in ext4_file_write_iter 1 1423d 1423d 0/1 auto-closed as invalid on 2020/10/01 00:41
▶ ▼ Fix bisection attempts (2)
Created Duration User Patch Repo Result
2021/02/25 03:14 2h38m bisect fix linux-4.14.y job log (2)
2021/02/17 03:46 0m bisect fix linux-4.14.y error job log (0)

Sample crash report:
Bluetooth: hci3 command 0x0419 tx timeout
Bluetooth: hci5 command 0x0419 tx timeout
Bluetooth: hci4 command 0x0419 tx timeout
Bluetooth: hci2 command 0x0419 tx timeout
======================================================
WARNING: possible circular locking dependency detected
4.14.294-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor.2/9623 is trying to acquire lock:
 ("dio/%s"sb->s_id){+.+.}, at: [<ffffffff8135b8eb>] flush_workqueue+0xcb/0x1310 kernel/workqueue.c:2622

but task is already holding lock:
 (&sb->s_type->i_mutex_key#10){+.+.}, at: [<ffffffff81b78d8c>] inode_trylock include/linux/fs.h:739 [inline]
 (&sb->s_type->i_mutex_key#10){+.+.}, at: [<ffffffff81b78d8c>] ext4_file_write_iter+0x1cc/0xd20 fs/ext4/file.c:236

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #2 (&sb->s_type->i_mutex_key#10){+.+.}:
       down_write+0x34/0x90 kernel/locking/rwsem.c:54
       inode_lock include/linux/fs.h:719 [inline]
       __generic_file_fsync+0x9e/0x190 fs/libfs.c:989
       ext4_sync_file+0x8ed/0x12c0 fs/ext4/fsync.c:118
       vfs_fsync_range+0x103/0x260 fs/sync.c:196
       generic_write_sync include/linux/fs.h:2684 [inline]
       dio_complete+0x561/0x8d0 fs/direct-io.c:330
       process_one_work+0x793/0x14a0 kernel/workqueue.c:2117
       worker_thread+0x5cc/0xff0 kernel/workqueue.c:2251
       kthread+0x30d/0x420 kernel/kthread.c:232
       ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:404

-> #1 ((&dio->complete_work)){+.+.}:
       process_one_work+0x736/0x14a0 kernel/workqueue.c:2093
       worker_thread+0x5cc/0xff0 kernel/workqueue.c:2251
       kthread+0x30d/0x420 kernel/kthread.c:232
       ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:404

-> #0 ("dio/%s"sb->s_id){+.+.}:
       lock_acquire+0x170/0x3f0 kernel/locking/lockdep.c:3998
       flush_workqueue+0xfa/0x1310 kernel/workqueue.c:2625
       drain_workqueue+0x177/0x3e0 kernel/workqueue.c:2790
FAT-fs (loop4): Unrecognized mount option "./bus" or missing value
       destroy_workqueue+0x71/0x710 kernel/workqueue.c:4116
       sb_init_dio_done_wq+0x61/0x80 fs/direct-io.c:635
       do_blockdev_direct_IO fs/direct-io.c:1287 [inline]
       __blockdev_direct_IO+0x3df1/0xdcb0 fs/direct-io.c:1423
       ext4_direct_IO_write fs/ext4/inode.c:3756 [inline]
       ext4_direct_IO+0x888/0x1b80 fs/ext4/inode.c:3897
       generic_file_direct_write+0x1df/0x420 mm/filemap.c:2958
       __generic_file_write_iter+0x2a2/0x590 mm/filemap.c:3137
       ext4_file_write_iter+0x276/0xd20 fs/ext4/file.c:270
       call_write_iter include/linux/fs.h:1780 [inline]
       aio_write+0x2ed/0x560 fs/aio.c:1553
       io_submit_one fs/aio.c:1641 [inline]
       do_io_submit+0x847/0x1570 fs/aio.c:1709
       do_syscall_64+0x1d5/0x640 arch/x86/entry/common.c:292
       entry_SYSCALL_64_after_hwframe+0x46/0xbb

other info that might help us debug this:

Chain exists of:
  "dio/%s"sb->s_id --> (&dio->complete_work) --> &sb->s_type->i_mutex_key#10

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&sb->s_type->i_mutex_key#10);
                               lock((&dio->complete_work));
                               lock(&sb->s_type->i_mutex_key#10);
  lock("dio/%s"sb->s_id);

 *** DEADLOCK ***

2 locks held by syz-executor.2/9623:
 #0:  (sb_writers#3){.+.+}, at: [<ffffffff819847e8>] file_start_write include/linux/fs.h:2714 [inline]
 #0:  (sb_writers#3){.+.+}, at: [<ffffffff819847e8>] aio_write+0x408/0x560 fs/aio.c:1552
 #1:  (&sb->s_type->i_mutex_key#10){+.+.}, at: [<ffffffff81b78d8c>] inode_trylock include/linux/fs.h:739 [inline]
 #1:  (&sb->s_type->i_mutex_key#10){+.+.}, at: [<ffffffff81b78d8c>] ext4_file_write_iter+0x1cc/0xd20 fs/ext4/file.c:236

stack backtrace:
CPU: 0 PID: 9623 Comm: syz-executor.2 Not tainted 4.14.294-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/26/2022
Call Trace:
 __dump_stack lib/dump_stack.c:17 [inline]
 dump_stack+0x1b2/0x281 lib/dump_stack.c:58
 print_circular_bug.constprop.0.cold+0x2d7/0x41e kernel/locking/lockdep.c:1258
 check_prev_add kernel/locking/lockdep.c:1905 [inline]
 check_prevs_add kernel/locking/lockdep.c:2022 [inline]
 validate_chain kernel/locking/lockdep.c:2464 [inline]
 __lock_acquire+0x2e0e/0x3f20 kernel/locking/lockdep.c:3491
 lock_acquire+0x170/0x3f0 kernel/locking/lockdep.c:3998
 flush_workqueue+0xfa/0x1310 kernel/workqueue.c:2625
 drain_workqueue+0x177/0x3e0 kernel/workqueue.c:2790
 destroy_workqueue+0x71/0x710 kernel/workqueue.c:4116
 sb_init_dio_done_wq+0x61/0x80 fs/direct-io.c:635
 do_blockdev_direct_IO fs/direct-io.c:1287 [inline]
 __blockdev_direct_IO+0x3df1/0xdcb0 fs/direct-io.c:1423
 ext4_direct_IO_write fs/ext4/inode.c:3756 [inline]
 ext4_direct_IO+0x888/0x1b80 fs/ext4/inode.c:3897
 generic_file_direct_write+0x1df/0x420 mm/filemap.c:2958
 __generic_file_write_iter+0x2a2/0x590 mm/filemap.c:3137
 ext4_file_write_iter+0x276/0xd20 fs/ext4/file.c:270
 call_write_iter include/linux/fs.h:1780 [inline]
 aio_write+0x2ed/0x560 fs/aio.c:1553
 io_submit_one fs/aio.c:1641 [inline]
 do_io_submit+0x847/0x1570 fs/aio.c:1709
 do_syscall_64+0x1d5/0x640 arch/x86/entry/common.c:292
 entry_SYSCALL_64_after_hwframe+0x46/0xbb
RIP: 0033:0x7fb105029669
RSP: 002b:00007fb10477c168 EFLAGS: 00000246 ORIG_RAX: 00000000000000d1
RAX: ffffffffffffffda RBX: 00007fb10514b050 RCX: 00007fb105029669
RDX: 0000000020000540 RSI: 0000000000001801 RDI: 00007fb105126000
RBP: 00007fb105084560 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007ffccb103e3f R14: 00007fb10477c300 R15: 0000000000022000
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop3): Unrecognized mount option "./bus" or missing value
FAT-fs (loop1): Unrecognized mount option "./bus" or missing value
FAT-fs (loop0): Unrecognized mount option "./bus" or missing value
FAT-fs (loop2): Unrecognized mount option "./bus" or missing value
FAT-fs (loop4): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop3): Unrecognized mount option "./bus" or missing value
FAT-fs (loop0): Unrecognized mount option "./bus" or missing value
FAT-fs (loop3): Unrecognized mount option "./bus" or missing value
FAT-fs (loop1): Unrecognized mount option "./bus" or missing value
FAT-fs (loop0): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop0): Unrecognized mount option "./bus" or missing value
FAT-fs (loop4): Unrecognized mount option "./bus" or missing value
FAT-fs (loop1): Unrecognized mount option "./bus" or missing value
FAT-fs (loop2): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop4): Unrecognized mount option "./bus" or missing value
FAT-fs (loop2): Unrecognized mount option "./bus" or missing value
FAT-fs (loop1): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop0): Unrecognized mount option "./bus" or missing value
FAT-fs (loop1): Unrecognized mount option "./bus" or missing value
FAT-fs (loop0): Unrecognized mount option "./bus" or missing value
FAT-fs (loop1): Unrecognized mount option "./bus" or missing value
FAT-fs (loop3): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop1): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop1): Unrecognized mount option "./bus" or missing value
FAT-fs (loop4): Unrecognized mount option "./bus" or missing value
FAT-fs (loop2): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop0): Unrecognized mount option "./bus" or missing value
FAT-fs (loop1): Unrecognized mount option "./bus" or missing value
FAT-fs (loop2): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop1): Unrecognized mount option "./bus" or missing value
FAT-fs (loop2): Unrecognized mount option "./bus" or missing value

Crashes (19):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2022/09/24 00:05 linux-4.14.y 4edbf74132a4 0042f2b4 .config console log report syz [disk image] [vmlinux] ci2-linux-4-14 possible deadlock in ext4_file_write_iter
2022/09/10 18:45 linux-4.14.y 65640c873dcf 356d8217 .config console log report syz ci2-linux-4-14 possible deadlock in ext4_file_write_iter
2022/08/24 02:24 linux-4.14.y b641242202ed cea8b0f7 .config console log report syz ci2-linux-4-14 possible deadlock in ext4_file_write_iter
2022/07/31 11:50 linux-4.14.y b641242202ed fef302b1 .config console log report syz ci2-linux-4-14 possible deadlock in ext4_file_write_iter
2022/07/12 05:45 linux-4.14.y 1048779a1d7d da3d6955 .config console log report syz ci2-linux-4-14 possible deadlock in ext4_file_write_iter
2022/03/29 02:29 linux-4.14.y af1af6ebca0e 6bdac766 .config console log report syz ci2-linux-4-14 possible deadlock in ext4_file_write_iter
2021/01/18 03:46 linux-4.14.y 2762b48e9611 fd103621 .config console log report syz ci2-linux-4-14 possible deadlock in ext4_file_write_iter
2022/11/07 21:09 linux-4.14.y a901bb6c7db7 a779b11a .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-4-14 possible deadlock in ext4_file_write_iter
2022/10/27 13:12 linux-4.14.y 41f36d7859a7 5c716ff6 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-4-14 possible deadlock in ext4_file_write_iter
2022/09/26 13:53 linux-4.14.y 4edbf74132a4 d59ba983 .config console log report info ci2-linux-4-14 possible deadlock in ext4_file_write_iter
2022/07/27 16:26 linux-4.14.y 9c3bf9cf362f da9d0366 .config console log report info ci2-linux-4-14 possible deadlock in ext4_file_write_iter
2022/07/06 08:10 linux-4.14.y ed2e96e11936 bff65f44 .config console log report info ci2-linux-4-14 possible deadlock in ext4_file_write_iter
2022/07/05 14:18 linux-4.14.y ed2e96e11936 bff65f44 .config console log report info ci2-linux-4-14 possible deadlock in ext4_file_write_iter
2022/05/07 15:00 linux-4.14.y e3a56aaade89 e60b1103 .config console log report info ci2-linux-4-14 possible deadlock in ext4_file_write_iter
2022/05/07 02:35 linux-4.14.y e3a56aaade89 e60b1103 .config console log report info ci2-linux-4-14 possible deadlock in ext4_file_write_iter
2022/03/28 20:37 linux-4.14.y af1af6ebca0e 6bdac766 .config console log report info ci2-linux-4-14 possible deadlock in ext4_file_write_iter
2020/12/06 22:29 linux-4.14.y c196b3a9c83a c521566d .config console log report info ci2-linux-4-14
2020/11/13 10:24 linux-4.14.y 27ce4f2a6817 16fca0c8 .config console log report info ci2-linux-4-14
2020/10/24 08:35 linux-4.14.y 5b7a52cd2eef 2bb6666c .config console log report info ci2-linux-4-14
* Struck through repros no longer work on HEAD.