syzbot

 sign-in | mailing list | source | docs
🐞 Open [717] 🐞 Fixed [181] 🐞 Invalid [640] 📈 Kernel Health 📈 Bug Lifetimes 📈 Fuzzing 📈 Crashes 💬 Send us feedback

possible deadlock in __generic_file_fsync

Status: upstream: reported syz repro on 2019/06/19 20:20
Subsystems: ext4 fat
[Documentation on labels]
Reported-by: syzbot+86818ef71401e07e80a1@syzkaller.appspotmail.com
First crash: 1765d, last: 428d
Fix bisection: failed (error log, bisect log)
  
▶ ▼ Similar bugs (3)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.19 possible deadlock in __generic_file_fsync ext4 btrfs fat C 577 424d 1771d 0/1 upstream: reported C repro on 2019/06/13 18:12
upstream possible deadlock in __generic_file_fsync ext4 C done unreliable 80 1872d 2012d 0/26 auto-obsoleted due to no activity on 2022/09/04 03:10
android-414 possible deadlock in __generic_file_fsync syz 262 1599d 1834d 0/1 public: reported syz repro on 2019/04/12 00:01
▶ ▼ Fix bisection attempts (6)
Created Duration User Patch Repo Result
2021/09/26 01:53 18m bisect fix linux-4.14.y error job log (0)
2021/08/27 01:14 39m bisect fix linux-4.14.y job log (0) log
2021/07/28 00:07 30m bisect fix linux-4.14.y job log (0) log
2021/06/27 23:29 37m bisect fix linux-4.14.y job log (0) log
2021/02/10 06:26 0m bisect fix linux-4.14.y error job log (0)
2020/10/01 22:19 37m bisect fix linux-4.14.y job log (0) log

Sample crash report:
IPv6: ADDRCONF(NETDEV_CHANGE): veth1_to_batadv: link becomes ready
Bluetooth: hci5 command 0x0419 tx timeout
IPv6: ADDRCONF(NETDEV_CHANGE): batadv_slave_1: link becomes ready
IPv6: ADDRCONF(NETDEV_CHANGE): veth1_to_batadv: link becomes ready
======================================================
WARNING: possible circular locking dependency detected
4.14.290-syzkaller #0 Not tainted
------------------------------------------------------
kworker/1:1/24 is trying to acquire lock:
 (&sb->s_type->i_mutex_key#10){+.+.}, at: [<ffffffff818f942e>] inode_lock include/linux/fs.h:719 [inline]
 (&sb->s_type->i_mutex_key#10){+.+.}, at: [<ffffffff818f942e>] __generic_file_fsync+0x9e/0x190 fs/libfs.c:989

but task is already holding lock:
 ((&dio->complete_work)){+.+.}, at: [<ffffffff81364ee6>] process_one_work+0x6e6/0x14a0 kernel/workqueue.c:2092

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #2 ((&dio->complete_work)){+.+.}:
       process_one_work+0x736/0x14a0 kernel/workqueue.c:2093
       worker_thread+0x5cc/0xff0 kernel/workqueue.c:2251
       kthread+0x30d/0x420 kernel/kthread.c:232
       ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:404

-> #1 ("dio/%s"sb->s_id){+.+.}:
       flush_workqueue+0xfa/0x1310 kernel/workqueue.c:2625
       drain_workqueue+0x177/0x3e0 kernel/workqueue.c:2790
       destroy_workqueue+0x71/0x710 kernel/workqueue.c:4116
       sb_init_dio_done_wq+0x61/0x80 fs/direct-io.c:635
       do_blockdev_direct_IO fs/direct-io.c:1287 [inline]
       __blockdev_direct_IO+0x3df1/0xdcb0 fs/direct-io.c:1423
       ext4_direct_IO_write fs/ext4/inode.c:3749 [inline]
       ext4_direct_IO+0x888/0x1b80 fs/ext4/inode.c:3890
       generic_file_direct_write+0x1df/0x420 mm/filemap.c:2958
       __generic_file_write_iter+0x2a2/0x590 mm/filemap.c:3137
       ext4_file_write_iter+0x276/0xd20 fs/ext4/file.c:270
       call_write_iter include/linux/fs.h:1780 [inline]
       aio_write+0x2ed/0x560 fs/aio.c:1553
       io_submit_one fs/aio.c:1641 [inline]
       do_io_submit+0x847/0x1570 fs/aio.c:1709
       do_syscall_64+0x1d5/0x640 arch/x86/entry/common.c:292
       entry_SYSCALL_64_after_hwframe+0x46/0xbb

-> #0 (&sb->s_type->i_mutex_key#10){+.+.}:
       lock_acquire+0x170/0x3f0 kernel/locking/lockdep.c:3998
       down_write+0x34/0x90 kernel/locking/rwsem.c:54
       inode_lock include/linux/fs.h:719 [inline]
       __generic_file_fsync+0x9e/0x190 fs/libfs.c:989
       ext4_sync_file+0x8ed/0x12c0 fs/ext4/fsync.c:118
       vfs_fsync_range+0x103/0x260 fs/sync.c:196
       generic_write_sync include/linux/fs.h:2684 [inline]
       dio_complete+0x561/0x8d0 fs/direct-io.c:330
       process_one_work+0x793/0x14a0 kernel/workqueue.c:2117
       worker_thread+0x5cc/0xff0 kernel/workqueue.c:2251
       kthread+0x30d/0x420 kernel/kthread.c:232
       ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:404

other info that might help us debug this:

Chain exists of:
  &sb->s_type->i_mutex_key#10 --> "dio/%s"sb->s_id --> (&dio->complete_work)

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock((&dio->complete_work));
                               lock("dio/%s"sb->s_id);
                               lock((&dio->complete_work));
  lock(&sb->s_type->i_mutex_key#10);

 *** DEADLOCK ***

2 locks held by kworker/1:1/24:
 #0:  ("dio/%s"sb->s_id){+.+.}, at: [<ffffffff81364eb0>] process_one_work+0x6b0/0x14a0 kernel/workqueue.c:2088
 #1:  ((&dio->complete_work)){+.+.}, at: [<ffffffff81364ee6>] process_one_work+0x6e6/0x14a0 kernel/workqueue.c:2092

stack backtrace:
CPU: 1 PID: 24 Comm: kworker/1:1 Not tainted 4.14.290-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/22/2022
Workqueue: dio/sda1 dio_aio_complete_work
Call Trace:
 __dump_stack lib/dump_stack.c:17 [inline]
 dump_stack+0x1b2/0x281 lib/dump_stack.c:58
 print_circular_bug.constprop.0.cold+0x2d7/0x41e kernel/locking/lockdep.c:1258
 check_prev_add kernel/locking/lockdep.c:1905 [inline]
 check_prevs_add kernel/locking/lockdep.c:2022 [inline]
 validate_chain kernel/locking/lockdep.c:2464 [inline]
 __lock_acquire+0x2e0e/0x3f20 kernel/locking/lockdep.c:3491
 lock_acquire+0x170/0x3f0 kernel/locking/lockdep.c:3998
 down_write+0x34/0x90 kernel/locking/rwsem.c:54
 inode_lock include/linux/fs.h:719 [inline]
 __generic_file_fsync+0x9e/0x190 fs/libfs.c:989
 ext4_sync_file+0x8ed/0x12c0 fs/ext4/fsync.c:118
 vfs_fsync_range+0x103/0x260 fs/sync.c:196
 generic_write_sync include/linux/fs.h:2684 [inline]
 dio_complete+0x561/0x8d0 fs/direct-io.c:330
 process_one_work+0x793/0x14a0 kernel/workqueue.c:2117
 worker_thread+0x5cc/0xff0 kernel/workqueue.c:2251
 kthread+0x30d/0x420 kernel/kthread.c:232
 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:404
FAT-fs (loop0): Unrecognized mount option "./bus" or missing value
FAT-fs (loop3): Unrecognized mount option "./bus" or missing value
FAT-fs (loop2): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop1): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop2): Unrecognized mount option "./bus" or missing value
FAT-fs (loop1): Unrecognized mount option "./bus" or missing value
FAT-fs (loop0): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop3): Unrecognized mount option "./bus" or missing value
FAT-fs (loop4): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop2): Unrecognized mount option "./bus" or missing value
FAT-fs (loop0): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop1): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop3): Unrecognized mount option "./bus" or missing value
FAT-fs (loop0): Unrecognized mount option "./bus" or missing value
FAT-fs (loop2): Unrecognized mount option "./bus" or missing value
FAT-fs (loop1): Unrecognized mount option "./bus" or missing value
FAT-fs (loop3): Unrecognized mount option "./bus" or missing value
FAT-fs (loop0): Unrecognized mount option "./bus" or missing value
FAT-fs (loop2): Unrecognized mount option "./bus" or missing value
FAT-fs (loop0): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop1): Unrecognized mount option "./bus" or missing value

Crashes (171):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2022/08/16 06:57 linux-4.14.y b641242202ed 7a7cb304 .config console log report syz ci2-linux-4-14 possible deadlock in __generic_file_fsync
2022/06/24 03:01 linux-4.14.y 84bae26850e3 912f5df7 .config console log report syz ci2-linux-4-14 possible deadlock in __generic_file_fsync
2022/06/16 18:55 linux-4.14.y 84bae26850e3 1719ee24 .config console log report syz ci2-linux-4-14 possible deadlock in __generic_file_fsync
2021/12/23 22:01 linux-4.14.y 8ee0807eedf3 6caa12e4 .config console log report syz ci2-linux-4-14 possible deadlock in __generic_file_fsync
2020/08/16 04:10 linux-4.14.y 14b58326976d 5ce13532 .config console log report syz ci2-linux-4-14
2020/03/22 04:42 linux-4.14.y 01364dad1d45 78267cec .config console log report syz ci2-linux-4-14
2023/02/15 20:44 linux-4.14.y a8ad60f2af58 6be0f1f5 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-4-14 possible deadlock in __generic_file_fsync
2023/01/13 19:11 linux-4.14.y c4215ee4771b 529798b0 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-4-14 possible deadlock in __generic_file_fsync
2022/11/13 23:07 linux-4.14.y e911713e40ca 7ba4d859 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-4-14 possible deadlock in __generic_file_fsync
2022/09/23 15:37 linux-4.14.y 4edbf74132a4 0042f2b4 .config console log report info [disk image] [vmlinux] ci2-linux-4-14 possible deadlock in __generic_file_fsync
2022/07/31 10:12 linux-4.14.y b641242202ed fef302b1 .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2022/07/25 21:35 linux-4.14.y 9c3bf9cf362f 34795c51 .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2022/07/14 13:14 linux-4.14.y 424a46ea058e 5d921b08 .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2022/07/12 18:30 linux-4.14.y 424a46ea058e d91dd8ea .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2022/07/12 00:42 linux-4.14.y 1048779a1d7d da3d6955 .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2022/06/16 13:15 linux-4.14.y 84bae26850e3 1719ee24 .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2022/05/02 02:25 linux-4.14.y e3a56aaade89 2df221f6 .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2022/05/01 11:07 linux-4.14.y e3a56aaade89 2df221f6 .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2022/04/07 22:33 linux-4.14.y 74766a973637 c6ff3e05 .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2022/01/19 03:16 linux-4.14.y 4ba8e26127c3 731a2d23 .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2022/01/17 23:43 linux-4.14.y 4ba8e26127c3 731a2d23 .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2022/01/17 01:54 linux-4.14.y 4ba8e26127c3 723cfaf0 .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2022/01/12 10:56 linux-4.14.y 4ba8e26127c3 44d1319a .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2022/01/11 07:51 linux-4.14.y bfdef05c8da4 ddb0ab8c .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2021/12/06 16:48 linux-4.14.y 66722c42ec91 579a8754 .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2021/12/06 09:19 linux-4.14.y 66722c42ec91 a617004c .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2021/05/28 23:23 linux-4.14.y ad8397a84e1e 858ea628 .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2021/05/09 23:14 linux-4.14.y 7d7d1c0ab3eb bc5434be .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2021/05/06 05:11 linux-4.14.y 7d7d1c0ab3eb 06c27ff5 .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2021/04/15 17:31 linux-4.14.y 958e517f4e16 c59079a6 .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2021/04/01 20:05 linux-4.14.y bd634aa64163 6a81331a .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2021/03/28 12:49 linux-4.14.y 670d6552eda8 a8529b82 .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2021/03/07 23:57 linux-4.14.y 1d177c0872ab 09fbf400 .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2021/02/26 10:48 linux-4.14.y 3242aa3a635c 4c37c133 .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2021/02/26 05:34 linux-4.14.y 3242aa3a635c 76f7fc95 .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2021/02/16 19:22 linux-4.14.y 2c8a3fceddf0 98682e5e .config console log report info ci2-linux-4-14 possible deadlock in __generic_file_fsync
2021/01/11 06:26 linux-4.14.y ec822b3e8bf4 2c1f2513 .config console log report info ci2-linux-4-14
2021/01/08 06:00 linux-4.14.y 1752938529c6 c104d4a3 .config console log report info ci2-linux-4-14
2020/12/29 04:12 linux-4.14.y 3f2ecb86cb90 8259d56c .config console log report info ci2-linux-4-14
2020/12/23 13:24 linux-4.14.y 3f2ecb86cb90 c2c1d1dd .config console log report info ci2-linux-4-14
2020/12/21 23:44 linux-4.14.y 3f2ecb86cb90 04201c06 .config console log report info ci2-linux-4-14
2020/12/19 17:24 linux-4.14.y 3f2ecb86cb90 04201c06 .config console log report info ci2-linux-4-14
2020/12/19 10:53 linux-4.14.y 3f2ecb86cb90 04201c06 .config console log report info ci2-linux-4-14
2020/12/06 08:56 linux-4.14.y c196b3a9c83a f12ba0c5 .config console log report info ci2-linux-4-14
2020/11/15 08:28 linux-4.14.y 27ce4f2a6817 1bf9a662 .config console log report info ci2-linux-4-14
2020/10/28 20:29 linux-4.14.y 5b7a52cd2eef f24824d3 .config console log report info ci2-linux-4-14
2020/09/01 17:20 linux-4.14.y d7e78d08fa77 d5a3ae1f .config console log report ci2-linux-4-14
2020/08/30 17:00 linux-4.14.y d7e78d08fa77 d5a3ae1f .config console log report ci2-linux-4-14
2020/08/30 15:17 linux-4.14.y d7e78d08fa77 d5a3ae1f .config console log report ci2-linux-4-14
2019/06/19 19:19 linux-4.14.y bb263a2a2d43 34bf9440 .config console log report ci2-linux-4-14
* Struck through repros no longer work on HEAD.