===================================================== WARNING: SOFTIRQ-safe -> SOFTIRQ-unsafe lock order detected 5.4.0-rc5-next-20191028+ #0 Not tainted ----------------------------------------------------- khugepaged/1083 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: ffff888090de7928 (&sb->s_type->i_lock_key#23){+.+.}, at: spin_lock include/linux/spinlock.h:338 [inline] ffff888090de7928 (&sb->s_type->i_lock_key#23){+.+.}, at: wbc_attach_fdatawrite_inode include/linux/writeback.h:266 [inline] ffff888090de7928 (&sb->s_type->i_lock_key#23){+.+.}, at: __filemap_fdatawrite_range+0x26e/0x3b0 mm/filemap.c:420 and this task is already holding: ffff888090de7ab0 (&(&xa->xa_lock)->rlock#4){..-.}, at: spin_lock_irq include/linux/spinlock.h:363 [inline] ffff888090de7ab0 (&(&xa->xa_lock)->rlock#4){..-.}, at: collapse_file+0x24d/0x4580 mm/khugepaged.c:1524 which would create a new lock dependency: (&(&xa->xa_lock)->rlock#4){..-.} -> (&sb->s_type->i_lock_key#23){+.+.} but this new dependency connects a SOFTIRQ-irq-safe lock: (&(&xa->xa_lock)->rlock#4){..-.} ... which became SOFTIRQ-irq-safe at: lock_acquire+0x190/0x410 kernel/locking/lockdep.c:4487 __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline] _raw_spin_lock_irqsave+0x95/0xcd kernel/locking/spinlock.c:159 test_clear_page_writeback+0x1da/0x11b0 mm/page-writeback.c:2728 end_page_writeback+0x244/0x530 mm/filemap.c:1339 end_buffer_async_write+0x679/0x980 fs/buffer.c:349 end_bio_bh_io_sync+0xed/0x140 fs/buffer.c:3015 bio_endio+0x609/0xaf0 block/bio.c:1818 req_bio_endio block/blk-core.c:271 [inline] blk_update_request+0x49e/0x10d0 block/blk-core.c:1491 scsi_end_request+0x7f/0x830 drivers/scsi/scsi_lib.c:579 scsi_io_completion+0x20a/0x1420 drivers/scsi/scsi_lib.c:963 scsi_finish_command+0x3b7/0x670 drivers/scsi/scsi.c:228 scsi_softirq_done+0x326/0x3b0 drivers/scsi/scsi_lib.c:1477 blk_done_softirq+0x2fe/0x4d0 block/blk-softirq.c:37 __do_softirq+0x262/0x98c kernel/softirq.c:292 invoke_softirq kernel/softirq.c:373 [inline] irq_exit+0x19b/0x1e0 kernel/softirq.c:413 exiting_irq arch/x86/include/asm/apic.h:536 [inline] do_IRQ+0xe3/0x280 arch/x86/kernel/irq.c:263 ret_from_intr+0x0/0x36 native_safe_halt+0xe/0x10 arch/x86/include/asm/irqflags.h:60 arch_cpu_idle+0xa/0x10 arch/x86/kernel/process.c:571 default_idle_call+0x84/0xb0 kernel/sched/idle.c:94 cpuidle_idle_call kernel/sched/idle.c:154 [inline] do_idle+0x3b7/0x6e0 kernel/sched/idle.c:263 cpu_startup_entry+0x1b/0x20 kernel/sched/idle.c:355 start_secondary+0x2f4/0x410 arch/x86/kernel/smpboot.c:264 secondary_startup_64+0xa4/0xb0 arch/x86/kernel/head_64.S:242 to a SOFTIRQ-irq-unsafe lock: (&sb->s_type->i_lock_key#23){+.+.} ... which became SOFTIRQ-irq-unsafe at: ... lock_acquire+0x190/0x410 kernel/locking/lockdep.c:4487 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline] _raw_spin_lock+0x2f/0x40 kernel/locking/spinlock.c:151 spin_lock include/linux/spinlock.h:338 [inline] iget_locked+0x320/0x4b0 fs/inode.c:1193 __ext4_iget+0x265/0x3e20 fs/ext4/inode.c:4835 ext4_fill_super+0x7b11/0xcdb0 fs/ext4/super.c:4489 mount_bdev+0x304/0x3c0 fs/super.c:1415 ext4_mount+0x35/0x40 fs/ext4/super.c:6039 legacy_get_tree+0x108/0x220 fs/fs_context.c:647 vfs_get_tree+0x8e/0x300 fs/super.c:1545 do_new_mount fs/namespace.c:2822 [inline] do_mount+0x135a/0x1b50 fs/namespace.c:3142 ksys_mount+0xdb/0x150 fs/namespace.c:3351 do_mount_root+0x35/0x1d3 init/do_mounts.c:393 mount_block_root+0x353/0x61d init/do_mounts.c:422 mount_root+0x283/0x2cd init/do_mounts.c:612 prepare_namespace+0x26f/0x2ae init/do_mounts.c:671 kernel_init_freeable+0x5a0/0x5b9 init/main.c:1210 kernel_init+0x12/0x1bf init/main.c:1109 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(&sb->s_type->i_lock_key#23); local_irq_disable(); lock(&(&xa->xa_lock)->rlock#4); lock(&sb->s_type->i_lock_key#23); lock(&(&xa->xa_lock)->rlock#4); *** DEADLOCK *** 1 lock held by khugepaged/1083: #0: ffff888090de7ab0 (&(&xa->xa_lock)->rlock#4){..-.}, at: spin_lock_irq include/linux/spinlock.h:363 [inline] #0: ffff888090de7ab0 (&(&xa->xa_lock)->rlock#4){..-.}, at: collapse_file+0x24d/0x4580 mm/khugepaged.c:1524 the dependencies between SOFTIRQ-irq-safe lock and the holding lock: -> (&(&xa->xa_lock)->rlock#4){..-.} { IN-SOFTIRQ-W at: lock_acquire+0x190/0x410 kernel/locking/lockdep.c:4487 __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline] _raw_spin_lock_irqsave+0x95/0xcd kernel/locking/spinlock.c:159 test_clear_page_writeback+0x1da/0x11b0 mm/page-writeback.c:2728 end_page_writeback+0x244/0x530 mm/filemap.c:1339 end_buffer_async_write+0x679/0x980 fs/buffer.c:349 end_bio_bh_io_sync+0xed/0x140 fs/buffer.c:3015 bio_endio+0x609/0xaf0 block/bio.c:1818 req_bio_endio block/blk-core.c:271 [inline] blk_update_request+0x49e/0x10d0 block/blk-core.c:1491 scsi_end_request+0x7f/0x830 drivers/scsi/scsi_lib.c:579 scsi_io_completion+0x20a/0x1420 drivers/scsi/scsi_lib.c:963 scsi_finish_command+0x3b7/0x670 drivers/scsi/scsi.c:228 scsi_softirq_done+0x326/0x3b0 drivers/scsi/scsi_lib.c:1477 blk_done_softirq+0x2fe/0x4d0 block/blk-softirq.c:37 __do_softirq+0x262/0x98c kernel/softirq.c:292 invoke_softirq kernel/softirq.c:373 [inline] irq_exit+0x19b/0x1e0 kernel/softirq.c:413 exiting_irq arch/x86/include/asm/apic.h:536 [inline] do_IRQ+0xe3/0x280 arch/x86/kernel/irq.c:263 ret_from_intr+0x0/0x36 native_safe_halt+0xe/0x10 arch/x86/include/asm/irqflags.h:60 arch_cpu_idle+0xa/0x10 arch/x86/kernel/process.c:571 default_idle_call+0x84/0xb0 kernel/sched/idle.c:94 cpuidle_idle_call kernel/sched/idle.c:154 [inline] do_idle+0x3b7/0x6e0 kernel/sched/idle.c:263 cpu_startup_entry+0x1b/0x20 kernel/sched/idle.c:355 start_secondary+0x2f4/0x410 arch/x86/kernel/smpboot.c:264 secondary_startup_64+0xa4/0xb0 arch/x86/kernel/head_64.S:242 INITIAL USE at: lock_acquire+0x190/0x410 kernel/locking/lockdep.c:4487 __raw_spin_lock_irq include/linux/spinlock_api_smp.h:128 [inline] _raw_spin_lock_irq+0x60/0x80 kernel/locking/spinlock.c:167 spin_lock_irq include/linux/spinlock.h:363 [inline] __add_to_page_cache_locked+0x677/0xec0 mm/filemap.c:877 add_to_page_cache_lru+0x1d8/0x790 mm/filemap.c:943 do_read_cache_page+0x9fc/0x2140 mm/filemap.c:2770 read_cache_page+0x5e/0x70 mm/filemap.c:2874 read_mapping_page include/linux/pagemap.h:396 [inline] read_dev_sector+0x71/0x310 block/partition-generic.c:667 read_part_sector block/partitions/check.h:38 [inline] adfspart_check_ICS+0x12d/0xc90 block/partitions/acorn.c:361 check_partition+0x3bc/0x6ce block/partitions/check.c:167 rescan_partitions+0x230/0xa30 block/partition-generic.c:531 __blkdev_get+0xbae/0x1600 fs/block_dev.c:1599 blkdev_get+0x47/0x2c0 fs/block_dev.c:1707 register_disk block/genhd.c:655 [inline] __device_add_disk+0xabf/0x1230 block/genhd.c:745 device_add_disk+0x2b/0x40 block/genhd.c:763 add_disk include/linux/genhd.h:429 [inline] brd_init+0x237/0x41c drivers/block/brd.c:514 do_one_initcall+0x120/0x81a init/main.c:938 do_initcall_level init/main.c:1006 [inline] do_initcalls init/main.c:1014 [inline] do_basic_setup init/main.c:1031 [inline] kernel_init_freeable+0x4ca/0x5b9 init/main.c:1191 kernel_init+0x12/0x1bf init/main.c:1109 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352 } ... key at: [] __key.18618+0x0/0x40 ... acquired at: lock_acquire+0x190/0x410 kernel/locking/lockdep.c:4487 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline] _raw_spin_lock+0x2f/0x40 kernel/locking/spinlock.c:151 spin_lock include/linux/spinlock.h:338 [inline] wbc_attach_fdatawrite_inode include/linux/writeback.h:266 [inline] __filemap_fdatawrite_range+0x26e/0x3b0 mm/filemap.c:420 __filemap_fdatawrite mm/filemap.c:429 [inline] filemap_flush+0x24/0x30 mm/filemap.c:456 collapse_file+0x3b28/0x4580 mm/khugepaged.c:1609 khugepaged_scan_file mm/khugepaged.c:1890 [inline] khugepaged_scan_mm_slot mm/khugepaged.c:1988 [inline] khugepaged_do_scan mm/khugepaged.c:2072 [inline] khugepaged+0x2da9/0x4360 mm/khugepaged.c:2117 kthread+0x361/0x430 kernel/kthread.c:255 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352 the dependencies between the lock to be acquired and SOFTIRQ-irq-unsafe lock: -> (&sb->s_type->i_lock_key#23){+.+.} { HARDIRQ-ON-W at: lock_acquire+0x190/0x410 kernel/locking/lockdep.c:4487 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline] _raw_spin_lock+0x2f/0x40 kernel/locking/spinlock.c:151 spin_lock include/linux/spinlock.h:338 [inline] iget_locked+0x320/0x4b0 fs/inode.c:1193 __ext4_iget+0x265/0x3e20 fs/ext4/inode.c:4835 ext4_fill_super+0x7b11/0xcdb0 fs/ext4/super.c:4489 mount_bdev+0x304/0x3c0 fs/super.c:1415 ext4_mount+0x35/0x40 fs/ext4/super.c:6039 legacy_get_tree+0x108/0x220 fs/fs_context.c:647 vfs_get_tree+0x8e/0x300 fs/super.c:1545 do_new_mount fs/namespace.c:2822 [inline] do_mount+0x135a/0x1b50 fs/namespace.c:3142 ksys_mount+0xdb/0x150 fs/namespace.c:3351 do_mount_root+0x35/0x1d3 init/do_mounts.c:393 mount_block_root+0x353/0x61d init/do_mounts.c:422 mount_root+0x283/0x2cd init/do_mounts.c:612 prepare_namespace+0x26f/0x2ae init/do_mounts.c:671 kernel_init_freeable+0x5a0/0x5b9 init/main.c:1210 kernel_init+0x12/0x1bf init/main.c:1109 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352 SOFTIRQ-ON-W at: lock_acquire+0x190/0x410 kernel/locking/lockdep.c:4487 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline] _raw_spin_lock+0x2f/0x40 kernel/locking/spinlock.c:151 spin_lock include/linux/spinlock.h:338 [inline] iget_locked+0x320/0x4b0 fs/inode.c:1193 __ext4_iget+0x265/0x3e20 fs/ext4/inode.c:4835 ext4_fill_super+0x7b11/0xcdb0 fs/ext4/super.c:4489 mount_bdev+0x304/0x3c0 fs/super.c:1415 ext4_mount+0x35/0x40 fs/ext4/super.c:6039 legacy_get_tree+0x108/0x220 fs/fs_context.c:647 vfs_get_tree+0x8e/0x300 fs/super.c:1545 do_new_mount fs/namespace.c:2822 [inline] do_mount+0x135a/0x1b50 fs/namespace.c:3142 ksys_mount+0xdb/0x150 fs/namespace.c:3351 do_mount_root+0x35/0x1d3 init/do_mounts.c:393 mount_block_root+0x353/0x61d init/do_mounts.c:422 mount_root+0x283/0x2cd init/do_mounts.c:612 prepare_namespace+0x26f/0x2ae init/do_mounts.c:671 kernel_init_freeable+0x5a0/0x5b9 init/main.c:1210 kernel_init+0x12/0x1bf init/main.c:1109 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352 INITIAL USE at: lock_acquire+0x190/0x410 kernel/locking/lockdep.c:4487 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline] _raw_spin_lock+0x2f/0x40 kernel/locking/spinlock.c:151 spin_lock include/linux/spinlock.h:338 [inline] iget_locked+0x320/0x4b0 fs/inode.c:1193 __ext4_iget+0x265/0x3e20 fs/ext4/inode.c:4835 ext4_fill_super+0x7b11/0xcdb0 fs/ext4/super.c:4489 mount_bdev+0x304/0x3c0 fs/super.c:1415 ext4_mount+0x35/0x40 fs/ext4/super.c:6039 legacy_get_tree+0x108/0x220 fs/fs_context.c:647 vfs_get_tree+0x8e/0x300 fs/super.c:1545 do_new_mount fs/namespace.c:2822 [inline] do_mount+0x135a/0x1b50 fs/namespace.c:3142 ksys_mount+0xdb/0x150 fs/namespace.c:3351 do_mount_root+0x35/0x1d3 init/do_mounts.c:393 mount_block_root+0x353/0x61d init/do_mounts.c:422 mount_root+0x283/0x2cd init/do_mounts.c:612 prepare_namespace+0x26f/0x2ae init/do_mounts.c:671 kernel_init_freeable+0x5a0/0x5b9 init/main.c:1210 kernel_init+0x12/0x1bf init/main.c:1109 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352 } ... key at: [] ext4_fs_type+0xa8/0x100 ... acquired at: lock_acquire+0x190/0x410 kernel/locking/lockdep.c:4487 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline] _raw_spin_lock+0x2f/0x40 kernel/locking/spinlock.c:151 spin_lock include/linux/spinlock.h:338 [inline] wbc_attach_fdatawrite_inode include/linux/writeback.h:266 [inline] __filemap_fdatawrite_range+0x26e/0x3b0 mm/filemap.c:420 __filemap_fdatawrite mm/filemap.c:429 [inline] filemap_flush+0x24/0x30 mm/filemap.c:456 collapse_file+0x3b28/0x4580 mm/khugepaged.c:1609 khugepaged_scan_file mm/khugepaged.c:1890 [inline] khugepaged_scan_mm_slot mm/khugepaged.c:1988 [inline] khugepaged_do_scan mm/khugepaged.c:2072 [inline] khugepaged+0x2da9/0x4360 mm/khugepaged.c:2117 kthread+0x361/0x430 kernel/kthread.c:255 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352 stack backtrace: CPU: 0 PID: 1083 Comm: khugepaged Not tainted 5.4.0-rc5-next-20191028+ #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x172/0x1f0 lib/dump_stack.c:113 print_bad_irq_dependency kernel/locking/lockdep.c:2095 [inline] check_irq_usage.cold+0x586/0x6fe kernel/locking/lockdep.c:2293 check_prev_add kernel/locking/lockdep.c:2480 [inline] check_prevs_add kernel/locking/lockdep.c:2581 [inline] validate_chain kernel/locking/lockdep.c:2971 [inline] __lock_acquire+0x25b4/0x4a00 kernel/locking/lockdep.c:3955 lock_acquire+0x190/0x410 kernel/locking/lockdep.c:4487 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline] _raw_spin_lock+0x2f/0x40 kernel/locking/spinlock.c:151 spin_lock include/linux/spinlock.h:338 [inline] wbc_attach_fdatawrite_inode include/linux/writeback.h:266 [inline] __filemap_fdatawrite_range+0x26e/0x3b0 mm/filemap.c:420 __filemap_fdatawrite mm/filemap.c:429 [inline] filemap_flush+0x24/0x30 mm/filemap.c:456 collapse_file+0x3b28/0x4580 mm/khugepaged.c:1609 khugepaged_scan_file mm/khugepaged.c:1890 [inline] khugepaged_scan_mm_slot mm/khugepaged.c:1988 [inline] khugepaged_do_scan mm/khugepaged.c:2072 [inline] khugepaged+0x2da9/0x4360 mm/khugepaged.c:2117 kthread+0x361/0x430 kernel/kthread.c:255 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352 BUG: sleeping function called from invalid context at include/linux/percpu-rwsem.h:38 in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 1083, name: khugepaged INFO: lockdep is turned off. irq event stamp: 58 hardirqs last enabled at (57): [] rmqueue mm/page_alloc.c:3304 [inline] hardirqs last enabled at (57): [] get_page_from_freelist+0x3437/0x4330 mm/page_alloc.c:3692 hardirqs last disabled at (58): [] __raw_spin_lock_irq include/linux/spinlock_api_smp.h:126 [inline] hardirqs last disabled at (58): [] _raw_spin_lock_irq+0x3a/0x80 kernel/locking/spinlock.c:167 softirqs last enabled at (0): [] copy_process+0x1822/0x6880 kernel/fork.c:2019 softirqs last disabled at (0): [<0000000000000000>] 0x0 Preemption disabled at: [] spin_lock_irq include/linux/spinlock.h:363 [inline] [] collapse_file+0x24d/0x4580 mm/khugepaged.c:1524 CPU: 0 PID: 1083 Comm: khugepaged Not tainted 5.4.0-rc5-next-20191028+ #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x172/0x1f0 lib/dump_stack.c:113 ___might_sleep.cold+0x1fb/0x23e kernel/sched/core.c:6788 __might_sleep+0x95/0x190 kernel/sched/core.c:6741 percpu_down_read include/linux/percpu-rwsem.h:38 [inline] ext4_writepages+0x1cb/0x2e70 fs/ext4/inode.c:2728 do_writepages+0xfa/0x2a0 mm/page-writeback.c:2344 __filemap_fdatawrite_range+0x2bc/0x3b0 mm/filemap.c:421 __filemap_fdatawrite mm/filemap.c:429 [inline] filemap_flush+0x24/0x30 mm/filemap.c:456 collapse_file+0x3b28/0x4580 mm/khugepaged.c:1609 khugepaged_scan_file mm/khugepaged.c:1890 [inline] khugepaged_scan_mm_slot mm/khugepaged.c:1988 [inline] khugepaged_do_scan mm/khugepaged.c:2072 [inline] khugepaged+0x2da9/0x4360 mm/khugepaged.c:2117 kthread+0x361/0x430 kernel/kthread.c:255 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352