================================= [ INFO: inconsistent lock state ] 4.9.124-g09eb2ba #35 Not tainted --------------------------------- inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage. ksoftirqd/0/3 [HC0[0]:SC1[1]:HE1:SE0] takes: (&(&rnp->exp_lock)->rlock){+.?...}, at: [] spin_lock include/linux/spinlock.h:302 [inline] (&(&rnp->exp_lock)->rlock){+.?...}, at: [] exp_funnel_lock kernel/rcu/tree_exp.h:271 [inline] (&(&rnp->exp_lock)->rlock){+.?...}, at: [] _synchronize_rcu_expedited+0x1fb/0x840 kernel/rcu/tree_exp.h:569 mark_irqflags kernel/locking/lockdep.c:2941 [inline] __lock_acquire+0xdd0/0x4070 kernel/locking/lockdep.c:3302 lock_acquire+0x130/0x3e0 kernel/locking/lockdep.c:3756 __raw_spin_lock include/linux/spinlock_api_smp.h:144 [inline] _raw_spin_lock+0x36/0x50 kernel/locking/spinlock.c:151 spin_lock include/linux/spinlock.h:302 [inline] rcu_exp_wait_wake+0x2d2/0x5e0 kernel/rcu/tree_exp.h:504 rcu_exp_sel_wait_wake kernel/rcu/tree_exp.h:535 [inline] _synchronize_rcu_expedited+0x73c/0x840 kernel/rcu/tree_exp.h:575 synchronize_rcu_expedited kernel/rcu/tree_exp.h:687 [inline] synchronize_rcu.part.62+0xd3/0x110 kernel/rcu/tree_plugin.h:682 synchronize_rcu+0x27/0x90 kernel/rcu/tree_plugin.h:685 rcu_test_sync_prims+0x9/0x30 kernel/rcu/update.c:829 rcu_scheduler_starting+0x51/0x80 kernel/rcu/tree.c:4016 rest_init+0xf/0x189 init/main.c:388 start_kernel+0x67e/0x6b2 init/main.c:664 x86_64_start_reservations+0x29/0x2b arch/x86/kernel/head64.c:196 x86_64_start_kernel+0x13f/0x162 arch/x86/kernel/head64.c:177 irq event stamp: 9093782 hardirqs last enabled at (9093782): [] __local_bh_enable_ip+0x6a/0xd0 kernel/softirq.c:190 hardirqs last disabled at (9093781): [] __local_bh_enable_ip+0x32/0xd0 kernel/softirq.c:167 softirqs last enabled at (9093708): [] __do_softirq+0x475/0x940 kernel/softirq.c:314 softirqs last disabled at (9093713): [] run_ksoftirqd+0x30/0x60 kernel/softirq.c:680 other info that might help us debug this: Possible unsafe locking scenario: CPU0 ---- lock(&(&rnp->exp_lock)->rlock); lock(&(&rnp->exp_lock)->rlock); *** DEADLOCK *** 1 lock held by ksoftirqd/0/3: #0: (rcu_callback){......}, at: [] __rcu_reclaim kernel/rcu/rcu.h:108 [inline] #0: (rcu_callback){......}, at: [] rcu_do_batch kernel/rcu/tree.c:2789 [inline] #0: (rcu_callback){......}, at: [] invoke_rcu_callbacks kernel/rcu/tree.c:3053 [inline] #0: (rcu_callback){......}, at: [] __rcu_process_callbacks kernel/rcu/tree.c:3020 [inline] #0: (rcu_callback){......}, at: [] rcu_process_callbacks+0x98e/0x12b0 kernel/rcu/tree.c:3037 stack backtrace: CPU: 0 PID: 3 Comm: ksoftirqd/0 Not tainted 4.9.124-g09eb2ba #35 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 ffff8801d99ef498 ffffffff81eb95e9 ffff8801d99bb000 ffffffff85597020 ffff8801d99bb8f0 ffff8801d99bb910 0000000000000000 ffff8801d99ef508 ffffffff81429ccd 0000000000000001 0000000000000001 ffff880100000000 Call Trace: [] __dump_stack lib/dump_stack.c:15 [inline] [] dump_stack+0xc1/0x128 lib/dump_stack.c:51 [] print_usage_bug.cold.57+0x327/0x421 kernel/locking/lockdep.c:2387 [] valid_state kernel/locking/lockdep.c:2400 [inline] [] mark_lock_irq kernel/locking/lockdep.c:2602 [inline] [] mark_lock+0xcc6/0x1280 kernel/locking/lockdep.c:3065 [] mark_irqflags kernel/locking/lockdep.c:2923 [inline] [] __lock_acquire+0xd40/0x4070 kernel/locking/lockdep.c:3302 IPVS: Scheduler module ip_vs_sh: not found [] lock_acquire+0x130/0x3e0 kernel/locking/lockdep.c:3756 [] __raw_spin_lock include/linux/spinlock_api_smp.h:144 [inline] [] _raw_spin_lock+0x36/0x50 kernel/locking/spinlock.c:151 [] spin_lock include/linux/spinlock.h:302 [inline] [] exp_funnel_lock kernel/rcu/tree_exp.h:271 [inline] [] _synchronize_rcu_expedited+0x1fb/0x840 kernel/rcu/tree_exp.h:569 IPVS: Scheduler module ip_vs_sh: not found [] synchronize_rcu_expedited kernel/rcu/tree_exp.h:687 [inline] [] synchronize_rcu.part.62+0xd3/0x110 kernel/rcu/tree_plugin.h:682 [] synchronize_rcu+0x27/0x90 kernel/rcu/tree_plugin.h:685 [] __l2tp_session_unhash+0x38a/0x520 net/l2tp/l2tp_core.c:1796 [] l2tp_tunnel_closeall+0x1f7/0x350 net/l2tp/l2tp_core.c:1364 [] l2tp_tunnel_destruct+0x2f2/0x590 net/l2tp/l2tp_core.c:1324 [] __sk_destruct+0x55/0x590 net/core/sock.c:1428 [] __rcu_reclaim kernel/rcu/rcu.h:118 [inline] [] rcu_do_batch kernel/rcu/tree.c:2789 [inline] [] invoke_rcu_callbacks kernel/rcu/tree.c:3053 [inline] [] __rcu_process_callbacks kernel/rcu/tree.c:3020 [inline] [] rcu_process_callbacks+0x8ae/0x12b0 kernel/rcu/tree.c:3037 [] __do_softirq+0x210/0x940 kernel/softirq.c:288 [] run_ksoftirqd+0x30/0x60 kernel/softirq.c:680 [] smpboot_thread_fn+0x5c9/0x900 kernel/smpboot.c:163 [] kthread+0x26d/0x300 kernel/kthread.c:211 [] ret_from_fork+0x5c/0x70 arch/x86/entry/entry_64.S:373 BUG: sleeping function called from invalid context at kernel/locking/mutex.c:620 in_atomic(): 1, irqs_disabled(): 0, pid: 3, name: ksoftirqd/0 INFO: lockdep is turned off. Preemption disabled at:[ 478.806584] [] __do_softirq+0xdd/0x940 kernel/softirq.c:265 CPU: 0 PID: 3 Comm: ksoftirqd/0 Not tainted 4.9.124-g09eb2ba #35 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 ffff8801d99ef630 ffffffff81eb95e9 ffffffff83a0848d 0000000000000000 0000000000000100 ffff8801d99bb000 ffff8801d99bb000 ffff8801d99ef668 ffffffff81426851 ffff8801d99bb000 ffffffff83c5b880 000000000000026c Call Trace: [] __dump_stack lib/dump_stack.c:15 [inline] [] dump_stack+0xc1/0x128 lib/dump_stack.c:51 [] ___might_sleep.cold.123+0x1bc/0x1f5 kernel/sched/core.c:7988 [] __might_sleep+0x95/0x1a0 kernel/sched/core.c:7945 [] mutex_lock_nested+0x87/0x870 kernel/locking/mutex.c:620 IPVS: Scheduler module ip_vs_sh: not found [] exp_funnel_lock kernel/rcu/tree_exp.h:289 [inline] [] _synchronize_rcu_expedited+0x3a7/0x840 kernel/rcu/tree_exp.h:569 [] synchronize_rcu_expedited kernel/rcu/tree_exp.h:687 [inline] [] synchronize_rcu.part.62+0xd3/0x110 kernel/rcu/tree_plugin.h:682 [] synchronize_rcu+0x27/0x90 kernel/rcu/tree_plugin.h:685 [] __l2tp_session_unhash+0x38a/0x520 net/l2tp/l2tp_core.c:1796 [] l2tp_tunnel_closeall+0x1f7/0x350 net/l2tp/l2tp_core.c:1364 [] l2tp_tunnel_destruct+0x2f2/0x590 net/l2tp/l2tp_core.c:1324 [] __sk_destruct+0x55/0x590 net/core/sock.c:1428 [] __rcu_reclaim kernel/rcu/rcu.h:118 [inline] [] rcu_do_batch kernel/rcu/tree.c:2789 [inline] [] invoke_rcu_callbacks kernel/rcu/tree.c:3053 [inline] [] __rcu_process_callbacks kernel/rcu/tree.c:3020 [inline] [] rcu_process_callbacks+0x8ae/0x12b0 kernel/rcu/tree.c:3037 [] __do_softirq+0x210/0x940 kernel/softirq.c:288 [] run_ksoftirqd+0x30/0x60 kernel/softirq.c:680 [] smpboot_thread_fn+0x5c9/0x900 kernel/smpboot.c:163 [] kthread+0x26d/0x300 kernel/kthread.c:211 [] ret_from_fork+0x5c/0x70 arch/x86/entry/entry_64.S:373 BUG: scheduling while atomic: ksoftirqd/0/3/0x00000101 INFO: lockdep is turned off. Modules linked in: Preemption disabled at:[ 479.184275] [] __do_softirq+0xdd/0x940 kernel/softirq.c:265