syzbot


possible deadlock in pie_timer

Status: upstream: reported C repro on 2022/02/03 09:38
Reported-by: syzbot+de0e3bbaa3f13a456d32@syzkaller.appspotmail.com
First crash: 813d, last: 813d
Fix bisection: failed (error log, bisect log)
  
Similar bugs (4)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream possible deadlock in pie_timer (2) net 2 308d 308d 23/26 fixed on 2023/09/28 17:51
upstream possible deadlock in pie_timer net C done 2 1671d 1672d 13/26 fixed on 2019/10/15 23:40
linux-6.1 possible deadlock in pie_timer 1 268d 268d 0/3 auto-obsoleted due to no activity on 2023/11/10 02:26
linux-4.14 possible deadlock in pie_timer C inconclusive 3 802d 1663d 0/1 upstream: reported C repro on 2019/10/06 16:17

Sample crash report:
======================================================
WARNING: possible circular locking dependency detected
4.19.211-syzkaller #0 Not tainted
------------------------------------------------------
swapper/1/0 is trying to acquire lock:
00000000eeabeb81 (&qdisc_rx_lock){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline]
00000000eeabeb81 (&qdisc_rx_lock){+.-.}, at: pie_timer+0x92/0x740 net/sched/sch_pie.c:433

but task is already holding lock:
000000006bcb8190 ((&q->adapt_timer)){+.-.}, at: lockdep_copy_map include/linux/lockdep.h:168 [inline]
000000006bcb8190 ((&q->adapt_timer)){+.-.}, at: call_timer_fn+0xc9/0x700 kernel/time/timer.c:1328

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 ((&q->adapt_timer)){+.-.}:
       qdisc_destroy+0x180/0x790 net/sched/sch_generic.c:983
       sfb_change+0x2c9/0xb30 net/sched/sch_sfb.c:527
       qdisc_change net/sched/sch_api.c:1240 [inline]
       tc_modify_qdisc+0xf6a/0x1a80 net/sched/sch_api.c:1543
       rtnetlink_rcv_msg+0x453/0xb80 net/core/rtnetlink.c:4782
       netlink_rcv_skb+0x160/0x440 net/netlink/af_netlink.c:2463
       netlink_unicast_kernel net/netlink/af_netlink.c:1325 [inline]
       netlink_unicast+0x4d5/0x690 net/netlink/af_netlink.c:1351
       netlink_sendmsg+0x6c3/0xc50 net/netlink/af_netlink.c:1917
       sock_sendmsg_nosec net/socket.c:651 [inline]
       sock_sendmsg+0xc3/0x120 net/socket.c:661
       ___sys_sendmsg+0x7bb/0x8e0 net/socket.c:2227
       __sys_sendmsg net/socket.c:2265 [inline]
       __do_sys_sendmsg net/socket.c:2274 [inline]
       __se_sys_sendmsg net/socket.c:2272 [inline]
       __x64_sys_sendmsg+0x132/0x220 net/socket.c:2272
       do_syscall_64+0xf9/0x620 arch/x86/entry/common.c:293
       entry_SYSCALL_64_after_hwframe+0x49/0xbe

-> #0 (&qdisc_rx_lock){+.-.}:
       __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
       _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
       spin_lock include/linux/spinlock.h:329 [inline]
       pie_timer+0x92/0x740 net/sched/sch_pie.c:433
       call_timer_fn+0x177/0x700 kernel/time/timer.c:1338
       expire_timers+0x243/0x4e0 kernel/time/timer.c:1375
       __run_timers kernel/time/timer.c:1696 [inline]
       run_timer_softirq+0x21c/0x670 kernel/time/timer.c:1709
       __do_softirq+0x265/0x980 kernel/softirq.c:292
       invoke_softirq kernel/softirq.c:372 [inline]
       irq_exit+0x215/0x260 kernel/softirq.c:412
       exiting_irq arch/x86/include/asm/apic.h:536 [inline]
       smp_apic_timer_interrupt+0x136/0x550 arch/x86/kernel/apic/apic.c:1098
       apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:894
       native_safe_halt+0xe/0x10 arch/x86/include/asm/irqflags.h:60
       arch_safe_halt arch/x86/include/asm/paravirt.h:94 [inline]
       default_idle+0x49/0x310 arch/x86/kernel/process.c:557
       cpuidle_idle_call kernel/sched/idle.c:153 [inline]
       do_idle+0x2ec/0x4b0 kernel/sched/idle.c:263
       cpu_startup_entry+0xc5/0xe0 kernel/sched/idle.c:369
       start_secondary+0x435/0x5c0 arch/x86/kernel/smpboot.c:271
       secondary_startup_64+0xa4/0xb0 arch/x86/kernel/head_64.S:243

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock((&q->adapt_timer));
                               lock(&qdisc_rx_lock);
                               lock((&q->adapt_timer));
  lock(&qdisc_rx_lock);

 *** DEADLOCK ***

1 lock held by swapper/1/0:
 #0: 000000006bcb8190 ((&q->adapt_timer)){+.-.}, at: lockdep_copy_map include/linux/lockdep.h:168 [inline]
 #0: 000000006bcb8190 ((&q->adapt_timer)){+.-.}, at: call_timer_fn+0xc9/0x700 kernel/time/timer.c:1328

stack backtrace:
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.19.211-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x1fc/0x2ef lib/dump_stack.c:118
 print_circular_bug.constprop.0.cold+0x2d7/0x41e kernel/locking/lockdep.c:1222
 check_prev_add kernel/locking/lockdep.c:1866 [inline]
 check_prevs_add kernel/locking/lockdep.c:1979 [inline]
 validate_chain kernel/locking/lockdep.c:2420 [inline]
 __lock_acquire+0x30c9/0x3ff0 kernel/locking/lockdep.c:3416
 lock_acquire+0x170/0x3c0 kernel/locking/lockdep.c:3908
 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
 _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
 spin_lock include/linux/spinlock.h:329 [inline]
 pie_timer+0x92/0x740 net/sched/sch_pie.c:433
 call_timer_fn+0x177/0x700 kernel/time/timer.c:1338
 expire_timers+0x243/0x4e0 kernel/time/timer.c:1375
 __run_timers kernel/time/timer.c:1696 [inline]
 run_timer_softirq+0x21c/0x670 kernel/time/timer.c:1709
 __do_softirq+0x265/0x980 kernel/softirq.c:292
 invoke_softirq kernel/softirq.c:372 [inline]
 irq_exit+0x215/0x260 kernel/softirq.c:412
 exiting_irq arch/x86/include/asm/apic.h:536 [inline]
 smp_apic_timer_interrupt+0x136/0x550 arch/x86/kernel/apic/apic.c:1098
 apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:894
 </IRQ>
RIP: 0010:native_safe_halt+0xe/0x10 arch/x86/include/asm/irqflags.h:61
Code: 48 89 df e8 f4 20 7f f9 e9 2e ff ff ff 48 89 df e8 e7 20 7f f9 eb 82 90 90 90 90 90 e9 07 00 00 00 0f 00 2d 14 43 4e 00 fb f4 <c3> 90 e9 07 00 00 00 0f 00 2d 04 43 4e 00 f4 c3 90 90 41 56 41 55
RSP: 0018:ffff8880b5a9fd40 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff13
RAX: 1ffffffff13e3054 RBX: dffffc0000000000 RCX: 0000000000000000
RDX: dffffc0000000000 RSI: 0000000000000001 RDI: ffff8880b5a86c44
RBP: 0000000000000001 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff89f18290
R13: 1ffff11016b53fb2 R14: 0000000000000000 R15: 0000000000000000
 arch_safe_halt arch/x86/include/asm/paravirt.h:94 [inline]
 default_idle+0x49/0x310 arch/x86/kernel/process.c:557
 cpuidle_idle_call kernel/sched/idle.c:153 [inline]
 do_idle+0x2ec/0x4b0 kernel/sched/idle.c:263
 cpu_startup_entry+0xc5/0xe0 kernel/sched/idle.c:369
 start_secondary+0x435/0x5c0 arch/x86/kernel/smpboot.c:271
 secondary_startup_64+0xa4/0xb0 arch/x86/kernel/head_64.S:243
----------------
Code disassembly (best guess):
   0:	48 89 df             	mov    %rbx,%rdi
   3:	e8 f4 20 7f f9       	callq  0xf97f20fc
   8:	e9 2e ff ff ff       	jmpq   0xffffff3b
   d:	48 89 df             	mov    %rbx,%rdi
  10:	e8 e7 20 7f f9       	callq  0xf97f20fc
  15:	eb 82                	jmp    0xffffff99
  17:	90                   	nop
  18:	90                   	nop
  19:	90                   	nop
  1a:	90                   	nop
  1b:	90                   	nop
  1c:	e9 07 00 00 00       	jmpq   0x28
  21:	0f 00 2d 14 43 4e 00 	verw   0x4e4314(%rip)        # 0x4e433c
  28:	fb                   	sti
  29:	f4                   	hlt
* 2a:	c3                   	retq <-- trapping instruction
  2b:	90                   	nop
  2c:	e9 07 00 00 00       	jmpq   0x38
  31:	0f 00 2d 04 43 4e 00 	verw   0x4e4304(%rip)        # 0x4e433c
  38:	f4                   	hlt
  39:	c3                   	retq
  3a:	90                   	nop
  3b:	90                   	nop
  3c:	41 56                	push   %r14
  3e:	41 55                	push   %r13

Crashes (1):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2022/02/03 09:38 linux-4.19.y 3f8a27f9e27b 4ebb2798 .config console log report syz C ci2-linux-4-19 possible deadlock in pie_timer
* Struck through repros no longer work on HEAD.