syzbot

 sign-in | mailing list | source | docs
🐞 Open [712] 🐞 Fixed [297] 🐞 Invalid [838] 📈 Kernel Health 📈 Bug Lifetimes 📈 Fuzzing 📈 Crashes 💬 Send us feedback

possible deadlock in red_adaptative_timer

Status: upstream: reported C repro on 2020/07/28 13:48
Reported-by: syzbot+9f57216a517eb79b590e@syzkaller.appspotmail.com
First crash: 1361d, last: 1192d
Fix bisection: failed (error log, bisect log)
  
▶ ▼ Similar bugs (2)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.14 possible deadlock in red_adaptative_timer C done 2 1124d 1215d 1/1 fixed on 2021/04/21 07:52
linux-6.1 possible deadlock in red_adaptative_timer 1 248d 248d 0/3 auto-obsoleted due to no activity on 2023/11/23 10:12

Sample crash report:
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
======================================================
WARNING: possible circular locking dependency detected
4.19.167-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor237/10868 is trying to acquire lock:
00000000d5244787 (&qdisc_rx_lock){+.-.}, at: spin_lock include/linux/spinlock.h:329 [inline]
00000000d5244787 (&qdisc_rx_lock){+.-.}, at: red_adaptative_timer+0x92/0x5c0 net/sched/sch_red.c:265

but task is already holding lock:
000000005a88ab3f ((&q->adapt_timer)){+.-.}, at: lockdep_copy_map include/linux/lockdep.h:168 [inline]
000000005a88ab3f ((&q->adapt_timer)){+.-.}, at: call_timer_fn+0xc9/0x700 kernel/time/timer.c:1328

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 ((&q->adapt_timer)){+.-.}:
       red_destroy+0x15/0x60 net/sched/sch_red.c:182
       qdisc_destroy+0x180/0x790 net/sched/sch_generic.c:983
       red_change+0x588/0x10b0 net/sched/sch_red.c:236
       qdisc_change net/sched/sch_api.c:1239 [inline]
       tc_modify_qdisc+0xf6a/0x1a80 net/sched/sch_api.c:1542
       rtnetlink_rcv_msg+0x453/0xb80 net/core/rtnetlink.c:4778
       netlink_rcv_skb+0x160/0x440 net/netlink/af_netlink.c:2455
       netlink_unicast_kernel net/netlink/af_netlink.c:1318 [inline]
       netlink_unicast+0x4d5/0x690 net/netlink/af_netlink.c:1344
       netlink_sendmsg+0x6bb/0xc40 net/netlink/af_netlink.c:1909
       sock_sendmsg_nosec net/socket.c:622 [inline]
       sock_sendmsg+0xc3/0x120 net/socket.c:632
       ___sys_sendmsg+0x7bb/0x8e0 net/socket.c:2115
       __sys_sendmsg net/socket.c:2153 [inline]
       __do_sys_sendmsg net/socket.c:2162 [inline]
       __se_sys_sendmsg net/socket.c:2160 [inline]
       __x64_sys_sendmsg+0x132/0x220 net/socket.c:2160
       do_syscall_64+0xf9/0x620 arch/x86/entry/common.c:293
       entry_SYSCALL_64_after_hwframe+0x49/0xbe

-> #0 (&qdisc_rx_lock){+.-.}:
       __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
       _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
       spin_lock include/linux/spinlock.h:329 [inline]
       red_adaptative_timer+0x92/0x5c0 net/sched/sch_red.c:265
       call_timer_fn+0x177/0x700 kernel/time/timer.c:1338
       expire_timers+0x243/0x4e0 kernel/time/timer.c:1375
       __run_timers kernel/time/timer.c:1696 [inline]
       run_timer_softirq+0x21c/0x670 kernel/time/timer.c:1709
       __do_softirq+0x26c/0x9a0 kernel/softirq.c:292
       invoke_softirq kernel/softirq.c:372 [inline]
       irq_exit+0x215/0x260 kernel/softirq.c:412
       exiting_irq arch/x86/include/asm/apic.h:545 [inline]
       smp_apic_timer_interrupt+0x136/0x550 arch/x86/kernel/apic/apic.c:1094
       apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:894
       arch_local_irq_restore arch/x86/include/asm/paravirt.h:789 [inline]
       console_unlock+0xe7b/0x1110 kernel/printk/printk.c:2468
       vprintk_emit+0x2d1/0x740 kernel/printk/printk.c:1965
       vprintk_func+0x79/0x17e kernel/printk/printk_safe.c:397
       printk+0xba/0xed kernel/printk/printk.c:2040
       __ip_vs_ftp_init.cold+0x5b/0x6a net/netfilter/ipvs/ip_vs_ftp.c:600
       ops_init+0xb3/0x410 net/core/net_namespace.c:129
       setup_net+0x2c2/0x720 net/core/net_namespace.c:315
       copy_net_ns+0x1f7/0x335 net/core/net_namespace.c:438
       create_new_namespaces+0x3f6/0x7b0 kernel/nsproxy.c:107
       copy_namespaces+0x325/0x3c0 kernel/nsproxy.c:165
       copy_process.part.0+0x3a59/0x8200 kernel/fork.c:1919
       copy_process kernel/fork.c:1713 [inline]
       _do_fork+0x22f/0xf30 kernel/fork.c:2228
       do_syscall_64+0xf9/0x620 arch/x86/entry/common.c:293
       entry_SYSCALL_64_after_hwframe+0x49/0xbe

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock((&q->adapt_timer));
                               lock(&qdisc_rx_lock);
                               lock((&q->adapt_timer));
  lock(&qdisc_rx_lock);

 *** DEADLOCK ***

3 locks held by syz-executor237/10868:
 #0: 0000000050f5def9 (pernet_ops_rwsem){++++}, at: copy_net_ns+0x1d8/0x335 net/core/net_namespace.c:434
 #1: 0000000012e72b14 (console_lock){+.+.}, at: vprintk_func+0x79/0x17e kernel/printk/printk_safe.c:397
 #2: 000000005a88ab3f ((&q->adapt_timer)){+.-.}, at: lockdep_copy_map include/linux/lockdep.h:168 [inline]
 #2: 000000005a88ab3f ((&q->adapt_timer)){+.-.}, at: call_timer_fn+0xc9/0x700 kernel/time/timer.c:1328

stack backtrace:
CPU: 1 PID: 10868 Comm: syz-executor237 Not tainted 4.19.167-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x1fc/0x2fe lib/dump_stack.c:118
 print_circular_bug.constprop.0.cold+0x2d7/0x41e kernel/locking/lockdep.c:1221
 check_prev_add kernel/locking/lockdep.c:1865 [inline]
 check_prevs_add kernel/locking/lockdep.c:1978 [inline]
 validate_chain kernel/locking/lockdep.c:2419 [inline]
 __lock_acquire+0x30c9/0x3ff0 kernel/locking/lockdep.c:3415
 lock_acquire+0x170/0x3c0 kernel/locking/lockdep.c:3907
 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
 _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:144
 spin_lock include/linux/spinlock.h:329 [inline]
 red_adaptative_timer+0x92/0x5c0 net/sched/sch_red.c:265
 call_timer_fn+0x177/0x700 kernel/time/timer.c:1338
 expire_timers+0x243/0x4e0 kernel/time/timer.c:1375
 __run_timers kernel/time/timer.c:1696 [inline]
 run_timer_softirq+0x21c/0x670 kernel/time/timer.c:1709
 __do_softirq+0x26c/0x9a0 kernel/softirq.c:292
 invoke_softirq kernel/softirq.c:372 [inline]
 irq_exit+0x215/0x260 kernel/softirq.c:412
 exiting_irq arch/x86/include/asm/apic.h:545 [inline]
 smp_apic_timer_interrupt+0x136/0x550 arch/x86/kernel/apic/apic.c:1094
 apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:894
 </IRQ>
RIP: 0010:arch_local_irq_restore arch/x86/include/asm/paravirt.h:789 [inline]
RIP: 0010:console_unlock+0xe7b/0x1110 kernel/printk/printk.c:2468
Code: ff df 48 c1 e8 03 80 3c 08 00 0f 85 66 02 00 00 48 83 3d df eb a1 08 00 0f 84 9e 00 00 00 e8 4c 3c 15 00 48 8b 7c 24 30 57 9d <0f> 1f 44 00 00 e9 9b fc ff ff e8 36 3c 15 00 0f 0b e8 2f 3c 15 00
RSP: 0018:ffff8880849cf7f8 EFLAGS: 00000293 ORIG_RAX: ffffffffffffff13
RAX: ffff88809f3a6240 RBX: 0000000000000200 RCX: dffffc0000000000
RDX: 0000000000000000 RSI: ffffffff814f95b4 RDI: 0000000000000293
RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff84546460
R13: 0000000000000039 R14: 0000000000000000 R15: ffffffff8a6da0b0
 vprintk_emit+0x2d1/0x740 kernel/printk/printk.c:1965
 vprintk_func+0x79/0x17e kernel/printk/printk_safe.c:397
 printk+0xba/0xed kernel/printk/printk.c:2040
 __ip_vs_ftp_init.cold+0x5b/0x6a net/netfilter/ipvs/ip_vs_ftp.c:600
 ops_init+0xb3/0x410 net/core/net_namespace.c:129
 setup_net+0x2c2/0x720 net/core/net_namespace.c:315
 copy_net_ns+0x1f7/0x335 net/core/net_namespace.c:438
 create_new_namespaces+0x3f6/0x7b0 kernel/nsproxy.c:107
 copy_namespaces+0x325/0x3c0 kernel/nsproxy.c:165
 copy_process.part.0+0x3a59/0x8200 kernel/fork.c:1919
 copy_process kernel/fork.c:1713 [inline]
 _do_fork+0x22f/0xf30 kernel/fork.c:2228
 do_syscall_64+0xf9/0x620 arch/x86/entry/common.c:293
 entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x443689
Code: e8 0c e8 ff ff 48 83 c4 18 c3 0f 1f 80 00 00 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 cb 06 fc ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007ffc89eed828 EFLAGS: 00000246 ORIG_RAX: 0000000000000038
RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 0000000000443689
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000070224100
RBP: 00007ffc89eed860 R08: 0000000000000000 R09: 0000000000000004
R10: 0000000000000000 R11: 0000000000000246 R12: 000000000000a61d
R13: 0000000000000000 R14: 000000000000000c R15: 0000000000000004
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21
IPVS: ftp: loaded support on port[0] = 21

Crashes (10):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2021/01/13 04:23 linux-4.19.y 675cc038067f 0cdd6185 .config console log report syz C ci2-linux-4-19
2020/12/16 06:43 linux-4.19.y 13d2ce42de8c f213e07e .config console log report syz ci2-linux-4-19
2020/11/11 16:09 linux-4.19.y 31acccdc8774 cca87986 .config console log report syz ci2-linux-4-19
2020/09/04 09:48 linux-4.19.y c37da90efff5 abf9ba4f .config console log report ci2-linux-4-19
2020/09/01 03:41 linux-4.19.y f6d5cb9e2c06 d5a3ae1f .config console log report ci2-linux-4-19
2020/08/26 06:25 linux-4.19.y d18b78abc0c6 344da168 .config console log report ci2-linux-4-19
2020/08/20 08:59 linux-4.19.y a834132bd465 ed282a3a .config console log report ci2-linux-4-19
2020/08/16 14:26 linux-4.19.y c14d30dc9987 5ce13532 .config console log report ci2-linux-4-19
2020/08/10 18:41 linux-4.19.y 961f830af065 7adc7b65 .config console log report ci2-linux-4-19
2020/07/28 13:48 linux-4.19.y 20b3a3dfdf6c cb93dc6a .config console log report ci2-linux-4-19
* Struck through repros no longer work on HEAD.