bisecting fixing commit since 3207316b3beec7e38e5dbe2f463df0cec71e0b97 building syzkaller on 5cc121d679e3f161f29503eeba9288431b6d644d testing commit 3207316b3beec7e38e5dbe2f463df0cec71e0b97 with gcc (GCC) 8.4.1 20210217 kernel signature: 1c1de12fd969683bc89d5f194b825c68357d470b33783e5ec1f6b5e1c4481ee2 run #0: crashed: possible deadlock in red_adaptative_timer run #1: crashed: possible deadlock in red_adaptative_timer run #2: crashed: possible deadlock in red_change run #3: crashed: possible deadlock in red_change run #4: crashed: possible deadlock in red_change run #5: crashed: possible deadlock in red_change run #6: crashed: possible deadlock in red_change run #7: crashed: possible deadlock in red_change run #8: crashed: possible deadlock in red_change run #9: crashed: possible deadlock in red_change run #10: crashed: possible deadlock in red_change run #11: crashed: possible deadlock in red_change run #12: crashed: possible deadlock in red_change run #13: crashed: possible deadlock in red_change run #14: crashed: possible deadlock in red_change run #15: crashed: possible deadlock in red_change run #16: crashed: possible deadlock in red_change run #17: crashed: possible deadlock in red_change run #18: crashed: possible deadlock in red_change run #19: crashed: possible deadlock in red_change testing current HEAD 125222814e7b8f84df767d6ab622aff2a6d2f234 testing commit 125222814e7b8f84df767d6ab622aff2a6d2f234 with gcc (GCC) 8.4.1 20210217 kernel signature: b112cc0eff58a26d69349d3c79671bb3bcc2f9781871dc05b938c5bf8540a0ad run #0: crashed: possible deadlock in red_adaptative_timer run #1: crashed: possible deadlock in red_change run #2: crashed: possible deadlock in red_change run #3: crashed: possible deadlock in red_change run #4: crashed: possible deadlock in red_change run #5: crashed: possible deadlock in red_change run #6: crashed: possible deadlock in red_change run #7: crashed: possible deadlock in red_change run #8: crashed: possible deadlock in red_adaptative_timer run #9: crashed: possible deadlock in red_change revisions tested: 2, total time: 21m38.447502259s (build: 14m49.915403933s, test: 6m19.892065473s) the crash still happens on HEAD commit msg: Linux 4.19.182 crash: possible deadlock in red_change IPv6: ADDRCONF(NETDEV_CHANGE): wlan1: link becomes ready IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 ====================================================== WARNING: possible circular locking dependency detected 4.19.182-syzkaller #0 Not tainted ------------------------------------------------------ syz-executor.5/10101 is trying to acquire lock: 00000000644ef9f2 ((&q->adapt_timer)){+.-.}, at: arch_local_irq_disable arch/x86/include/asm/paravirt.h:794 [inline] 00000000644ef9f2 ((&q->adapt_timer)){+.-.}, at: arch_local_irq_save arch/x86/include/asm/paravirt.h:807 [inline] 00000000644ef9f2 ((&q->adapt_timer)){+.-.}, at: del_timer_sync+0x76/0x1e0 kernel/time/timer.c:1294 but task is already holding lock: 000000000c5bcdcd (&qdisc_rx_lock){+.-.}, at: spin_lock_bh include/linux/spinlock.h:334 [inline] 000000000c5bcdcd (&qdisc_rx_lock){+.-.}, at: sch_tree_lock include/net/sch_generic.h:477 [inline] 000000000c5bcdcd (&qdisc_rx_lock){+.-.}, at: red_change+0x40a/0x1a60 net/sched/sch_red.c:230 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #1 (&qdisc_rx_lock){+.-.}: __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline] _raw_spin_lock+0x2d/0x40 kernel/locking/spinlock.c:144 spin_lock include/linux/spinlock.h:329 [inline] red_adaptative_timer+0x94/0x540 net/sched/sch_red.c:265 call_timer_fn+0x14c/0x510 kernel/time/timer.c:1338 expire_timers+0x255/0x3a0 kernel/time/timer.c:1375 __run_timers kernel/time/timer.c:1696 [inline] run_timer_softirq+0x1dc/0x570 kernel/time/timer.c:1709 __do_softirq+0x25f/0x919 kernel/softirq.c:292 invoke_softirq kernel/softirq.c:372 [inline] irq_exit+0x17f/0x1c0 kernel/softirq.c:412 exiting_irq arch/x86/include/asm/apic.h:535 [inline] smp_apic_timer_interrupt+0x13e/0x540 arch/x86/kernel/apic/apic.c:1098 apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:894 arch_local_irq_restore arch/x86/include/asm/paravirt.h:789 [inline] qlink_free mm/kasan/quarantine.c:150 [inline] qlist_free_all+0xf4/0x120 mm/kasan/quarantine.c:166 quarantine_reduce+0x167/0x1a0 mm/kasan/quarantine.c:259 kasan_kmalloc+0x9b/0xc0 mm/kasan/kasan.c:538 kmem_cache_alloc_node_trace+0x157/0x3e0 mm/slab.c:3668 __do_kmalloc_node mm/slab.c:3688 [inline] __kmalloc_node+0x3c/0x70 mm/slab.c:3696 kmalloc_node include/linux/slab.h:557 [inline] kvmalloc_node+0x68/0x70 mm/util.c:423 kvmalloc include/linux/mm.h:577 [inline] kvzalloc include/linux/mm.h:585 [inline] allocate_hook_entries_size+0x30/0x70 net/netfilter/core.c:60 nf_hook_entries_grow+0xcd/0x860 net/netfilter/core.c:127 __nf_register_net_hook+0x117/0x340 net/netfilter/core.c:336 nf_register_net_hook+0x45/0xb0 net/netfilter/core.c:463 nf_register_net_hooks+0x44/0xa0 net/netfilter/core.c:479 __ip_vs_dev_init+0x15/0x30 net/netfilter/ipvs/ip_vs_core.c:2292 ops_init+0x99/0x380 net/core/net_namespace.c:129 setup_net+0x2c1/0x660 net/core/net_namespace.c:315 copy_net_ns+0x1b4/0x2a0 net/core/net_namespace.c:438 create_new_namespaces+0x483/0x750 kernel/nsproxy.c:107 copy_namespaces+0x22b/0x2b0 kernel/nsproxy.c:165 copy_process.part.2+0x3524/0x7960 kernel/fork.c:1915 copy_process kernel/fork.c:1718 [inline] _do_fork+0x166/0xc30 kernel/fork.c:2218 __do_sys_clone kernel/fork.c:2325 [inline] __se_sys_clone kernel/fork.c:2319 [inline] __x64_sys_clone+0xba/0x140 kernel/fork.c:2319 do_syscall_64+0xd0/0x4e0 arch/x86/entry/common.c:293 entry_SYSCALL_64_after_hwframe+0x49/0xbe -> #0 ((&q->adapt_timer)){+.-.}: lock_acquire+0x180/0x3a0 kernel/locking/lockdep.c:3907 del_timer_sync+0xa2/0x1e0 kernel/time/timer.c:1295 red_destroy+0x14/0x60 net/sched/sch_red.c:182 qdisc_destroy+0x128/0x650 net/sched/sch_generic.c:983 red_change+0x626/0x1a60 net/sched/sch_red.c:236 qdisc_change net/sched/sch_api.c:1240 [inline] tc_modify_qdisc+0xbd9/0x18e0 net/sched/sch_api.c:1543 rtnetlink_rcv_msg+0x34f/0x950 net/core/rtnetlink.c:4778 netlink_rcv_skb+0x13e/0x3d0 net/netlink/af_netlink.c:2455 rtnetlink_rcv+0x10/0x20 net/core/rtnetlink.c:4796 netlink_unicast_kernel net/netlink/af_netlink.c:1318 [inline] netlink_unicast+0x443/0x660 net/netlink/af_netlink.c:1344 netlink_sendmsg+0x667/0xc60 net/netlink/af_netlink.c:1909 sock_sendmsg_nosec net/socket.c:622 [inline] sock_sendmsg+0xac/0xf0 net/socket.c:632 ___sys_sendmsg+0x647/0x950 net/socket.c:2115 __sys_sendmsg+0xd9/0x180 net/socket.c:2153 __do_sys_sendmsg net/socket.c:2162 [inline] __se_sys_sendmsg net/socket.c:2160 [inline] __x64_sys_sendmsg+0x73/0xb0 net/socket.c:2160 do_syscall_64+0xd0/0x4e0 arch/x86/entry/common.c:293 entry_SYSCALL_64_after_hwframe+0x49/0xbe other info that might help us debug this: Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(&qdisc_rx_lock); lock((&q->adapt_timer)); lock(&qdisc_rx_lock); lock((&q->adapt_timer)); *** DEADLOCK *** 2 locks held by syz-executor.5/10101: #0: 0000000087dbcdb3 (rtnl_mutex){+.+.}, at: rtnl_lock net/core/rtnetlink.c:77 [inline] #0: 0000000087dbcdb3 (rtnl_mutex){+.+.}, at: rtnetlink_rcv_msg+0x301/0x950 net/core/rtnetlink.c:4775 #1: 000000000c5bcdcd (&qdisc_rx_lock){+.-.}, at: spin_lock_bh include/linux/spinlock.h:334 [inline] #1: 000000000c5bcdcd (&qdisc_rx_lock){+.-.}, at: sch_tree_lock include/net/sch_generic.h:477 [inline] #1: 000000000c5bcdcd (&qdisc_rx_lock){+.-.}, at: red_change+0x40a/0x1a60 net/sched/sch_red.c:230 stack backtrace: CPU: 0 PID: 10101 Comm: syz-executor.5 Not tainted 4.19.182-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x17c/0x226 lib/dump_stack.c:118 print_circular_bug.isra.17.cold.34+0x2e3/0x41e kernel/locking/lockdep.c:1221 check_prev_add kernel/locking/lockdep.c:1865 [inline] check_prevs_add kernel/locking/lockdep.c:1978 [inline] validate_chain kernel/locking/lockdep.c:2419 [inline] __lock_acquire+0x35c2/0x47c0 kernel/locking/lockdep.c:3415 lock_acquire+0x180/0x3a0 kernel/locking/lockdep.c:3907 del_timer_sync+0xa2/0x1e0 kernel/time/timer.c:1295 red_destroy+0x14/0x60 net/sched/sch_red.c:182 qdisc_destroy+0x128/0x650 net/sched/sch_generic.c:983 red_change+0x626/0x1a60 net/sched/sch_red.c:236 qdisc_change net/sched/sch_api.c:1240 [inline] tc_modify_qdisc+0xbd9/0x18e0 net/sched/sch_api.c:1543 rtnetlink_rcv_msg+0x34f/0x950 net/core/rtnetlink.c:4778 netlink_rcv_skb+0x13e/0x3d0 net/netlink/af_netlink.c:2455 rtnetlink_rcv+0x10/0x20 net/core/rtnetlink.c:4796 netlink_unicast_kernel net/netlink/af_netlink.c:1318 [inline] netlink_unicast+0x443/0x660 net/netlink/af_netlink.c:1344 netlink_sendmsg+0x667/0xc60 net/netlink/af_netlink.c:1909 sock_sendmsg_nosec net/socket.c:622 [inline] sock_sendmsg+0xac/0xf0 net/socket.c:632 ___sys_sendmsg+0x647/0x950 net/socket.c:2115 __sys_sendmsg+0xd9/0x180 net/socket.c:2153 __do_sys_sendmsg net/socket.c:2162 [inline] __se_sys_sendmsg net/socket.c:2160 [inline] __x64_sys_sendmsg+0x73/0xb0 net/socket.c:2160 do_syscall_64+0xd0/0x4e0 arch/x86/entry/common.c:293 entry_SYSCALL_64_after_hwframe+0x49/0xbe RIP: 0033:0x465b09 Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 bc ff ff ff f7 d8 64 89 01 48 RSP: 002b:00007f55bd84b188 EFLAGS: 00000246 ORIG_RAX: 000000000000002e RAX: ffffffffffffffda RBX: 000000000056bf60 RCX: 0000000000465b09 RDX: 0000000000000000 RSI: 00000000200007c0 RDI: 0000000000000006 RBP: 00000000004b069f R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 000000000056bf60 R13: 00007ffefe748b1f R14: 00007f55bd84b300 R15: 0000000000022000 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21 IPVS: ftp: loaded support on port[0] = 21