bisecting fixing commit since ad347abe4a9876b1f65f408ab467137e88f77eb4 building syzkaller on 1ba81399e2d03b53a7e631c4ab05f25af1fb1911 testing commit ad347abe4a9876b1f65f408ab467137e88f77eb4 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 865d3c40c172cafc1eeff01355f98b9738f56605a5dc669616daa8f5df3b4827 run #0: crashed: BUG: sleeping function called from invalid context in lock_sock_nested run #1: crashed: KASAN: out-of-bounds Read in __show_regs run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK reproducer seems to be flaky testing current HEAD 64222515138e43da1fcf288f0289ef1020427b87 testing commit 64222515138e43da1fcf288f0289ef1020427b87 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: ee9bc46a9e5c63d5a6c7e3570810587b24b3c472df258efd968fb5730051ab13 all runs: OK # git bisect start 64222515138e43da1fcf288f0289ef1020427b87 ad347abe4a9876b1f65f408ab467137e88f77eb4 Bisecting: 14784 revisions left to test after this (roughly 14 steps) [450405cdc0e18b899925b87aa4d1e84775082450] Merge tag 'gvt-fixes-2021-07-15' of https://github.com/intel/gvt-linux into drm-intel-fixes testing commit 450405cdc0e18b899925b87aa4d1e84775082450 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 8796d6568993862b982cc00999a2f4095010c73d001ab3307302147b0a611f1e run #0: crashed: KASAN: stack-out-of-bounds Read in __show_regs run #1: crashed: INFO: task hung in linkwatch_event run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK # git bisect good 450405cdc0e18b899925b87aa4d1e84775082450 Bisecting: 7722 revisions left to test after this (roughly 13 steps) [1b4f3dfb4792f03b139edf10124fcbeb44e608e6] Merge tag 'usb-serial-5.15-rc1' of https://git.kernel.org/pub/scm/linux/kernel/git/johan/usb-serial into usb-next testing commit 1b4f3dfb4792f03b139edf10124fcbeb44e608e6 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 6d4e78528c702e2d938d8cb4ec69a943c49c4f9f0b2a309be340b495900cbc85 run #0: crashed: KFENCE: use-after-free in kvm_fastop_exception run #1: crashed: KASAN: use-after-free Read in __d_alloc run #2: crashed: KFENCE: use-after-free in kvm_fastop_exception run #3: crashed: KASAN: use-after-free Read in __d_alloc run #4: crashed: KASAN: use-after-free Read in __d_alloc run #5: crashed: KASAN: use-after-free Read in __d_alloc run #6: crashed: KFENCE: use-after-free in kvm_fastop_exception run #7: OK run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK # git bisect good 1b4f3dfb4792f03b139edf10124fcbeb44e608e6 Bisecting: 3862 revisions left to test after this (roughly 12 steps) [f1583cb1be35c23df60b1c39e3e7e6704d749d0b] Merge tag 'linux-kselftest-next-5.15-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest testing commit f1583cb1be35c23df60b1c39e3e7e6704d749d0b compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 06558a1912a30a73ce4a4791a1edd11650dacfb4bc28346477e715dfeb36a401 run #0: crashed: KFENCE: use-after-free in kvm_fastop_exception run #1: OK run #2: crashed: KASAN: use-after-free Read in __d_alloc run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: crashed: KASAN: use-after-free Read in __d_alloc run #17: OK run #18: boot failed: possible deadlock in blktrans_open run #19: boot failed: possible deadlock in blktrans_open # git bisect good f1583cb1be35c23df60b1c39e3e7e6704d749d0b Bisecting: 1930 revisions left to test after this (roughly 11 steps) [e700ac213a0f793fb4f83098413303e3dd080892] Merge branch 'pruss-fix' into fixes testing commit e700ac213a0f793fb4f83098413303e3dd080892 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: ce277ad2be0f82b5108c1da6522fba72bdc74763c4e02b5f9312026ba37c8cfb run #0: crashed: KASAN: out-of-bounds Read in __show_regs run #1: OK run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: boot failed: can't ssh into the instance # git bisect good e700ac213a0f793fb4f83098413303e3dd080892 Bisecting: 962 revisions left to test after this (roughly 10 steps) [84928ce3bb4e20ec7ef0e990630a690855dd44cc] Merge tag 'driver-core-5.15-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core testing commit 84928ce3bb4e20ec7ef0e990630a690855dd44cc compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: d1a7a0d9bf26a72d1f6a4a7b96f3aeeda00c679ea1464138e4c67b00ade05347 run #0: crashed: KASAN: out-of-bounds Read in __show_regs run #1: crashed: WARNING in __cfg80211_ibss_joined run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK # git bisect good 84928ce3bb4e20ec7ef0e990630a690855dd44cc Bisecting: 470 revisions left to test after this (roughly 9 steps) [ec681c53f8d2d0ee362ff67f5b98dd8263c15002] Merge tag 'net-5.15-rc6' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net testing commit ec681c53f8d2d0ee362ff67f5b98dd8263c15002 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: f529a6cfa94d7113167c56c5558518c3b08a78928a70f8efb0ad9c3aeb59cf3c all runs: OK # git bisect bad ec681c53f8d2d0ee362ff67f5b98dd8263c15002 Bisecting: 245 revisions left to test after this (roughly 8 steps) [9973a43012b6ad1720dbc4d5faf5302c28635b8c] r8152: select CRC32 and CRYPTO/CRYPTO_HASH/CRYPTO_SHA256 testing commit 9973a43012b6ad1720dbc4d5faf5302c28635b8c compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: e840cf5367d7401bbdf034b8cdcef887eca59121fc0e2d35d74f78ef3fc3931f run #0: crashed: KASAN: stack-out-of-bounds Read in __show_regs run #1: crashed: KASAN: out-of-bounds Read in __show_regs run #2: crashed: KASAN: stack-out-of-bounds Read in __show_regs run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK # git bisect good 9973a43012b6ad1720dbc4d5faf5302c28635b8c Bisecting: 124 revisions left to test after this (roughly 7 steps) [c22ccc4a3ef198752855f44e9279732260c889d5] Merge tag 'x86_urgent_for_v5.15_rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip testing commit c22ccc4a3ef198752855f44e9279732260c889d5 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 13c02fcb0e7f1bb7faea8de566ee1f7b910aba82c036b1c1fbd95c287a297963 run #0: crashed: KASAN: out-of-bounds Read in __show_regs run #1: OK run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK # git bisect good c22ccc4a3ef198752855f44e9279732260c889d5 Bisecting: 63 revisions left to test after this (roughly 6 steps) [ed47291911d375f7d0bf63b9afb7516988305d94] Merge tag 'platform-drivers-x86-v5.15-3' of git://git.kernel.org/pub/scm/linux/kernel/git/pdx86/platform-drivers-x86 testing commit ed47291911d375f7d0bf63b9afb7516988305d94 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: ad324476b14e2d7c7f64cfe9d763755c181428a96dc8fa57b3229de82a86d9a7 run #0: crashed: KASAN: stack-out-of-bounds Read in __show_regs run #1: OK run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK # git bisect good ed47291911d375f7d0bf63b9afb7516988305d94 Bisecting: 31 revisions left to test after this (roughly 5 steps) [1f922d9e374f0ca7d1b241666c792f403c6120aa] Revert "net: procfs: add seq_puts() statement for dev_mcast" testing commit 1f922d9e374f0ca7d1b241666c792f403c6120aa compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 9a8beb61aacc1ae1b679efad7fd34b4b92c0c35c2932f7f4d7b718fe1b308829 all runs: OK # git bisect bad 1f922d9e374f0ca7d1b241666c792f403c6120aa Bisecting: 15 revisions left to test after this (roughly 4 steps) [deab6b1cd9789bb9bd466d5e76aecb8b336259b4] net: dsa: tag_ocelot: break circular dependency with ocelot switch lib driver testing commit deab6b1cd9789bb9bd466d5e76aecb8b336259b4 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: cbf390c9ca894469ceb6e8e95491f8f6c8f6a626eb3b75bc70f9cba501926fa2 run #0: crashed: KASAN: out-of-bounds Read in __show_regs run #1: OK run #2: OK run #3: OK run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK # git bisect good deab6b1cd9789bb9bd466d5e76aecb8b336259b4 Bisecting: 8 revisions left to test after this (roughly 3 steps) [427f974d9727ca681085ddcd0530c97ab5811ae0] net: korina: select CRC32 testing commit 427f974d9727ca681085ddcd0530c97ab5811ae0 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 1768a13d1c5ca588658a77a3cd3a2416db781327bec89c5862149aea81f7ecb7 all runs: OK # git bisect bad 427f974d9727ca681085ddcd0530c97ab5811ae0 Bisecting: 3 revisions left to test after this (roughly 2 steps) [43ba33b4f143965a451cfdc1e826b61f6933c887] net: dsa: tag_ocelot_8021q: fix inability to inject STP BPDUs into BLOCKING ports testing commit 43ba33b4f143965a451cfdc1e826b61f6933c887 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 1768a13d1c5ca588658a77a3cd3a2416db781327bec89c5862149aea81f7ecb7 all runs: OK # git bisect bad 43ba33b4f143965a451cfdc1e826b61f6933c887 Bisecting: 0 revisions left to test after this (roughly 1 step) [1328a883258b4507909090ed0a9ad63771f9f780] net: dsa: felix: purge skb from TX timestamping queue if it cannot be sent testing commit 1328a883258b4507909090ed0a9ad63771f9f780 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 1768a13d1c5ca588658a77a3cd3a2416db781327bec89c5862149aea81f7ecb7 all runs: OK # git bisect bad 1328a883258b4507909090ed0a9ad63771f9f780 Bisecting: 0 revisions left to test after this (roughly 0 steps) [49f885b2d97093451410e7279aa29d81e094e108] net: dsa: tag_ocelot_8021q: break circular dependency with ocelot switch lib testing commit 49f885b2d97093451410e7279aa29d81e094e108 compiler: gcc (GCC) 10.2.1 20210217, GNU ld (GNU Binutils for Debian) 2.35.2 kernel signature: 1768a13d1c5ca588658a77a3cd3a2416db781327bec89c5862149aea81f7ecb7 run #0: crashed: KASAN: out-of-bounds Read in __show_regs run #1: crashed: KASAN: out-of-bounds Read in __show_regs run #2: crashed: KASAN: stack-out-of-bounds Read in __show_regs run #3: crashed: KASAN: stack-out-of-bounds Read in __show_regs run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK run #10: OK run #11: OK run #12: OK run #13: OK run #14: OK run #15: OK run #16: OK run #17: OK run #18: OK run #19: OK # git bisect good 49f885b2d97093451410e7279aa29d81e094e108 1328a883258b4507909090ed0a9ad63771f9f780 is the first bad commit commit 1328a883258b4507909090ed0a9ad63771f9f780 Author: Vladimir Oltean Date: Tue Oct 12 14:40:42 2021 +0300 net: dsa: felix: purge skb from TX timestamping queue if it cannot be sent At present, when a PTP packet which requires TX timestamping gets dropped under congestion by the switch, things go downhill very fast. The driver keeps a clone of that skb in a queue of packets awaiting TX timestamp interrupts, but interrupts will never be raised for the dropped packets. Moreover, matching timestamped packets to timestamps is done by a 2-bit timestamp ID, and this can wrap around and we can match on the wrong skb. Since with the default NPI-based tagging protocol, we get no notification about packet drops, the best we can do is eventually recover from the drop of a PTP frame: its skb will be dead memory until another skb which was assigned the same timestamp ID happens to find it. However, with the ocelot-8021q tagger which injects packets using the manual register interface, it appears that we can check for more information, such as: - whether the input queue has reached the high watermark or not - whether the injection group's FIFO can accept additional data or not so we know that a PTP frame is likely to get dropped before actually sending it, and drop it ourselves (because DSA uses NETIF_F_LLTX, so it can't return NETDEV_TX_BUSY to ask the qdisc to requeue the packet). But when we do that, we can also remove the skb from the timestamping queue, because there surely won't be any timestamp that matches it. Fixes: 0a6f17c6ae21 ("net: dsa: tag_ocelot_8021q: add support for PTP timestamping") Signed-off-by: Vladimir Oltean Signed-off-by: Jakub Kicinski drivers/net/dsa/ocelot/felix.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) culprit signature: 1768a13d1c5ca588658a77a3cd3a2416db781327bec89c5862149aea81f7ecb7 parent signature: 1768a13d1c5ca588658a77a3cd3a2416db781327bec89c5862149aea81f7ecb7 Reproducer flagged being flaky revisions tested: 17, total time: 4h36m10.747180134s (build: 1h53m13.33518336s, test: 2h41m1.78837243s) first good commit: 1328a883258b4507909090ed0a9ad63771f9f780 net: dsa: felix: purge skb from TX timestamping queue if it cannot be sent recipients (to): ["UNGLinuxDriver@microchip.com" "alexandre.belloni@bootlin.com" "andrew@lunn.ch" "claudiu.manoil@nxp.com" "davem@davemloft.net" "f.fainelli@gmail.com" "kuba@kernel.org" "kuba@kernel.org" "netdev@vger.kernel.org" "vivien.didelot@gmail.com" "vladimir.oltean@nxp.com" "vladimir.oltean@nxp.com"] recipients (cc): ["linux-kernel@vger.kernel.org"]