bisecting cause commit starting from 87d5034d0758d4490c6524445774d89e318a6383 building syzkaller on 9602ddf403bdf3cfd87efef14becc76f9a38b81d testing commit 87d5034d0758d4490c6524445774d89e318a6383 with gcc (GCC) 8.1.0 kernel signature: be3fd3cf5004bd20c40150a2fbe3cee86c975895e18bb1b41198409efc2e6ccc run #0: crashed: INFO: task hung in synchronize_rcu run #1: crashed: INFO: task hung in synchronize_rcu run #2: crashed: INFO: task hung in synchronize_rcu run #3: crashed: INFO: task hung in synchronize_rcu run #4: OK run #5: OK run #6: OK run #7: OK run #8: OK run #9: OK testing release v5.8 testing commit bcf876870b95592b52519ed4aafcf9d95999bc9c with gcc (GCC) 8.1.0 kernel signature: e17d6c2144cd9e9700d281171001795a9352453f126f79c59ce33b01a0fc5e99 all runs: OK # git bisect start 87d5034d0758d4490c6524445774d89e318a6383 bcf876870b95592b52519ed4aafcf9d95999bc9c Bisecting: 8558 revisions left to test after this (roughly 13 steps) [3f9df56480fc8ce492fc9e988d67bdea884ed15c] Merge tag 'sound-5.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/sound testing commit 3f9df56480fc8ce492fc9e988d67bdea884ed15c with gcc (GCC) 8.1.0 kernel signature: 8d9aadc6121413fd870f90ade7f778447e85b0cba17b6f23f2cae4708c36f54e all runs: OK # git bisect good 3f9df56480fc8ce492fc9e988d67bdea884ed15c Bisecting: 4279 revisions left to test after this (roughly 12 steps) [71a50419c7307bef6367e8f3787570f546ae96f8] Merge tag 'linux-can-fixes-for-5.9-20200815' of git://git.kernel.org/pub/scm/linux/kernel/git/mkl/linux-can testing commit 71a50419c7307bef6367e8f3787570f546ae96f8 with gcc (GCC) 8.1.0 kernel signature: b4088f92345d6e788b03d3aa8f9e00d58e591b08396b9554e95b8afcfd435879 all runs: OK # git bisect good 71a50419c7307bef6367e8f3787570f546ae96f8 Bisecting: 2097 revisions left to test after this (roughly 11 steps) [d3017135c43373b06eef1eb70dfeb948b8ae159f] Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net testing commit d3017135c43373b06eef1eb70dfeb948b8ae159f with gcc (GCC) 8.1.0 kernel signature: 73879aca690479c4accf3b2218515d49ca69c02b19e4218efe5ce293ba3f69ed all runs: OK # git bisect good d3017135c43373b06eef1eb70dfeb948b8ae159f Bisecting: 1048 revisions left to test after this (roughly 10 steps) [88bdef8be9f649ce4adf0d775a9a1cba7d5cc524] net: dsa: mt7530: Extend device data ready for adding a new hardware testing commit 88bdef8be9f649ce4adf0d775a9a1cba7d5cc524 with gcc (GCC) 8.1.0 kernel signature: e9cedbce8b38da646aaa7b5d61e678bb507cb8c7f487bb3d11a57bb27160168d run #0: crashed: INFO: task hung in synchronize_rcu run #1: crashed: INFO: task hung in synchronize_rcu run #2: crashed: INFO: task hung in synchronize_rcu run #3: crashed: INFO: task hung in synchronize_rcu run #4: crashed: INFO: task hung in synchronize_rcu run #5: crashed: INFO: task hung in synchronize_rcu run #6: OK run #7: OK run #8: OK run #9: OK # git bisect bad 88bdef8be9f649ce4adf0d775a9a1cba7d5cc524 Bisecting: 524 revisions left to test after this (roughly 9 steps) [ac99a822c67b960c17e165a01c00c6813e496f1c] net: ethernet/neterion/vxge: fix spelling of "functionality" testing commit ac99a822c67b960c17e165a01c00c6813e496f1c with gcc (GCC) 8.1.0 kernel signature: 8e8f2f4a43d236cbc878fb4b3ded6e01bda9827d2ef1494a10ced2115a80324c all runs: OK # git bisect good ac99a822c67b960c17e165a01c00c6813e496f1c Bisecting: 282 revisions left to test after this (roughly 8 steps) [0dc0b5c29be240e4d0b6e1cb31be39116cd237b2] rtlwifi: switch from 'pci_' to 'dma_' API testing commit 0dc0b5c29be240e4d0b6e1cb31be39116cd237b2 with gcc (GCC) 8.1.0 kernel signature: af04fbc48033e927c7140b6a4347b406d163230f8757e9f4aeeba500d4b45c75 all runs: OK # git bisect good 0dc0b5c29be240e4d0b6e1cb31be39116cd237b2 Bisecting: 124 revisions left to test after this (roughly 7 steps) [e7a08121e0f4954101605b30c2cf725ccdef7ec7] Merge tag 'wireless-drivers-next-2020-09-11' of git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/wireless-drivers-next testing commit e7a08121e0f4954101605b30c2cf725ccdef7ec7 with gcc (GCC) 8.1.0 kernel signature: 3892da341137ca5961ba5b1632131ac202686d8fddac3ebfc87a9bc4c5110f64 all runs: OK # git bisect good e7a08121e0f4954101605b30c2cf725ccdef7ec7 Bisecting: 62 revisions left to test after this (roughly 6 steps) [456b2f2dc7e585e1a031214c5698f1b00e02448b] rxrpc: Fix an error goto in rxrpc_connect_call() testing commit 456b2f2dc7e585e1a031214c5698f1b00e02448b with gcc (GCC) 8.1.0 kernel signature: 73d2d05b296deae36147cd6827ad1eb9ab86da7be4e4b8305e8b56e1f2d1f43f all runs: OK # git bisect good 456b2f2dc7e585e1a031214c5698f1b00e02448b Bisecting: 31 revisions left to test after this (roughly 5 steps) [4596a2c1b7f55b8dac9ef4d8724cbef6e11eb74c] mptcp: allow creating non-backup subflows testing commit 4596a2c1b7f55b8dac9ef4d8724cbef6e11eb74c with gcc (GCC) 8.1.0 kernel signature: 520321a44aec2be7d40163d4bf869d19a164678c81dcd4257a370cb1f44e880d run #0: crashed: INFO: task hung in synchronize_rcu run #1: crashed: INFO: task hung in synchronize_rcu run #2: crashed: INFO: task hung in synchronize_rcu run #3: crashed: INFO: task hung in synchronize_rcu run #4: crashed: INFO: task hung in synchronize_rcu run #5: crashed: INFO: task hung in synchronize_rcu run #6: crashed: INFO: task hung in synchronize_rcu run #7: crashed: INFO: task hung in synchronize_rcu run #8: crashed: INFO: task hung in synchronize_rcu run #9: OK # git bisect bad 4596a2c1b7f55b8dac9ef4d8724cbef6e11eb74c Bisecting: 15 revisions left to test after this (roughly 4 steps) [54e977f01384d45519fdfe4f523ea195bcac8ed8] net: natsemi: convert tasklets to use new tasklet_setup() API testing commit 54e977f01384d45519fdfe4f523ea195bcac8ed8 with gcc (GCC) 8.1.0 kernel signature: 163e3c96f10685efd0202d9393b275fd8608bf3c14b7b97b9a55ce5da9c11ee3 all runs: OK # git bisect good 54e977f01384d45519fdfe4f523ea195bcac8ed8 Bisecting: 7 revisions left to test after this (roughly 3 steps) [47bebdf365ade17534f6616dc228d347085f4501] mptcp: set data_ready status bit in subflow_check_data_avail() testing commit 47bebdf365ade17534f6616dc228d347085f4501 with gcc (GCC) 8.1.0 kernel signature: 440d9161b3103a77d3c0ff6863c4b02590c28cfc51b604bccdb3fdb5a4ffa3c8 all runs: OK # git bisect good 47bebdf365ade17534f6616dc228d347085f4501 Bisecting: 3 revisions left to test after this (roughly 2 steps) [ab174ad8ef76276cadfdae98731d31797d265927] mptcp: move ooo skbs into msk out of order queue. testing commit ab174ad8ef76276cadfdae98731d31797d265927 with gcc (GCC) 8.1.0 kernel signature: 92ed8c0ab3066f96486d0672a3a8440957632af505487f6b926deec7ff7a47af run #0: crashed: INFO: task hung in synchronize_rcu run #1: crashed: INFO: task hung in synchronize_rcu run #2: crashed: INFO: task hung in synchronize_rcu run #3: crashed: INFO: task hung in synchronize_rcu run #4: crashed: INFO: task hung in synchronize_rcu run #5: crashed: INFO: task hung in synchronize_rcu run #6: OK run #7: OK run #8: OK run #9: OK # git bisect bad ab174ad8ef76276cadfdae98731d31797d265927 Bisecting: 1 revision left to test after this (roughly 1 step) [da51aef5fe5b9ef389055f693472d4fb5a3f58f9] mptcp: basic sndbuf autotuning testing commit da51aef5fe5b9ef389055f693472d4fb5a3f58f9 with gcc (GCC) 8.1.0 kernel signature: 87c0db9b9dde86316783bf3a21a9e5f5d9ab645760996536e5c2f786db799162 all runs: OK # git bisect good da51aef5fe5b9ef389055f693472d4fb5a3f58f9 Bisecting: 0 revisions left to test after this (roughly 0 steps) [8268ed4c9d197fc6448b6262c62aad62ef9aa4d5] mptcp: introduce and use mptcp_try_coalesce() testing commit 8268ed4c9d197fc6448b6262c62aad62ef9aa4d5 with gcc (GCC) 8.1.0 kernel signature: ec4dcd7dd7661d41babad9ef78425caffb8a2a2a580fdc0e8e5d929360578137 all runs: OK # git bisect good 8268ed4c9d197fc6448b6262c62aad62ef9aa4d5 ab174ad8ef76276cadfdae98731d31797d265927 is the first bad commit commit ab174ad8ef76276cadfdae98731d31797d265927 Author: Paolo Abeni Date: Mon Sep 14 10:01:12 2020 +0200 mptcp: move ooo skbs into msk out of order queue. Add an RB-tree to cope with OoO (at MPTCP level) data. __mptcp_move_skb() insert into the RB tree "future" data, eventually coalescing skb as allowed by the MPTCP DSN. To simplify sequence accounting, move the DSN inside the cb. After successfully enqueuing in sequence data, check if we can use any data from the RB tree. Additionally move the data_fin check after spooling data from the OoO tree, otherwise we could miss shutdown events. The RB tree code is copied as verbatim as possible from tcp_data_queue_ofo(), with a few simplifications due to the fact that MPTCP doesn't need to cope with sacks. All bugs here are added by me. Signed-off-by: Paolo Abeni Reviewed-by: Mat Martineau Signed-off-by: David S. Miller net/mptcp/protocol.c | 264 ++++++++++++++++++++++++++++++++++++++++----------- net/mptcp/protocol.h | 2 + net/mptcp/subflow.c | 1 + 3 files changed, 211 insertions(+), 56 deletions(-) culprit signature: 92ed8c0ab3066f96486d0672a3a8440957632af505487f6b926deec7ff7a47af parent signature: ec4dcd7dd7661d41babad9ef78425caffb8a2a2a580fdc0e8e5d929360578137 revisions tested: 16, total time: 4h16m11.522285045s (build: 1h22m0.595225352s, test: 2h52m35.890398815s) first bad commit: ab174ad8ef76276cadfdae98731d31797d265927 mptcp: move ooo skbs into msk out of order queue. recipients (to): ["davem@davemloft.net" "mathew.j.martineau@linux.intel.com" "pabeni@redhat.com"] recipients (cc): [] crash: INFO: task hung in synchronize_rcu INFO: task kworker/u4:0:7 blocked for more than 143 seconds. Not tainted 5.9.0-rc3-syzkaller #0 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. task:kworker/u4:0 state:D stack:12104 pid: 7 ppid: 2 flags:0x00004000 Workqueue: events_unbound fsnotify_mark_destroy_workfn Call Trace: context_switch kernel/sched/core.c:3778 [inline] __schedule+0x418/0x910 kernel/sched/core.c:4527 schedule+0x37/0xe0 kernel/sched/core.c:4602 schedule_timeout+0x266/0x350 kernel/time/timer.c:1855 do_wait_for_common kernel/sched/completion.c:85 [inline] __wait_for_common kernel/sched/completion.c:106 [inline] wait_for_common kernel/sched/completion.c:117 [inline] wait_for_completion+0xa7/0x110 kernel/sched/completion.c:138 __synchronize_srcu+0xcf/0x170 kernel/rcu/srcutree.c:935 fsnotify_mark_destroy_workfn+0x74/0xf0 fs/notify/mark.c:836 process_one_work+0x26a/0x5f0 kernel/workqueue.c:2269 worker_thread+0x38/0x380 kernel/workqueue.c:2415 kthread+0x148/0x170 kernel/kthread.c:292 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:294 INFO: task kworker/u4:5:8339 blocked for more than 143 seconds. Not tainted 5.9.0-rc3-syzkaller #0 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. task:kworker/u4:5 state:D stack:12432 pid: 8339 ppid: 2 flags:0x00004000 Workqueue: events_unbound fsnotify_connector_destroy_workfn Call Trace: context_switch kernel/sched/core.c:3778 [inline] __schedule+0x418/0x910 kernel/sched/core.c:4527 schedule+0x37/0xe0 kernel/sched/core.c:4602 schedule_timeout+0x266/0x350 kernel/time/timer.c:1855 do_wait_for_common kernel/sched/completion.c:85 [inline] __wait_for_common kernel/sched/completion.c:106 [inline] wait_for_common kernel/sched/completion.c:117 [inline] wait_for_completion+0xa7/0x110 kernel/sched/completion.c:138 __synchronize_srcu+0xcf/0x170 kernel/rcu/srcutree.c:935 fsnotify_connector_destroy_workfn+0x3f/0x70 fs/notify/mark.c:164 process_one_work+0x26a/0x5f0 kernel/workqueue.c:2269 worker_thread+0x38/0x380 kernel/workqueue.c:2415 kthread+0x148/0x170 kernel/kthread.c:292 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:294 INFO: task syz-executor.1:5955 blocked for more than 143 seconds. Not tainted 5.9.0-rc3-syzkaller #0 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. task:syz-executor.1 state:D stack:13704 pid: 5955 ppid: 6972 flags:0x00004004 Call Trace: context_switch kernel/sched/core.c:3778 [inline] __schedule+0x418/0x910 kernel/sched/core.c:4527 schedule+0x37/0xe0 kernel/sched/core.c:4602 __lock_sock+0x92/0x100 net/core/sock.c:2504 lock_sock_nested+0x86/0x90 net/core/sock.c:3036 lock_sock include/net/sock.h:1583 [inline] sk_stream_wait_memory+0x19d/0x360 net/core/stream.c:145 mptcp_sendmsg+0x114/0x5b0 net/mptcp/protocol.c:1107 sock_sendmsg_nosec net/socket.c:651 [inline] sock_sendmsg+0x2b/0x40 net/socket.c:671 __sys_sendto+0xec/0x160 net/socket.c:1992 __do_sys_sendto net/socket.c:2004 [inline] __se_sys_sendto net/socket.c:2000 [inline] __x64_sys_sendto+0x1f/0x30 net/socket.c:2000 do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46 entry_SYSCALL_64_after_hwframe+0x44/0xa9 RIP: 0033:0x45dd99 Code: Bad RIP value. RSP: 002b:00007f5280886c78 EFLAGS: 00000246 ORIG_RAX: 000000000000002c RAX: ffffffffffffffda RBX: 000000000002e880 RCX: 000000000045dd99 RDX: 00000000ffffffe7 RSI: 0000000020000100 RDI: 0000000000000003 RBP: 000000000118bf78 R08: 0000000000000000 R09: 0000000000000000 R10: 000000000000c000 R11: 0000000000000246 R12: 000000000118bf2c R13: 00007ffc443c4bcf R14: 00007f52808879c0 R15: 000000000118bf2c Showing all locks held in the system: 2 locks held by kworker/u4:0/7: #0: ffff88812bc57938 ((wq_completion)events_unbound){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:615 [inline] #0: ffff88812bc57938 ((wq_completion)events_unbound){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:643 [inline] #0: ffff88812bc57938 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x1de/0x5f0 kernel/workqueue.c:2240 #1: ffffc90000ca3e70 ((reaper_work).work){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:615 [inline] #1: ffffc90000ca3e70 ((reaper_work).work){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:643 [inline] #1: ffffc90000ca3e70 ((reaper_work).work){+.+.}-{0:0}, at: process_one_work+0x1de/0x5f0 kernel/workqueue.c:2240 1 lock held by khungtaskd/1148: #0: ffffffff845192a0 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x15/0x17a kernel/locking/lockdep.c:5830 4 locks held by kworker/0:3/3932: 1 lock held by in:imklog/6367: #0: ffff888122c48ef0 (&f->f_pos_lock){+.+.}-{3:3}, at: __fdget_pos+0x45/0x50 fs/file.c:930 2 locks held by kworker/u4:5/8339: #0: ffff88812bc57938 ((wq_completion)events_unbound){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:615 [inline] #0: ffff88812bc57938 ((wq_completion)events_unbound){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:643 [inline] #0: ffff88812bc57938 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x1de/0x5f0 kernel/workqueue.c:2240 #1: ffffc90002d7fe70 (connector_reaper_work){+.+.}-{0:0}, at: set_work_data kernel/workqueue.c:615 [inline] #1: ffffc90002d7fe70 (connector_reaper_work){+.+.}-{0:0}, at: set_work_pool_and_clear_pending kernel/workqueue.c:643 [inline] #1: ffffc90002d7fe70 (connector_reaper_work){+.+.}-{0:0}, at: process_one_work+0x1de/0x5f0 kernel/workqueue.c:2240 ============================================= NMI backtrace for cpu 1 CPU: 1 PID: 1148 Comm: khungtaskd Not tainted 5.9.0-rc3-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0xa3/0xcc lib/dump_stack.c:118 nmi_cpu_backtrace.cold.8+0x3e/0x58 lib/nmi_backtrace.c:101 nmi_trigger_cpumask_backtrace+0xd5/0xec lib/nmi_backtrace.c:62 trigger_all_cpu_backtrace include/linux/nmi.h:146 [inline] check_hung_uninterruptible_tasks kernel/hung_task.c:209 [inline] watchdog+0x58e/0x680 kernel/hung_task.c:295 kthread+0x148/0x170 kernel/kthread.c:292 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:294 Sending NMI from CPU 1 to CPUs 0: NMI backtrace for cpu 0 CPU: 0 PID: 8368 Comm: kworker/u4:6 Not tainted 5.9.0-rc3-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Workqueue: bat_events batadv_nc_worker RIP: 0010:check_preemption_disabled+0x2/0xd0 lib/smp_processor_id.c:13 Code: ec 20 75 01 85 c9 74 03 31 c0 c3 8b 15 47 ed 2a 01 b8 01 00 00 00 85 d2 74 0a c7 05 34 ed 2a 01 0f 00 00 00 c3 cc cc cc 41 55 <41> 54 55 53 65 8b 1d 63 a0 da 7c 65 8b 05 7c 0b db 7c a9 ff ff ff RSP: 0018:ffffc90002dc7d08 EFLAGS: 00000282 RAX: 0000000000000001 RBX: ffffc90002dc7d47 RCX: 0000000000000002 RDX: 0000000000000000 RSI: ffffffff842613ce RDI: ffffffff84094c35 RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 R10: ffff88810ee48580 R11: 1853f4cb28d6180c R12: 0000000000000002 R13: 0000000000000000 R14: ffffffff845192a0 R15: 0000000000000000 FS: 0000000000000000(0000) GS:ffff88812c000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f991d877028 CR3: 000000011feba000 CR4: 00000000001506f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: rcu_dynticks_curr_cpu_in_eqs kernel/rcu/tree.c:326 [inline] rcu_is_watching+0x11/0x70 kernel/rcu/tree.c:1111 rcu_read_lock_held_common+0x1c/0x40 kernel/rcu/update.c:119 rcu_read_lock_sched_held+0x1e/0x80 kernel/rcu/update.c:134 trace_lock_acquire include/trace/events/lock.h:13 [inline] lock_acquire+0x362/0x3e0 kernel/locking/lockdep.c:4980 rcu_lock_acquire include/linux/rcupdate.h:241 [inline] rcu_read_lock include/linux/rcupdate.h:634 [inline] batadv_nc_process_nc_paths.part.18+0x62/0x180 net/batman-adv/network-coding.c:686 batadv_nc_process_nc_paths net/batman-adv/network-coding.c:678 [inline] batadv_nc_worker+0x22c/0x240 net/batman-adv/network-coding.c:727 process_one_work+0x26a/0x5f0 kernel/workqueue.c:2269 worker_thread+0x38/0x380 kernel/workqueue.c:2415 kthread+0x148/0x170 kernel/kthread.c:292 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:294