bisecting fixing commit since 7cc2a8ea104820dd9e702202621e8fd4d9f6c8cf
building syzkaller on 510951950dc0ee69cfdaf746061d3dbe31b49fd8
testing commit 7cc2a8ea104820dd9e702202621e8fd4d9f6c8cf with gcc (GCC) 8.1.0
kernel signature: 4962ea5c9ddb79b774fd8c2626636cc4360fa586081f4bde2f26e6db78e5c967
run #0: crashed: INFO: rcu detected stall in corrupted
run #1: crashed: INFO: task hung in lru_add_drain_all
run #2: crashed: INFO: rcu detected stall in corrupted
run #3: crashed: INFO: rcu detected stall in sys_mount
run #4: crashed: INFO: task hung in lru_add_drain_all
run #5: crashed: INFO: task hung in linkwatch_event
run #6: crashed: INFO: task hung in lru_add_drain_all
run #7: crashed: INFO: rcu detected stall in corrupted
run #8: OK
run #9: boot failed: can't ssh into the instance
testing current HEAD fc4f28bb3daf3265d6bc5f73b497306985bb23ab
testing commit fc4f28bb3daf3265d6bc5f73b497306985bb23ab with gcc (GCC) 8.1.0
kernel signature: 89be6895e7b9601ed3d63bcc7b4335a3d3342730be3d0ac0349894d772c7ffd7
run #0: crashed: INFO: rcu detected stall in corrupted
run #1: crashed: INFO: task hung in linkwatch_event
run #2: crashed: BUG: workqueue lockup
run #3: crashed: INFO: task hung in switchdev_deferred_process_work
run #4: crashed: INFO: rcu detected stall in sys_mount
run #5: crashed: INFO: task hung in switchdev_deferred_process_work
run #6: crashed: BUG: workqueue lockup
run #7: OK
run #8: OK
run #9: OK
revisions tested: 2, total time: 30m38.331372607s (build: 9m53.234996554s, test: 20m8.785121973s)
the crash still happens on HEAD
commit msg: Merge tag 'for-5.9-rc5-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux
crash: BUG: workqueue lockup
BUG: workqueue lockup - pool cpus=0 node=0 flags=0x0 nice=0 stuck for 248s!
Showing busy workqueues and worker pools:
workqueue events: flags=0x0
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=25/256 refcnt=26
    pending: vmstat_shepherd, nsim_dev_trap_report_work, nsim_dev_trap_report_work, psi_avgs_work, psi_avgs_work, free_obj_work, macvlan_process_broadcast, macvlan_process_broadcast, cache_reap, psi_avgs_work, psi_avgs_work, psi_avgs_work, psi_avgs_work, macvlan_process_broadcast, kfree_rcu_monitor, ovs_dp_masks_rebalance, check_corruption, macvlan_process_broadcast, macvlan_process_broadcast, macvlan_process_broadcast, macvlan_process_broadcast, switchdev_deferred_process_work, rht_deferred_worker, rht_deferred_worker, key_garbage_collector
workqueue events_long: flags=0x0
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=4/256 refcnt=5
    pending: defense_work_handler, defense_work_handler, defense_work_handler, br_fdb_cleanup
workqueue events_power_efficient: flags=0x80
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4
    pending: neigh_periodic_work, do_cache_clean, check_lifetime
workqueue rcu_gp: flags=0x8
workqueue mm_percpu_wq: flags=0x8
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: vmstat_update
workqueue dm_bufio_cache: flags=0x8
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: work_fn
workqueue ipv6_addrconf: flags=0x40008
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/1 refcnt=2
    pending: addrconf_verify_work
workqueue krxrpcd: flags=0x0
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/1 refcnt=4
    pending: rxrpc_peer_keepalive_worker
    delayed: rxrpc_peer_keepalive_worker, rxrpc_peer_keepalive_worker
workqueue wg-kex-wg0: flags=0x24
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg0: flags=0x28
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker, wg_packet_decrypt_worker
workqueue wg-crypt-wg1: flags=0x28
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4
    pending: wg_packet_decrypt_worker, wg_packet_tx_worker, wg_packet_encrypt_worker
workqueue wg-kex-wg2: flags=0x24
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg2: flags=0x28
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=2/256 refcnt=3
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker
workqueue wg-kex-wg0: flags=0x24
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg0: flags=0x28
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker, wg_packet_decrypt_worker
workqueue wg-kex-wg1: flags=0x24
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg1: flags=0x28
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4
    pending: wg_packet_decrypt_worker, wg_packet_tx_worker, wg_packet_encrypt_worker
workqueue wg-kex-wg2: flags=0x24
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg2: flags=0x28
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=2/256 refcnt=3
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker
workqueue wg-kex-wg0: flags=0x24
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg0: flags=0x28
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4
    pending: wg_packet_tx_worker, wg_packet_encrypt_worker, wg_packet_decrypt_worker
workqueue wg-kex-wg1: flags=0x24
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg1: flags=0x28
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4
    pending: wg_packet_tx_worker, wg_packet_encrypt_worker, wg_packet_decrypt_worker
workqueue wg-kex-wg2: flags=0x24
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg2: flags=0x28
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=2/256 refcnt=3
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker
workqueue wg-kex-wg0: flags=0x24
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg0: flags=0x28
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker, wg_packet_decrypt_worker
workqueue wg-kex-wg1: flags=0x24
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg1: flags=0x28
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4
    pending: wg_packet_decrypt_worker, wg_packet_encrypt_worker, wg_packet_tx_worker
workqueue wg-kex-wg2: flags=0x24
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg2: flags=0x28
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=2/256 refcnt=3
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker
workqueue wg-kex-wg0: flags=0x24
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg0: flags=0x28
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4
    pending: wg_packet_tx_worker, wg_packet_encrypt_worker, wg_packet_decrypt_worker
workqueue wg-kex-wg1: flags=0x24
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg1: flags=0x28
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4
    pending: wg_packet_encrypt_worker, wg_packet_decrypt_worker, wg_packet_tx_worker
workqueue wg-kex-wg2: flags=0x24
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg2: flags=0x28
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=2/256 refcnt=3
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker
workqueue wg-kex-wg0: flags=0x24
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg0: flags=0x28
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker, wg_packet_decrypt_worker
workqueue wg-kex-wg1: flags=0x24
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg1: flags=0x28
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4
    pending: wg_packet_tx_worker, wg_packet_encrypt_worker, wg_packet_decrypt_worker
workqueue wg-kex-wg2: flags=0x24
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2
    pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg2: flags=0x28
  pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=2/256 refcnt=3
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker
pool 2: cpus=1 node=0 flags=0x0 nice=0 hung=0s workers=4 idle: 8157 17 23
pool 4: cpus=0-1 flags=0x4 nice=0 hung=0s workers=7 idle: 236 854 7 21 765 27