syzbot


INFO: task hung in nsim_dev_trap_report_work

Status: upstream: reported on 2024/02/19 09:22
Reported-by: syzbot+53be3883961c884e1dd7@syzkaller.appspotmail.com
First crash: 78d, last: 48d

Sample crash report:
INFO: task kworker/1:0:21 blocked for more than 143 seconds.
      Not tainted 5.15.152-syzkaller #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/1:0     state:D stack:24992 pid:   21 ppid:     2 flags:0x00004000
Workqueue: events nsim_dev_trap_report_work
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5030 [inline]
 __schedule+0x12c4/0x45b0 kernel/sched/core.c:6376
 schedule+0x11b/0x1f0 kernel/sched/core.c:6459
 schedule_preempt_disabled+0xf/0x20 kernel/sched/core.c:6518
 __mutex_lock_common+0xe34/0x25a0 kernel/locking/mutex.c:669
 __mutex_lock kernel/locking/mutex.c:729 [inline]
 mutex_lock_nested+0x17/0x20 kernel/locking/mutex.c:743
 nsim_dev_trap_report_work+0x56/0xab0 drivers/net/netdevsim/dev.c:757
 process_one_work+0x8a1/0x10c0 kernel/workqueue.c:2310
 worker_thread+0xaca/0x1280 kernel/workqueue.c:2457
 kthread+0x3f6/0x4f0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298
 </TASK>

Showing all locks held in the system:
3 locks held by kworker/1:0/21:
 #0: ffff888011c70938 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0x78a/0x10c0 kernel/workqueue.c:2283
 #1: ffffc90000db7d20 ((work_completion)(&(&nsim_dev->trap_data->trap_report_dw)->work)){+.+.}-{0:0}, at: process_one_work+0x7d0/0x10c0 kernel/workqueue.c:2285
 #2: ffff88807de62400 (&nsim_dev->port_list_lock){+.+.}-{3:3}, at: nsim_dev_trap_report_work+0x56/0xab0 drivers/net/netdevsim/dev.c:757
3 locks held by kworker/1:1/25:
 #0: ffff888011c70938 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0x78a/0x10c0 kernel/workqueue.c:2283
 #1: ffffc90000dffd20 (key_gc_work){+.+.}-{0:0}, at: process_one_work+0x7d0/0x10c0 kernel/workqueue.c:2285
 #2: ffffffff8c923ce8 (rcu_state.exp_mutex){+.+.}-{3:3}, at: exp_funnel_lock kernel/rcu/tree_exp.h:322 [inline]
 #2: ffffffff8c923ce8 (rcu_state.exp_mutex){+.+.}-{3:3}, at: synchronize_rcu_expedited+0x350/0x740 kernel/rcu/tree_exp.h:845
1 lock held by khungtaskd/27:
 #0: ffffffff8c91f720 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x0/0x30
4 locks held by kworker/u4:3/156:
 #0: ffff888011dcd138 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0x78a/0x10c0 kernel/workqueue.c:2283
 #1: ffffc90002047d20 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0x7d0/0x10c0 kernel/workqueue.c:2285
 #2: ffffffff8d9cfe50 (pernet_ops_rwsem){++++}-{3:3}, at: cleanup_net+0xf1/0xb60 net/core/net_namespace.c:558
 #3: ffffffff8d9dba08 (rtnl_mutex){+.+.}-{3:3}, at: netdev_run_todo+0xa4f/0xc40 net/core/dev.c:10652
2 locks held by getty/3261:
 #0: ffff8880242ee098 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x21/0x70 drivers/tty/tty_ldisc.c:252
 #1: ffffc9000209b2e8 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0x6af/0x1db0 drivers/tty/n_tty.c:2158
3 locks held by kworker/0:3/3518:
 #0: ffff888011c70938 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0x78a/0x10c0 kernel/workqueue.c:2283
 #1: ffffc90002ec7d20 ((linkwatch_work).work){+.+.}-{0:0}, at: process_one_work+0x7d0/0x10c0 kernel/workqueue.c:2285
 #2: ffffffff8d9dba08 (rtnl_mutex){+.+.}-{3:3}, at: linkwatch_event+0xa/0x50 net/core/link_watch.c:251
2 locks held by kworker/1:7/3565:
 #0: ffff888011c72138 ((wq_completion)rcu_gp){+.+.}-{0:0}, at: process_one_work+0x78a/0x10c0 kernel/workqueue.c:2283
 #1: ffffc900041b7d20 ((work_completion)(&rew.rew_work)){+.+.}-{0:0}, at: process_one_work+0x7d0/0x10c0 kernel/workqueue.c:2285
3 locks held by kworker/0:5/3567:
 #0: ffff88814ae0ed38 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: process_one_work+0x78a/0x10c0 kernel/workqueue.c:2283
 #1: ffffc900041d7d20 ((work_completion)(&(&ifa->dad_work)->work)){+.+.}-{0:0}, at: process_one_work+0x7d0/0x10c0 kernel/workqueue.c:2285
 #2: ffffffff8d9dba08 (rtnl_mutex){+.+.}-{3:3}, at: addrconf_dad_work+0xcc/0x1720 net/ipv6/addrconf.c:4111
4 locks held by kworker/u4:8/5562:
 #0: ffff888014db1938 ((wq_completion)writeback){+.+.}-{0:0}, at: process_one_work+0x78a/0x10c0 kernel/workqueue.c:2283
 #1: ffffc90003cb7d20 ((work_completion)(&(&wb->dwork)->work)){+.+.}-{0:0}, at: process_one_work+0x7d0/0x10c0 kernel/workqueue.c:2285
 #2: ffff88814b8160e0 (&type->s_umount_key#32){++++}-{3:3}, at: trylock_super+0x1b/0xf0 fs/super.c:418
 #3: ffff88814b8c8bd8 (&sbi->s_writepages_rwsem){.+.+}-{0:0}, at: ext4_writepages+0x1f6/0x3d10 fs/ext4/inode.c:2677
8 locks held by syz-executor.1/10642:
 #0: ffff888024b70460 (sb_writers#8){.+.+}-{0:0}, at: vfs_write+0x29a/0xe50 fs/read_write.c:590
 #1: ffff888079938888 (&of->mutex){+.+.}-{3:3}, at: kernfs_fop_write_iter+0x1e7/0x4f0 fs/kernfs/file.c:287
 #2: ffff888019bca748 (kn->active#233){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x20b/0x4f0 fs/kernfs/file.c:288
 #3: ffffffff8d357a88 (nsim_bus_dev_list_lock){+.+.}-{3:3}, at: del_device_store+0xf1/0x470 drivers/net/netdevsim/bus.c:344
 #4: ffff88807de61178 (&dev->mutex){....}-{3:3}, at: device_lock include/linux/device.h:760 [inline]
 #4: ffff88807de61178 (&dev->mutex){....}-{3:3}, at: __device_driver_lock drivers/base/dd.c:1044 [inline]
 #4: ffff88807de61178 (&dev->mutex){....}-{3:3}, at: device_release_driver_internal+0xc2/0x7f0 drivers/base/dd.c:1259
 #5: ffff88807de62400 (&nsim_dev->port_list_lock){+.+.}-{3:3}, at: nsim_dev_port_del_all drivers/net/netdevsim/dev.c:1359 [inline]
 #5: ffff88807de62400 (&nsim_dev->port_list_lock){+.+.}-{3:3}, at: nsim_dev_reload_destroy+0x118/0x240 drivers/net/netdevsim/dev.c:1561
 #6: ffffffff8d9dba08 (rtnl_mutex){+.+.}-{3:3}, at: nsim_destroy+0x3a/0x140 drivers/net/netdevsim/netdev.c:381
 #7: ffff8880b9b3a318 (&rq->__lock){-.-.}-{2:2}, at: raw_spin_rq_lock_nested+0x26/0x140 kernel/sched/core.c:475
1 lock held by syz-executor.2/10736:
1 lock held by syz-executor.3/11421:
 #0: ffffffff8d9dba08 (rtnl_mutex){+.+.}-{3:3}, at: netdev_run_todo+0xa4f/0xc40 net/core/dev.c:10652
2 locks held by syz-executor.0/11474:
 #0: ffffffff8d9cfe50 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x37b/0x5d0 net/core/net_namespace.c:469
 #1: ffffffff8d9dba08 (rtnl_mutex){+.+.}-{3:3}, at: netdev_run_todo+0xa4f/0xc40 net/core/dev.c:10652
1 lock held by syz-executor.2/12940:
1 lock held by syz-executor.4/12943:
3 locks held by syz-executor.4/13050:
2 locks held by syz-executor.4/13052:
1 lock held by syz-executor.2/13219:
1 lock held by syz-executor.2/13222:
2 locks held by syz-executor.4/13226:
2 locks held by syz-executor.4/13229:
1 lock held by syz-executor.2/13235:
1 lock held by syz-executor.4/13237:
2 locks held by syz-executor.2/13348:
2 locks held by syz-executor.2/13363:
2 locks held by syz-executor.4/13368:
3 locks held by syz-executor.2/13369:
3 locks held by syz-executor.4/13370:
2 locks held by syz-executor.4/13378:
3 locks held by syz-executor.4/13379:
2 locks held by syz-executor.2/13397:
1 lock held by syz-executor.4/13410:
1 lock held by syz-executor.4/13417:
1 lock held by syz-executor.2/13420:
1 lock held by syz-executor.2/13431:
1 lock held by syz-executor.2/13432:
1 lock held by syz-executor.4/13459:
1 lock held by syz-executor.4/13460:
2 locks held by syz-executor.4/13465:
1 lock held by syz-executor.4/13466:
1 lock held by syz-executor.2/13472:
4 locks held by syz-executor.2/14059:
1 lock held by syz-executor.4/14216:
1 lock held by syz-executor.4/14218:
1 lock held by syz-executor.4/14225:
1 lock held by syz-executor.4/14226:
1 lock held by syz-executor.4/14243:
2 locks held by syz-executor.2/14508:
 #0: ffffffff8d9cfe50 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x37b/0x5d0 net/core/net_namespace.c:469
 #1: ffffffff8d9dba08 (rtnl_mutex){+.+.}-{3:3}, at: register_nexthop_notifier+0x80/0x270 net/ipv4/nexthop.c:3606
2 locks held by syz-executor.3/14674:
 #0: ffffffff8d9cfe50 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x37b/0x5d0 net/core/net_namespace.c:469
 #1: ffffffff8d9dba08 (rtnl_mutex){+.+.}-{3:3}, at: register_netdev+0xf/0x50 net/core/dev.c:10476
2 locks held by syz-executor.0/14866:
 #0: ffffffff8d9cfe50 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x37b/0x5d0 net/core/net_namespace.c:469
 #1: ffffffff8d9dba08 (rtnl_mutex){+.+.}-{3:3}, at: ip_tunnel_init_net+0x20f/0x700 net/ipv4/ip_tunnel.c:1083
1 lock held by syz-executor.1/15583:
 #0: ffffffff8d9dba08 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock net/core/rtnetlink.c:72 [inline]
 #0: ffffffff8d9dba08 (rtnl_mutex){+.+.}-{3:3}, at: rtnetlink_rcv_msg+0x94c/0xee0 net/core/rtnetlink.c:5626
2 locks held by syz-executor.4/15590:
1 lock held by dhcpcd/15594:
 #0: ffff888057e6e120 (sk_lock-AF_PACKET){+.+.}-{0:0}, at: lock_sock include/net/sock.h:1668 [inline]
 #0: ffff888057e6e120 (sk_lock-AF_PACKET){+.+.}-{0:0}, at: packet_do_bind+0x33/0xd50 net/packet/af_packet.c:3164
3 locks held by syz-executor.4/15595:

=============================================

NMI backtrace for cpu 0
CPU: 0 PID: 27 Comm: khungtaskd Not tainted 5.15.152-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 02/29/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e3/0x2cb lib/dump_stack.c:106
 nmi_cpu_backtrace+0x46a/0x4a0 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x181/0x2a0 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:148 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:210 [inline]
 watchdog+0xe72/0xeb0 kernel/hung_task.c:295
 kthread+0x3f6/0x4f0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298
 </TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 9 Comm: kworker/u4:0 Not tainted 5.15.152-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 02/29/2024
Workqueue: phy30 ieee80211_iface_work
RIP: 0010:__lock_acquire+0x6d/0x1ff0 kernel/locking/lockdep.c:4882
Code: 7c 24 08 0f 85 3f 16 00 00 83 3d 31 2f 81 0c 00 0f 84 ec 14 00 00 83 3d a0 55 1c 0b 00 74 2d 48 89 f8 48 c1 e8 03 80 3c 10 00 <74> 14 e8 0c 64 67 00 48 8b 7c 24 08 48 ba 00 00 00 00 00 fc ff df
RSP: 0018:ffffc90000ce7700 EFLAGS: 00000046
RAX: 1ffffffff22b4a8c RBX: 0000000000000000 RCX: 0000000000000000
RDX: dffffc0000000000 RSI: 0000000000000000 RDI: ffffffff915a5460
RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000001
R10: 0000000000000000 R11: dffffc0000000001 R12: 0000000000000001
R13: ffff88813fe90000 R14: 0000000000000000 R15: 0000000000000001
FS:  0000000000000000(0000) GS:ffff8880b9b00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb66dae0000 CR3: 0000000079e68000 CR4: 00000000003506e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <TASK>
 lock_acquire+0x1db/0x4f0 kernel/locking/lockdep.c:5623
 __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
 _raw_spin_lock_irqsave+0xd1/0x120 kernel/locking/spinlock.c:162
 __debug_check_no_obj_freed lib/debugobjects.c:961 [inline]
 debug_check_no_obj_freed+0xd8/0x4e0 lib/debugobjects.c:1002
 slab_free_hook mm/slub.c:1680 [inline]
 slab_free_freelist_hook+0x86/0x160 mm/slub.c:1731
 slab_free mm/slub.c:3499 [inline]
 kfree+0xf1/0x270 mm/slub.c:4559
 skb_free_head net/core/skbuff.c:655 [inline]
 skb_release_data+0x73a/0x8a0 net/core/skbuff.c:677
 skb_release_all net/core/skbuff.c:742 [inline]
 __kfree_skb net/core/skbuff.c:756 [inline]
 kfree_skb_reason+0xb3/0x170 net/core/skbuff.c:776
 kfree_skb include/linux/skbuff.h:1123 [inline]
 ieee80211_iface_work+0x251/0xcc0 net/mac80211/iface.c:1497
 process_one_work+0x8a1/0x10c0 kernel/workqueue.c:2310
 worker_thread+0xaca/0x1280 kernel/workqueue.c:2457
 kthread+0x3f6/0x4f0 kernel/kthread.c:319
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298
 </TASK>

Crashes (6):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/03/20 19:09 linux-5.15.y b95c01af2113 5b7d42ae .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: task hung in nsim_dev_trap_report_work
2024/03/09 21:26 linux-5.15.y 574362648507 6ee49f2e .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: task hung in nsim_dev_trap_report_work
2024/03/06 14:26 linux-5.15.y 80efc6265290 f39a7eed .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: task hung in nsim_dev_trap_report_work
2024/02/25 12:09 linux-5.15.y 458ce51d0356 8d446f15 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: task hung in nsim_dev_trap_report_work
2024/02/24 07:21 linux-5.15.y 458ce51d0356 8d446f15 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: task hung in nsim_dev_trap_report_work
2024/02/19 09:21 linux-5.15.y 6139f2a02fe0 578f7538 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: task hung in nsim_dev_trap_report_work
* Struck through repros no longer work on HEAD.