syzbot


INFO: task hung in devlink_free

Status: auto-obsoleted due to no activity on 2023/08/05 16:02
Subsystems: net
[Documentation on labels]
First crash: 715d, last: 621d
Cause bisection: failed (error log, bisect log)
  
Fix bisection: fixed by (bisect log) :
commit e93c9378e33f68b61ea9318580d841caa22fb9ea
Author: Jiri Pirko <jiri@nvidia.com>
Date: Wed May 10 14:46:21 2023 +0000

  devlink: change per-devlink netdev notifier to static one

  
Last patch testing requests (1)
Created Duration User Patch Repo Result
2023/07/27 20:37 58m retest repro net-next OK log

Sample crash report:
INFO: task syz-executor.0:5321 blocked for more than 143 seconds.
      Not tainted 6.2.0-syzkaller-05251-g5b7c4cabbb65 #0
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:syz-executor.0  state:D stack:23936 pid:5321  ppid:1      flags:0x00004004
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5304 [inline]
 __schedule+0xcce/0x5b20 kernel/sched/core.c:6622
 schedule+0xde/0x1a0 kernel/sched/core.c:6698
 schedule_preempt_disabled+0x13/0x20 kernel/sched/core.c:6757
 rwsem_down_write_slowpath+0x3e2/0x1220 kernel/locking/rwsem.c:1178
 __down_write_common kernel/locking/rwsem.c:1306 [inline]
 __down_write kernel/locking/rwsem.c:1315 [inline]
 down_write+0x1d2/0x200 kernel/locking/rwsem.c:1574
 unregister_netdevice_notifier+0x22/0x180 net/core/dev.c:1759
 devlink_free+0x2cb/0x460 net/devlink/core.c:269
 nsim_drv_remove+0x158/0x1d0 drivers/net/netdevsim/dev.c:1687
 device_remove+0xc8/0x170 drivers/base/dd.c:548
 __device_release_driver drivers/base/dd.c:1253 [inline]
 device_release_driver_internal+0x4a5/0x700 drivers/base/dd.c:1279
 bus_remove_device+0x2e7/0x5a0 drivers/base/bus.c:529
 device_del+0x4f7/0xc80 drivers/base/core.c:3624
 device_unregister+0x1e/0xc0 drivers/base/core.c:3656
 nsim_bus_dev_del drivers/net/netdevsim/bus.c:310 [inline]
 del_device_store+0x34e/0x4a0 drivers/net/netdevsim/bus.c:219
 bus_attr_store+0x76/0xa0 drivers/base/bus.c:122
 sysfs_kf_write+0x114/0x170 fs/sysfs/file.c:136
 kernfs_fop_write_iter+0x3f1/0x600 fs/kernfs/file.c:334
 call_write_iter include/linux/fs.h:1849 [inline]
 new_sync_write fs/read_write.c:491 [inline]
 vfs_write+0x9ed/0xe10 fs/read_write.c:584
 ksys_write+0x12b/0x250 fs/read_write.c:637
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7f79fe03de7f
RSP: 002b:00007ffd3d5d4550 EFLAGS: 00000293 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f79fe03de7f
RDX: 0000000000000001 RSI: 00007ffd3d5d45a0 RDI: 0000000000000005
RBP: 0000000000000005 R08: 0000000000000000 R09: 00007ffd3d5d44f0
R10: 0000000000000000 R11: 0000000000000293 R12: 00007f79fe0e76fe
R13: 00007ffd3d5d45a0 R14: 0000000000000000 R15: 00007ffd3d5d4c70
 </TASK>

Showing all locks held in the system:
1 lock held by rcu_tasks_kthre/12:
 #0: ffffffff8c795f30 (rcu_tasks.tasks_gp_mutex){+.+.}-{3:3}, at: rcu_tasks_one_gp+0x31/0xd80 kernel/rcu/tasks.h:510
1 lock held by rcu_tasks_trace/13:
 #0: ffffffff8c795c30 (rcu_tasks_trace.tasks_gp_mutex){+.+.}-{3:3}, at: rcu_tasks_one_gp+0x31/0xd80 kernel/rcu/tasks.h:510
1 lock held by khungtaskd/28:
 #0: ffffffff8c796a80 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x55/0x340 kernel/locking/lockdep.c:6495
2 locks held by kworker/u4:5/2753:
2 locks held by getty/4760:
 #0: ffff888027e72098 (&tty->ldisc_sem){++++}-{0:0}, at: tty_ldisc_ref_wait+0x26/0x80 drivers/tty/tty_ldisc.c:244
 #1: ffffc900015a02f0 (&ldata->atomic_read_lock){+.+.}-{3:3}, at: n_tty_read+0xef4/0x13e0 drivers/tty/n_tty.c:2177
4 locks held by kworker/u4:2/5206:
6 locks held by syz-executor.0/5321:
 #0: ffff888027940460 (sb_writers#8){.+.+}-{0:0}, at: ksys_write+0x12b/0x250 fs/read_write.c:637
 #1: ffff888021c7dc88 (&of->mutex){+.+.}-{3:3}, at: kernfs_fop_write_iter+0x285/0x600 fs/kernfs/file.c:325
 #2: ffff888146ecea00 (kn->active#51){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x2a9/0x600 fs/kernfs/file.c:326
 #3: ffffffff8d7d9408 (nsim_bus_dev_list_lock){+.+.}-{3:3}, at: del_device_store+0xc9/0x4a0 drivers/net/netdevsim/bus.c:209
 #4: ffff88806fa450e8 (&dev->mutex){....}-{3:3}, at: device_lock include/linux/device.h:831 [inline]
 #4: ffff88806fa450e8 (&dev->mutex){....}-{3:3}, at: __device_driver_lock drivers/base/dd.c:1073 [inline]
 #4: ffff88806fa450e8 (&dev->mutex){....}-{3:3}, at: device_release_driver_internal+0xa4/0x700 drivers/base/dd.c:1276
 #5: ffffffff8e0de950 (pernet_ops_rwsem){++++}-{3:3}, at: unregister_netdevice_notifier+0x22/0x180 net/core/dev.c:1759
1 lock held by syz-executor.0/5457:
 #0: ffffffff8e0de950 (pernet_ops_rwsem){++++}-{3:3}, at: copy_net_ns+0x4cb/0x8e0 net/core/net_namespace.c:486

=============================================

NMI backtrace for cpu 1
CPU: 1 PID: 28 Comm: khungtaskd Not tainted 6.2.0-syzkaller-05251-g5b7c4cabbb65 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/21/2023
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xd9/0x150 lib/dump_stack.c:106
 nmi_cpu_backtrace+0x316/0x3e0 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x333/0x440 lib/nmi_backtrace.c:62
 trigger_all_cpu_backtrace include/linux/nmi.h:148 [inline]
 check_hung_uninterruptible_tasks kernel/hung_task.c:220 [inline]
 watchdog+0xe53/0x11b0 kernel/hung_task.c:377
 kthread+0x2e8/0x3a0 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
 </TASK>
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 5206 Comm: kworker/u4:2 Not tainted 6.2.0-syzkaller-05251-g5b7c4cabbb65 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/21/2023
Workqueue: netns cleanup_net
RIP: 0010:get_next_corpse net/netfilter/nf_conntrack_core.c:2362 [inline]
RIP: 0010:nf_ct_iterate_cleanup+0xa2/0x500 net/netfilter/nf_conntrack_core.c:2415
Code: eb 77 27 e9 5a 03 00 00 e8 9b 10 32 f9 8b 1d c1 0f 25 06 41 83 c5 01 44 89 ef 89 de e8 a7 0c 32 f9 41 39 dd 0f 83 38 03 00 00 <e8> 79 10 32 f9 48 8b 15 b2 0f 25 06 44 89 e8 48 8d 1c c2 48 89 d8
RSP: 0018:ffffc9000455fab8 EFLAGS: 00000283
RAX: 0000000000000000 RBX: 0000000000040000 RCX: ffffffff88529da9
RDX: ffff8880203b3a80 RSI: 0000000000000000 RDI: 0000000000000004
RBP: dffffc0000000000 R08: 0000000000000004 R09: 000000000003c257
R10: 0000000000040000 R11: 0000000000000000 R12: ffffc9000455fb90
R13: 000000000003c257 R14: 0000000000000001 R15: 0000000000000400
FS:  0000000000000000(0000) GS:ffff8880b9800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000558f3cf77680 CR3: 000000002c2e9000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <TASK>
 nf_ct_iterate_cleanup_net+0xdb/0x110 net/netfilter/nf_conntrack_core.c:2436
 nf_conntrack_cleanup_net_list+0x1cc/0x5d0 net/netfilter/nf_conntrack_core.c:2549
 ops_exit_list+0x125/0x170 net/core/net_namespace.c:174
 cleanup_net+0x4ee/0xb10 net/core/net_namespace.c:613
 process_one_work+0x9bf/0x1820 kernel/workqueue.c:2390
 worker_thread+0x669/0x1090 kernel/workqueue.c:2537
 kthread+0x2e8/0x3a0 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308
 </TASK>

Crashes (134):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2023/02/23 14:45 net-next-old 5b7c4cabbb65 9e2ebb3c .config console log report syz [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in devlink_free
2023/05/18 20:31 upstream 4d6d4c7f541d 3bb7af1d .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: task hung in devlink_free
2023/05/08 11:13 upstream ac9a78681b92 90c93c40 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: task hung in devlink_free
2023/05/06 22:06 upstream 994e2419f1e7 90c93c40 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: task hung in devlink_free
2023/04/30 13:32 upstream 1ae78a14516b 62df2017 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: task hung in devlink_free
2023/04/30 05:16 upstream 1ae78a14516b 62df2017 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: task hung in devlink_free
2023/04/29 02:06 upstream 22b8cc3e78f5 62df2017 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root INFO: task hung in devlink_free
2023/04/28 12:31 upstream 33afd4b76393 457a6e0a .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-selinux-root INFO: task hung in devlink_free
2023/04/26 06:49 upstream 4173cf6fb6b7 7560799c .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: task hung in devlink_free
2023/04/25 12:40 upstream 173ea743bf7a 65320f8e .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: task hung in devlink_free
2023/04/24 16:42 upstream 457391b03803 fdc18293 .config console log report info ci-upstream-kasan-gce-selinux-root INFO: task hung in devlink_free
2023/04/24 14:23 upstream 457391b03803 fdc18293 .config console log report info ci-upstream-kasan-gce-smack-root INFO: task hung in devlink_free
2023/04/24 00:37 upstream 457391b03803 2b32bd34 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: task hung in devlink_free
2023/04/22 21:14 upstream 2caeeb9d4a1b 2b32bd34 .config console log report info ci-upstream-kasan-gce-smack-root INFO: task hung in devlink_free
2023/04/22 12:22 upstream 8e41e0a57566 2b32bd34 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: task hung in devlink_free
2023/04/22 08:39 upstream 8e41e0a57566 2b32bd34 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: task hung in devlink_free
2023/04/22 07:26 upstream 8e41e0a57566 2b32bd34 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: task hung in devlink_free
2023/04/20 19:26 upstream cb0856346a60 2b32bd34 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-root INFO: task hung in devlink_free
2023/04/18 05:13 upstream 6a8f57ae2eb0 436577a9 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: task hung in devlink_free
2023/04/15 05:39 upstream 95abc817ab3a ec410564 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce INFO: task hung in devlink_free
2023/04/14 03:27 upstream 44149752e998 3cfcaa1b .config console log report info ci-upstream-kasan-gce-smack-root INFO: task hung in devlink_free
2023/04/11 20:21 upstream 2c40519251d6 49faf98d .config console log report info ci-upstream-kasan-gce INFO: task hung in devlink_free
2023/04/10 23:28 upstream 0d3eb744aed4 71147e29 .config console log report info ci-upstream-kasan-gce INFO: task hung in devlink_free
2023/04/02 01:53 upstream 00c7b5f4ddc5 f325deb0 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-386 INFO: task hung in devlink_free
2023/04/28 02:14 net 075cafffce24 70a605de .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in devlink_free
2023/04/24 22:54 net 60fd497c99b3 fdc18293 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in devlink_free
2023/04/18 03:00 net 338469d677e5 436577a9 .config console log report info ci-upstream-net-this-kasan-gce INFO: task hung in devlink_free
2023/04/17 11:42 net 853618d5886b c6ec7083 .config console log report info ci-upstream-net-this-kasan-gce INFO: task hung in devlink_free
2023/04/16 03:58 net 3037933448f6 ec410564 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in devlink_free
2023/04/15 07:08 net 3037933448f6 ec410564 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in devlink_free
2023/04/15 00:34 net 3037933448f6 ec410564 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-this-kasan-gce INFO: task hung in devlink_free
2023/04/09 14:17 net 5cc33f139e11 71147e29 .config console log report info ci-upstream-net-this-kasan-gce INFO: task hung in devlink_free
2023/04/09 12:39 net 5cc33f139e11 71147e29 .config console log report info ci-upstream-net-this-kasan-gce INFO: task hung in devlink_free
2023/05/17 16:53 net-next 833e24aeb4d9 eaac4681 .config console log report info ci-upstream-net-kasan-gce INFO: task hung in devlink_free
2023/05/02 15:17 net-next 042334a8d424 48e0a81d .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in devlink_free
2023/04/27 04:37 net-next 9b78d919632b 19a3dabe .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in devlink_free
2023/04/27 02:54 net-next 9b78d919632b 19a3dabe .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in devlink_free
2023/04/26 18:27 net-next c248b27cfc0a 19a3dabe .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in devlink_free
2023/04/26 16:20 net-next c248b27cfc0a 19a3dabe .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in devlink_free
2023/04/26 11:00 net-next c248b27cfc0a 7560799c .config console log report info ci-upstream-net-kasan-gce INFO: task hung in devlink_free
2023/04/25 06:00 net-next 938f65adc420 fdc18293 .config console log report info ci-upstream-net-kasan-gce INFO: task hung in devlink_free
2023/04/23 23:20 net-next 4eaeca1fc43a 2b32bd34 .config console log report info ci-upstream-net-kasan-gce INFO: task hung in devlink_free
2023/04/23 08:23 net-next fbc1449d385d 2b32bd34 .config console log report info ci-upstream-net-kasan-gce INFO: task hung in devlink_free
2023/04/23 06:37 net-next fbc1449d385d 2b32bd34 .config console log report info ci-upstream-net-kasan-gce INFO: task hung in devlink_free
2023/04/21 17:21 net-next dfc39d4026fb 2b32bd34 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in devlink_free
2023/04/21 13:43 net-next e315e7b83a22 2b32bd34 .config console log report info ci-upstream-net-kasan-gce INFO: task hung in devlink_free
2023/04/17 09:10 net-next 0af03871b6af c6ec7083 .config console log report info ci-upstream-net-kasan-gce INFO: task hung in devlink_free
2023/04/16 22:20 net-next e61caf04b9f8 ec410564 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-net-kasan-gce INFO: task hung in devlink_free
2023/04/13 10:12 net-next f2b3b6a22df7 82d5e53e .config console log report info ci-upstream-net-kasan-gce INFO: task hung in devlink_free
2023/04/11 15:52 net-next 4de00f0acc72 49faf98d .config console log report info ci-upstream-net-kasan-gce INFO: task hung in devlink_free
2023/04/10 09:58 net-next 490fde262f17 71147e29 .config console log report info ci-upstream-net-kasan-gce INFO: task hung in devlink_free
2023/05/06 18:09 linux-next 83e5775d7afd 90c93c40 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in devlink_free
2023/04/22 00:07 linux-next d3e1ee0e67e7 2b32bd34 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-linux-next-kasan-gce-root INFO: task hung in devlink_free
2023/04/28 23:52 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 14f8db1c0f9a 62df2017 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-gce-arm64 INFO: task hung in devlink_free
2023/04/13 14:29 git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci 7920df21c1b7 3cfcaa1b .config console log report info ci-upstream-gce-arm64 INFO: task hung in devlink_free
* Struck through repros no longer work on HEAD.