syzbot


BUG: soft lockup in tx

Status: upstream: reported C repro on 2021/10/09 08:02
Reported-by: syzbot+47baabaa396befe0fb36@syzkaller.appspotmail.com
First crash: 923d, last: 744d
Fix bisection: failed (error log, bisect log)
  
Similar bugs (8)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-6.1 BUG: soft lockup in tx (2) 1 76d 76d 0/3 upstream: reported on 2024/02/02 18:08
upstream BUG: soft lockup in tx serial C 63 8d19h 579d 0/26 upstream: reported C repro on 2022/09/18 05:29
linux-4.19 BUG: soft lockup in tx C done 1 959d 989d 1/1 fixed on 2021/10/04 07:44
linux-6.1 BUG: soft lockup in tx 3 238d 361d 0/3 auto-obsoleted due to no activity on 2023/12/03 12:31
linux-5.15 BUG: soft lockup in tx 21 123d 375d 0/3 auto-obsoleted due to no activity on 2024/03/27 03:04
linux-5.15 INFO: rcu detected stall in tx 1 7d01h 7d01h 0/3 upstream: reported on 2024/04/12 14:07
upstream INFO: rcu detected stall in tx net C error unreliable 3 958d 1096d 20/26 fixed on 2021/11/10 00:50
upstream INFO: rcu detected stall in tx (2) serial 79 724d 843d 0/26 auto-closed as invalid on 2022/08/24 07:21
Last patch testing requests (4)
Created Duration User Patch Repo Result
2023/02/26 22:32 11m retest repro linux-4.14.y report log
2023/02/26 21:32 10m retest repro linux-4.14.y report log
2022/11/08 03:30 10m retest repro linux-4.14.y report log
2022/11/08 02:30 9m retest repro linux-4.14.y report log
Fix bisection attempts (6)
Created Duration User Patch Repo Result
2022/05/06 08:17 12m bisect fix linux-4.14.y error job log (0)
2022/04/06 07:56 21m bisect fix linux-4.14.y job log (0) log
2022/03/07 07:35 20m bisect fix linux-4.14.y job log (0) log
2022/02/05 07:10 24m bisect fix linux-4.14.y job log (0) log
2021/12/08 08:26 25m bisect fix linux-4.14.y job log (0) log
2021/11/08 08:01 24m bisect fix linux-4.14.y job log (0) log

Sample crash report:
watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [aoe_tx0:3071]
Modules linked in:
irq event stamp: 22103
hardirqs last  enabled at (22102): [<ffffffff87400976>] restore_regs_and_return_to_kernel+0x0/0x2a
hardirqs last disabled at (22103): [<ffffffff874018ae>] apic_timer_interrupt+0x8e/0xa0 arch/x86/entry/entry_64.S:793
softirqs last  enabled at (1314): [<ffffffff85c45926>] rcu_read_unlock_bh include/linux/rcupdate.h:721 [inline]
softirqs last  enabled at (1314): [<ffffffff85c45926>] __dev_queue_xmit+0xc96/0x2480 net/core/dev.c:3548
softirqs last disabled at (1324): [<ffffffff85c44e3c>] __dev_queue_xmit+0x1ac/0x2480 net/core/dev.c:3455
CPU: 1 PID: 3071 Comm: aoe_tx0 Not tainted 4.14.261-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
task: ffff8880ab6d4500 task.stack: ffff8880ab780000
RIP: 0010:__sanitizer_cov_trace_pc+0x28/0x50 kernel/kcov.c:93
RSP: 0018:ffff8880ab787b20 EFLAGS: 00000297 ORIG_RAX: ffffffffffffff10
RAX: ffff8880ab6d4500 RBX: ffff8880b60906b8 RCX: ffff8880b60906a8
RDX: 0000000000000000 RSI: ffff8880b60906b8 RDI: ffff88808d460558
RBP: ffff8880b60906c0 R08: 0000000000000001 R09: 000000000004850e
R10: ffff8880ab6d4dd8 R11: ffff8880ab6d4500 R12: ffff8880b6090480
R13: dffffc0000000000 R14: ffff8880b6090480 R15: ffff88808d460550
FS:  0000000000000000(0000) GS:ffff8880ba500000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff04d9501d0 CR3: 0000000008e6a000 CR4: 00000000003406e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 __read_once_size include/linux/compiler.h:185 [inline]
 list_empty include/linux/list.h:203 [inline]
 fq_codel_dequeue+0x5a/0x2c10 net/sched/sch_fq_codel.c:296
 dequeue_skb net/sched/sch_generic.c:148 [inline]
 qdisc_restart net/sched/sch_generic.c:241 [inline]
 __qdisc_run+0x322/0xe00 net/sched/sch_generic.c:257
 __dev_xmit_skb net/core/dev.c:3231 [inline]
 __dev_queue_xmit+0x13ac/0x2480 net/core/dev.c:3489
 tx+0x68/0xa0 drivers/block/aoe/aoenet.c:63
 kthread+0x1b9/0x350 drivers/block/aoe/aoecmd.c:1259
 kthread+0x30d/0x420 kernel/kthread.c:232
 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:404
Code: 90 90 90 65 48 8b 04 25 c0 7f 02 00 48 85 c0 74 1a 65 8b 15 0b 44 ad 7e 81 e2 00 01 1f 00 75 0b 8b 90 58 13 00 00 83 fa 01 74 01 <c3> 48 8b 34 24 48 8b 88 60 13 00 00 8b 80 5c 13 00 00 48 8b 11 
----------------
Code disassembly (best guess):
   0:	90                   	nop
   1:	90                   	nop
   2:	90                   	nop
   3:	65 48 8b 04 25 c0 7f 	mov    %gs:0x27fc0,%rax
   a:	02 00
   c:	48 85 c0             	test   %rax,%rax
   f:	74 1a                	je     0x2b
  11:	65 8b 15 0b 44 ad 7e 	mov    %gs:0x7ead440b(%rip),%edx        # 0x7ead4423
  18:	81 e2 00 01 1f 00    	and    $0x1f0100,%edx
  1e:	75 0b                	jne    0x2b
  20:	8b 90 58 13 00 00    	mov    0x1358(%rax),%edx
  26:	83 fa 01             	cmp    $0x1,%edx
  29:	74 01                	je     0x2c
* 2b:	c3                   	retq <-- trapping instruction
  2c:	48 8b 34 24          	mov    (%rsp),%rsi
  30:	48 8b 88 60 13 00 00 	mov    0x1360(%rax),%rcx
  37:	8b 80 5c 13 00 00    	mov    0x135c(%rax),%eax
  3d:	48 8b 11             	mov    (%rcx),%rdx

Crashes (2):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2022/01/06 07:10 linux-4.14.y bfdef05c8da4 6acc789a .config console log report syz C ci2-linux-4-14 BUG: soft lockup in tx
2021/10/09 08:01 linux-4.14.y 756db2ba8bde efe0f24d .config console log report syz C ci2-linux-4-14 BUG: soft lockup in tx
* Struck through repros no longer work on HEAD.