syzbot


BUG: sleeping function called from invalid context in lock_sock_nested

Status: upstream: reported C repro on 2024/06/29 07:50
Bug presence: origin:upstream
[Documentation on labels]
Reported-by: syzbot+a6a36c56b601cd460922@syzkaller.appspotmail.com
First crash: 70d, last: 1d09h
Fix bisection: failed (error log, bisect log)
  
Bug presence (1)
Date Name Commit Repro Result
2024/07/19 upstream (ToT) 12cc3d5389f3 C [report] BUG: sleeping function called from invalid context in lock_sock_nested
Similar bugs (21)
Kernel Title Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.19 BUG: sleeping function called from invalid context in lock_sock_nested (2) C done 3909 1092d 1185d 1/1 fixed on 2021/10/12 13:38
upstream BUG: sleeping function called from invalid context in lock_sock_nested (2) bluetooth C done error 19391 1123d 1659d 20/27 fixed on 2021/11/10 13:22
linux-4.14 BUG: sleeping function called from invalid context in lock_sock_nested 1 1894d 1894d 0/1 auto-closed as invalid on 2019/10/30 11:24
linux-4.14 BUG: sleeping function called from invalid context in lock_sock_nested (2) syz done 1 1678d 1738d 1/1 fixed on 2020/03/04 10:17
linux-4.19 BUG: sleeping function called from invalid context in lock_sock_nested syz done 1 1708d 1738d 1/1 fixed on 2020/02/05 13:33
linux-6.1 BUG: sleeping function called from invalid context in lock_sock_nested 7 49d 70d 0/3 upstream: reported on 2024/06/29 07:54
upstream BUG: sleeping function called from invalid context in lock_sock_nested bpf net C 1232 1663d 1738d 15/27 fixed on 2020/02/18 14:31
linux-4.14 BUG: sleeping function called from invalid context in lock_sock_nested (3) syz error 144 679d 1185d 0/1 upstream: reported syz repro on 2021/06/10 20:33
upstream BUG: sleeping function called from invalid context in lock_sock_nested (3) bluetooth C inconclusive 50 1d20h 117d 0/27 upstream: reported C repro on 2024/05/13 12:58
upstream BUG: unable to handle kernel paging request in lock_sock_nested bluetooth 43 1054d 1494d 0/27 auto-closed as invalid on 2022/02/16 22:16
linux-4.19 KASAN: wild-memory-access Write in lock_sock_nested 2 1318d 1387d 0/1 auto-closed as invalid on 2021/05/28 14:35
linux-4.14 general protection fault in lock_sock_nested 4 1219d 1437d 0/1 auto-closed as invalid on 2021/09/04 19:35
linux-4.19 KASAN: use-after-free Read in lock_sock_nested C 471 556d 1870d 0/1 upstream: reported C repro on 2019/07/26 21:27
linux-4.14 KASAN: use-after-free Read in lock_sock_nested C inconclusive 331 597d 1963d 0/1 upstream: reported C repro on 2019/04/24 06:28
upstream KASAN: use-after-free Read in lock_sock_nested hams C inconclusive done 1856 466d 2075d 0/27 auto-obsoleted due to no activity on 2023/08/23 09:06
upstream general protection fault in lock_sock_nested bluetooth C done done 288 7d13h 362d 0/27 upstream: reported C repro on 2023/09/11 07:52
linux-5.15 general protection fault in lock_sock_nested origin:upstream missing-backport syz error 37 41d 156d 0/3 upstream: reported syz repro on 2024/04/04 13:25
linux-4.14 BUG: unable to handle kernel paging request in lock_sock_nested 4 1212d 1331d 0/1 auto-closed as invalid on 2021/09/11 11:51
linux-6.1 general protection fault in lock_sock_nested 47 4d16h 146d 0/3 upstream: reported on 2024/04/14 08:46
upstream KASAN: slab-out-of-bounds Read in lock_sock_nested bluetooth syz unreliable done 23 1048d 1488d 0/27 auto-obsoleted due to no activity on 2022/09/29 10:19
linux-4.19 KASAN: slab-out-of-bounds Read in lock_sock_nested 14 731d 1395d 0/1 auto-obsoleted due to no activity on 2023/01/05 15:59

Sample crash report:
BUG: sleeping function called from invalid context at net/core/sock.c:3252
in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 144, name: kworker/u5:0
6 locks held by kworker/u5:0/144:
 #0: ffff0000c8c58938 ((wq_completion)hci0#2){+.+.}-{0:0}, at: process_one_work+0x66c/0x11b8 kernel/workqueue.c:2283
 #1: ffff80001a3e7c00 ((work_completion)(&hdev->rx_work)){+.+.}-{0:0}, at: process_one_work+0x6ac/0x11b8 kernel/workqueue.c:2285
 #2: ffff0000c89ec078 (&hdev->lock){+.+.}-{3:3}, at: hci_sync_conn_complete_evt+0xb4/0x8c0 net/bluetooth/hci_event.c:4471
 #3: ffff800016b8af68 (hci_cb_list_lock){+.+.}-{3:3}, at: hci_connect_cfm include/net/bluetooth/hci_core.h:1507 [inline]
 #3: ffff800016b8af68 (hci_cb_list_lock){+.+.}-{3:3}, at: hci_sync_conn_complete_evt+0x404/0x8c0 net/bluetooth/hci_event.c:4551
 #4: ffff0000c0ff3c20 (&conn->lock#2){+.+.}-{2:2}, at: spin_lock include/linux/spinlock.h:363 [inline]
 #4: ffff0000c0ff3c20 (&conn->lock#2){+.+.}-{2:2}, at: sco_conn_ready net/bluetooth/sco.c:1116 [inline]
 #4: ffff0000c0ff3c20 (&conn->lock#2){+.+.}-{2:2}, at: sco_connect_cfm+0x260/0x908 net/bluetooth/sco.c:1201
 #5: ffff0000c9219120 (sk_lock-AF_BLUETOOTH-BTPROTO_SCO){+.+.}-{0:0}, at: lock_sock include/net/sock.h:1676 [inline]
 #5: ffff0000c9219120 (sk_lock-AF_BLUETOOTH-BTPROTO_SCO){+.+.}-{0:0}, at: sco_conn_ready net/bluetooth/sco.c:1129 [inline]
 #5: ffff0000c9219120 (sk_lock-AF_BLUETOOTH-BTPROTO_SCO){+.+.}-{0:0}, at: sco_connect_cfm+0x3e8/0x908 net/bluetooth/sco.c:1201
Preemption disabled at:
[<ffff800010d80660>] spin_lock include/linux/spinlock.h:363 [inline]
[<ffff800010d80660>] sco_conn_ready net/bluetooth/sco.c:1116 [inline]
[<ffff800010d80660>] sco_connect_cfm+0x260/0x908 net/bluetooth/sco.c:1201
CPU: 0 PID: 144 Comm: kworker/u5:0 Not tainted 5.15.163-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/07/2024
Workqueue: hci0 hci_rx_work
Call trace:
 dump_backtrace+0x0/0x530 arch/arm64/kernel/stacktrace.c:152
 show_stack+0x2c/0x3c arch/arm64/kernel/stacktrace.c:216
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x108/0x170 lib/dump_stack.c:106
 dump_stack+0x1c/0x58 lib/dump_stack.c:113
 ___might_sleep+0x380/0x4dc kernel/sched/core.c:9626
 __might_sleep+0x98/0xf0 kernel/sched/core.c:9580
 lock_sock_nested+0x100/0x1ec net/core/sock.c:3252
 lock_sock include/net/sock.h:1676 [inline]
 sco_conn_ready net/bluetooth/sco.c:1129 [inline]
 sco_connect_cfm+0x3e8/0x908 net/bluetooth/sco.c:1201
 hci_connect_cfm include/net/bluetooth/hci_core.h:1510 [inline]
 hci_sync_conn_complete_evt+0x468/0x8c0 net/bluetooth/hci_event.c:4551
 hci_event_packet+0x8e0/0x12b4 net/bluetooth/hci_event.c:6483
 hci_rx_work+0x1c0/0x7c4 net/bluetooth/hci_core.c:5156
 process_one_work+0x790/0x11b8 kernel/workqueue.c:2310
 worker_thread+0x910/0x1034 kernel/workqueue.c:2457
 kthread+0x37c/0x45c kernel/kthread.c:334
 ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:870
==================================================================
BUG: KASAN: use-after-free in __lock_acquire+0x114/0x7638 kernel/locking/lockdep.c:4882
Read of size 8 at addr ffff0000c92190a0 by task kworker/u5:0/144

CPU: 0 PID: 144 Comm: kworker/u5:0 Tainted: G        W         5.15.163-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/07/2024
Workqueue: hci0 hci_rx_work
Call trace:
 dump_backtrace+0x0/0x530 arch/arm64/kernel/stacktrace.c:152
 show_stack+0x2c/0x3c arch/arm64/kernel/stacktrace.c:216
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x108/0x170 lib/dump_stack.c:106
 print_address_description+0x7c/0x3f0 mm/kasan/report.c:248
 __kasan_report mm/kasan/report.c:434 [inline]
 kasan_report+0x174/0x1e4 mm/kasan/report.c:451
 __asan_report_load8_noabort+0x44/0x50 mm/kasan/report_generic.c:309
 __lock_acquire+0x114/0x7638 kernel/locking/lockdep.c:4882
 lock_acquire+0x240/0x77c kernel/locking/lockdep.c:5623
 __raw_spin_lock_bh include/linux/spinlock_api_smp.h:135 [inline]
 _raw_spin_lock_bh+0x124/0x1c4 kernel/locking/spinlock.c:178
 spin_lock_bh include/linux/spinlock.h:368 [inline]
 lock_sock_nested+0x108/0x1ec net/core/sock.c:3253
 lock_sock include/net/sock.h:1676 [inline]
 sco_conn_ready net/bluetooth/sco.c:1129 [inline]
 sco_connect_cfm+0x3e8/0x908 net/bluetooth/sco.c:1201
 hci_connect_cfm include/net/bluetooth/hci_core.h:1510 [inline]
 hci_sync_conn_complete_evt+0x468/0x8c0 net/bluetooth/hci_event.c:4551
 hci_event_packet+0x8e0/0x12b4 net/bluetooth/hci_event.c:6483
 hci_rx_work+0x1c0/0x7c4 net/bluetooth/hci_core.c:5156
 process_one_work+0x790/0x11b8 kernel/workqueue.c:2310
 worker_thread+0x910/0x1034 kernel/workqueue.c:2457
 kthread+0x37c/0x45c kernel/kthread.c:334
 ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:870

Allocated by task 3973:
 kasan_save_stack mm/kasan/common.c:38 [inline]
 kasan_set_track mm/kasan/common.c:46 [inline]
 set_alloc_info mm/kasan/common.c:434 [inline]
 ____kasan_kmalloc+0xbc/0xfc mm/kasan/common.c:513
 __kasan_kmalloc+0x10/0x1c mm/kasan/common.c:522
 kasan_kmalloc include/linux/kasan.h:264 [inline]
 __kmalloc+0x29c/0x4c8 mm/slub.c:4407
 kmalloc include/linux/slab.h:596 [inline]
 sk_prot_alloc+0xc4/0x1f0 net/core/sock.c:1863
 sk_alloc+0x40/0x3e0 net/core/sock.c:1916
 sco_sock_alloc net/bluetooth/sco.c:485 [inline]
 sco_sock_create+0xb8/0x2d4 net/bluetooth/sco.c:520
 bt_sock_create+0x14c/0x248 net/bluetooth/af_bluetooth.c:130
 __sock_create+0x43c/0x884 net/socket.c:1486
 sock_create net/socket.c:1537 [inline]
 __sys_socket+0x168/0x310 net/socket.c:1579
 __do_sys_socket net/socket.c:1588 [inline]
 __se_sys_socket net/socket.c:1586 [inline]
 __arm64_sys_socket+0x7c/0x94 net/socket.c:1586
 __invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
 invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:52
 el0_svc_common+0x138/0x258 arch/arm64/kernel/syscall.c:142
 do_el0_svc+0x58/0x14c arch/arm64/kernel/syscall.c:181
 el0_svc+0x7c/0x1f0 arch/arm64/kernel/entry-common.c:608
 el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:626
 el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584

Freed by task 3973:
 kasan_save_stack mm/kasan/common.c:38 [inline]
 kasan_set_track+0x4c/0x84 mm/kasan/common.c:46
 kasan_set_free_info+0x28/0x4c mm/kasan/generic.c:360
 ____kasan_slab_free+0x118/0x164 mm/kasan/common.c:366
 __kasan_slab_free+0x18/0x28 mm/kasan/common.c:374
 kasan_slab_free include/linux/kasan.h:230 [inline]
 slab_free_hook mm/slub.c:1705 [inline]
 slab_free_freelist_hook+0x128/0x1ec mm/slub.c:1731
 slab_free mm/slub.c:3499 [inline]
 kfree+0x178/0x410 mm/slub.c:4559
 sk_prot_free net/core/sock.c:1899 [inline]
 __sk_destruct+0x418/0x600 net/core/sock.c:1985
 sk_destruct net/core/sock.c:2000 [inline]
 __sk_free+0x37c/0x4e8 net/core/sock.c:2011
 sk_free+0x68/0xdc net/core/sock.c:2022
 sock_put include/net/sock.h:1921 [inline]
 sco_sock_kill+0x104/0x1c8 net/bluetooth/sco.c:406
 sco_sock_release+0x1fc/0x2c0 net/bluetooth/sco.c:1098
 __sock_release net/socket.c:649 [inline]
 sock_close+0xb8/0x1fc net/socket.c:1336
 __fput+0x1c4/0x800 fs/file_table.c:280
 ____fput+0x20/0x30 fs/file_table.c:308
 task_work_run+0x130/0x1e4 kernel/task_work.c:164
 tracehook_notify_resume include/linux/tracehook.h:189 [inline]
 do_notify_resume+0x262c/0x32b8 arch/arm64/kernel/signal.c:946
 prepare_exit_to_user_mode arch/arm64/kernel/entry-common.c:133 [inline]
 exit_to_user_mode arch/arm64/kernel/entry-common.c:138 [inline]
 el0_svc+0xfc/0x1f0 arch/arm64/kernel/entry-common.c:609
 el0t_64_sync_handler+0x84/0xe4 arch/arm64/kernel/entry-common.c:626
 el0t_64_sync+0x1a0/0x1a4 arch/arm64/kernel/entry.S:584

The buggy address belongs to the object at ffff0000c9219000
 which belongs to the cache kmalloc-2k of size 2048
The buggy address is located 160 bytes inside of
 2048-byte region [ffff0000c9219000, ffff0000c9219800)
The buggy address belongs to the page:
page:00000000ecec829d refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x109218
head:00000000ecec829d order:3 compound_mapcount:0 compound_pincount:0
flags: 0x5ffc00000010200(slab|head|node=0|zone=2|lastcpupid=0x7ff)
raw: 05ffc00000010200 0000000000000000 dead000000000122 ffff0000c0002900
raw: 0000000000000000 0000000080080008 00000001ffffffff 0000000000000000
page dumped because: kasan: bad access detected

Memory state around the buggy address:
 ffff0000c9218f80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
 ffff0000c9219000: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>ffff0000c9219080: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
                               ^
 ffff0000c9219100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
 ffff0000c9219180: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
==================================================================
Unable to handle kernel paging request at virtual address dfff800000000000
Mem abort info:
  ESR = 0x0000000096000006
  EC = 0x25: DABT (current EL), IL = 32 bits
  SET = 0, FnV = 0
  EA = 0, S1PTW = 0
  FSC = 0x06: level 2 translation fault
Data abort info:
  ISV = 0, ISS = 0x00000006
  CM = 0, WnR = 0
[dfff800000000000] address between user and kernel address ranges
Internal error: Oops: 0000000096000006 [#1] PREEMPT SMP
Modules linked in:
CPU: 0 PID: 144 Comm: kworker/u5:0 Tainted: G    B   W         5.15.163-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/07/2024
Workqueue: hci0 hci_rx_work
pstate: 80400005 (Nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : apparmor_sk_clone_security+0xf0/0x3d8 security/apparmor/lsm.c:818
lr : apparmor_sk_clone_security+0xd0/0x3d8 security/apparmor/lsm.c:816
sp : ffff80001a3e7730
x29: ffff80001a3e7730 x28: dfff800000000000 x27: 1ffff0000347cefc
x26: ffff0000c921a3aa x25: ffff0000c9219530 x24: 1fffe0001aac87a0
x23: dfff800000000000 x22: dfff800000000000 x21: 0000000000000000
x20: 0000000000000000 x19: ffff0000d5643d00 x18: 0000000000000204
x17: ffff800010d8081c x16: ffff8000088cd8b4 x15: ffff80000fd64278
x14: ffff80000a5cac84 x13: ffff80000802b56c x12: 0000000000000001
x11: 0000000000000001 x10: 0000000000000000 x9 : ffff0000c2c13680
x8 : 0000000000000000 x7 : ffffffffffffffff x6 : ffff800010c744a8
x5 : 0000000000000000 x4 : 0000000000000001 x3 : ffff80000a645ac8
x2 : 0000000000000000 x1 : 0000000000000008 x0 : 0000000000000000
Call trace:
 apparmor_sk_clone_security+0xf0/0x3d8 security/apparmor/lsm.c:818
 security_sk_clone+0x58/0x9c security/security.c:2318
 sco_sock_init net/bluetooth/sco.c:468 [inline]
 sco_conn_ready net/bluetooth/sco.c:1139 [inline]
 sco_connect_cfm+0x58c/0x908 net/bluetooth/sco.c:1201
 hci_connect_cfm include/net/bluetooth/hci_core.h:1510 [inline]
 hci_sync_conn_complete_evt+0x468/0x8c0 net/bluetooth/hci_event.c:4551
 hci_event_packet+0x8e0/0x12b4 net/bluetooth/hci_event.c:6483
 hci_rx_work+0x1c0/0x7c4 net/bluetooth/hci_core.c:5156
 process_one_work+0x790/0x11b8 kernel/workqueue.c:2310
 worker_thread+0x910/0x1034 kernel/workqueue.c:2457
 kthread+0x37c/0x45c kernel/kthread.c:334
 ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:870
Code: 710006df 540010ab 9779f740 d343fe88 (38776908) 
---[ end trace 8fbecba04ebb2580 ]---
----------------
Code disassembly (best guess):
   0:	710006df 	cmp	w22, #0x1
   4:	540010ab 	b.lt	0x218  // b.tstop
   8:	9779f740 	bl	0xfffffffffde7dd08
   c:	d343fe88 	lsr	x8, x20, #3
* 10:	38776908 	ldrb	w8, [x8, x23] <-- trapping instruction

Crashes (18):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/07/19 13:47 linux-5.15.y 7c6d66f0266f ee4e11c8 .config console log report syz / log C [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: sleeping function called from invalid context in lock_sock_nested
2024/07/21 00:55 linux-5.15.y 7c6d66f0266f b88348e9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan BUG: sleeping function called from invalid context in lock_sock_nested
2024/07/21 00:54 linux-5.15.y 7c6d66f0266f b88348e9 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan BUG: sleeping function called from invalid context in lock_sock_nested
2024/07/19 23:51 linux-5.15.y 7c6d66f0266f 890ce4f3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan BUG: sleeping function called from invalid context in lock_sock_nested
2024/07/19 23:51 linux-5.15.y 7c6d66f0266f 890ce4f3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan BUG: sleeping function called from invalid context in lock_sock_nested
2024/07/19 23:32 linux-5.15.y 7c6d66f0266f 890ce4f3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan BUG: sleeping function called from invalid context in lock_sock_nested
2024/07/19 23:32 linux-5.15.y 7c6d66f0266f 890ce4f3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan BUG: sleeping function called from invalid context in lock_sock_nested
2024/07/19 23:32 linux-5.15.y 7c6d66f0266f 890ce4f3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan BUG: sleeping function called from invalid context in lock_sock_nested
2024/07/19 14:44 linux-5.15.y 7c6d66f0266f ee4e11c8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan BUG: sleeping function called from invalid context in lock_sock_nested
2024/07/19 13:52 linux-5.15.y 7c6d66f0266f ee4e11c8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan BUG: sleeping function called from invalid context in lock_sock_nested
2024/07/19 13:52 linux-5.15.y 7c6d66f0266f ee4e11c8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan BUG: sleeping function called from invalid context in lock_sock_nested
2024/07/19 13:50 linux-5.15.y 7c6d66f0266f ee4e11c8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan BUG: sleeping function called from invalid context in lock_sock_nested
2024/07/19 13:50 linux-5.15.y 7c6d66f0266f ee4e11c8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan BUG: sleeping function called from invalid context in lock_sock_nested
2024/07/04 03:34 linux-5.15.y 4878aadf2d15 f76a75f3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan BUG: sleeping function called from invalid context in lock_sock_nested
2024/06/29 07:50 linux-5.15.y 4878aadf2d15 757f06b1 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan BUG: sleeping function called from invalid context in lock_sock_nested
2024/09/06 14:08 linux-5.15.y 14e468424d3e 464ac2ed .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: sleeping function called from invalid context in lock_sock_nested
2024/09/06 14:07 linux-5.15.y 14e468424d3e 464ac2ed .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: sleeping function called from invalid context in lock_sock_nested
2024/07/19 12:43 linux-5.15.y 7c6d66f0266f ee4e11c8 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-arm64 BUG: sleeping function called from invalid context in lock_sock_nested
* Struck through repros no longer work on HEAD.