syzbot

map[KernelCommit:v6.19-rc7 KernelRepo:git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git PatchDescription:fbcon: Fix OOB write in cw_putcs() if vc_resize() fails

Syzkaller reported a KASAN vmalloc-out-of-bounds write in
fb_write_offset(). The crash occurs during console rotation when
vc_resize() fails.

When the console is rotated, fbcon_modechanged() calls vc_resize() to
update the console dimensions. However, it ignores the return value. If
vc_resize() fails (e.g., due to memory allocation failure), the console
state (vc) retains the old dimensions, while the framebuffer info
reflects the new dimensions.

If the rotation effectively reduces the screen size (e.g., rotating from
portrait to landscape on a portrait screen), the old row count in `vc`
may exceed the new limits. Subsequently, update_screen() invokes
cw_putcs(), which calculates the destination column using:

  image.dx = vxres - ((yy + 1) * vc->vc_font.height);

Here, `vxres` is the new (smaller) width, but `yy` iterates up to the
old (larger) row count. This causes the subtraction to underflow,
producing a huge `image.dx` value. This invalid coordinate is passed to
fb_imageblit(), causing the out-of-bounds write.

Fix this by checking the return value of vc_resize() in
fbcon_modechanged(). If resizing fails, return immediately to prevent
inconsistent state and the subsequent crash. PatchDiff:diff --git a/drivers/video/fbdev/core/fbcon.c b/drivers/video/fbdev/core/fbcon.c
index 7be9e8653..b9e418747 100644
--- a/drivers/video/fbdev/core/fbcon.c
+++ b/drivers/video/fbdev/core/fbcon.c
@@ -2712,7 +2712,8 @@ static void fbcon_modechanged(struct fb_info *info)
 		rows = FBCON_SWAP(par->rotate, info->var.yres, info->var.xres);
 		cols /= vc->vc_font.width;
 		rows /= vc->vc_font.height;
-		vc_resize(vc, cols, rows);
+		if (vc_resize(vc, cols, rows))
+			return;
 		updatescrollmode(p, info, vc);
 		scrollback_max = 0;
 		scrollback_current = 0;
 Recipients:[map[Email:deller@gmx.de Name:Helge Deller To:true] map[Email:dri-devel@lists.freedesktop.org Name: To:true] map[Email:linux-fbdev@vger.kernel.org Name: To:true] map[Email:linux-kernel@vger.kernel.org Name: To:false] map[Email:simona@ffwll.ch Name:Simona Vetter To:true]]]

map[KernelCommit:v6.19-rc7 KernelRepo:git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git]

map[KernelSrc:/usr/local/google/home/dvyukov/syzkaller/agent/workdir/cache/src/5452af61ab2e1f5601403fe379188fc9d8a831af]

map[KernelObj:/usr/local/google/home/dvyukov/syzkaller/agent/workdir/cache/build/1646d0db3b4b30850056a46e4eada212d6d5a58b]

map[BugTitle:KASAN: vmalloc-out-of-bounds Write in imageblit CrashReport:R10: 0000000000000000 R11: 0000000000000246 R12: 00007f1d3b43e4bc
R13: 00007f1d3b43e424 R14: 00007ffcf358def0 R15: 0000000000000003
 </TASK>
==================================================================
BUG: KASAN: vmalloc-out-of-bounds in fb_write_offset drivers/video/fbdev/core/sysmem.h:30 [inline]
BUG: KASAN: vmalloc-out-of-bounds in fb_bitmap_2ppw drivers/video/fbdev/core/fb_imageblit.h:364 [inline]
BUG: KASAN: vmalloc-out-of-bounds in fb_bitmap_imageblit drivers/video/fbdev/core/fb_imageblit.h:462 [inline]
BUG: KASAN: vmalloc-out-of-bounds in fb_imageblit drivers/video/fbdev/core/fb_imageblit.h:492 [inline]
BUG: KASAN: vmalloc-out-of-bounds in sys_imageblit+0xfed/0x1ea0 drivers/video/fbdev/core/sysimgblt.c:24
Write of size 8 at addr ffffc900038123c0 by task syz-executor388/6015

CPU: 0 UID: 0 PID: 6015 Comm: syz-executor388 Not tainted syzkaller #1 PREEMPT(full) 
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:94 [inline]
 dump_stack_lvl+0x10e/0x190 lib/dump_stack.c:120
 print_address_description mm/kasan/report.c:378 [inline]
 print_report+0x17e/0x810 mm/kasan/report.c:482
 kasan_report+0x147/0x180 mm/kasan/report.c:595
 fb_write_offset drivers/video/fbdev/core/sysmem.h:30 [inline]
 fb_bitmap_2ppw drivers/video/fbdev/core/fb_imageblit.h:364 [inline]
 fb_bitmap_imageblit drivers/video/fbdev/core/fb_imageblit.h:462 [inline]
 fb_imageblit drivers/video/fbdev/core/fb_imageblit.h:492 [inline]
 sys_imageblit+0xfed/0x1ea0 drivers/video/fbdev/core/sysimgblt.c:24
 drm_fbdev_shmem_defio_imageblit+0x2c/0x110 drivers/gpu/drm/drm_fbdev_shmem.c:39
 cw_putcs_aligned drivers/video/fbdev/core/fbcon_cw.c:110 [inline]
 cw_putcs+0x15af/0x1880 drivers/video/fbdev/core/fbcon_cw.c:158
 fbcon_putcs+0x3bd/0x590 drivers/video/fbdev/core/fbcon.c:1343
 do_update_region+0x3a1/0x460 drivers/tty/vt/vt.c:628
 redraw_screen+0x91a/0xe90 drivers/tty/vt/vt.c:980
 fbcon_rotate drivers/video/fbdev/core/fbcon.c:231 [inline]
 rotate_store+0x33f/0x510 drivers/video/fbdev/core/fbcon.c:3237
 kernfs_fop_write_iter+0x3af/0x540 fs/kernfs/file.c:352
 new_sync_write fs/read_write.c:593 [inline]
 vfs_write+0x5d2/0xb40 fs/read_write.c:686
 ksys_write+0x149/0x250 fs/read_write.c:738
 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
 do_syscall_64+0xe8/0xf80 arch/x86/entry/syscall_64.c:94
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f1d3b3d97c9
Code: c0 79 93 eb d5 48 8d 7c 1d 00 eb 99 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 d8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffcf358dea8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f1d3b3d97c9
RDX: 0000000000000007 RSI: 0000000000000000 RDI: 0000000000000003
RBP: 00007ffcf358def0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007f1d3b43e4bc
R13: 00007f1d3b43e424 R14: 00007ffcf358def0 R15: 0000000000000003
 </TASK>

The buggy address belongs to a vmalloc virtual mapping
Memory state around the buggy address:
 ffffc90003812280: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
 ffffc90003812300: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
>ffffc90003812380: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
                                           ^
 ffffc90003812400: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
 ffffc90003812480: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
==================================================================
]

map[Index:codesearch-index]

gemini-3-pro-preview

You are an experienced Linux kernel developer tasked with debugging a kernel crash root cause.
You need to provide a detailed explanation of the root cause for another developer to be
able to write a fix for the bug based on your explanation.
Your final reply must contain only the explanation.

Prefer calling several tools at the same time to save round-trips.

The crash is:

R10: 0000000000000000 R11: 0000000000000246 R12: 00007f1d3b43e4bc
R13: 00007f1d3b43e424 R14: 00007ffcf358def0 R15: 0000000000000003
 </TASK>
==================================================================
BUG: KASAN: vmalloc-out-of-bounds in fb_write_offset drivers/video/fbdev/core/sysmem.h:30 [inline]
BUG: KASAN: vmalloc-out-of-bounds in fb_bitmap_2ppw drivers/video/fbdev/core/fb_imageblit.h:364 [inline]
BUG: KASAN: vmalloc-out-of-bounds in fb_bitmap_imageblit drivers/video/fbdev/core/fb_imageblit.h:462 [inline]
BUG: KASAN: vmalloc-out-of-bounds in fb_imageblit drivers/video/fbdev/core/fb_imageblit.h:492 [inline]
BUG: KASAN: vmalloc-out-of-bounds in sys_imageblit+0xfed/0x1ea0 drivers/video/fbdev/core/sysimgblt.c:24
Write of size 8 at addr ffffc900038123c0 by task syz-executor388/6015

CPU: 0 UID: 0 PID: 6015 Comm: syz-executor388 Not tainted syzkaller #1 PREEMPT(full) 
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:94 [inline]
 dump_stack_lvl+0x10e/0x190 lib/dump_stack.c:120
 print_address_description mm/kasan/report.c:378 [inline]
 print_report+0x17e/0x810 mm/kasan/report.c:482
 kasan_report+0x147/0x180 mm/kasan/report.c:595
 fb_write_offset drivers/video/fbdev/core/sysmem.h:30 [inline]
 fb_bitmap_2ppw drivers/video/fbdev/core/fb_imageblit.h:364 [inline]
 fb_bitmap_imageblit drivers/video/fbdev/core/fb_imageblit.h:462 [inline]
 fb_imageblit drivers/video/fbdev/core/fb_imageblit.h:492 [inline]
 sys_imageblit+0xfed/0x1ea0 drivers/video/fbdev/core/sysimgblt.c:24
 drm_fbdev_shmem_defio_imageblit+0x2c/0x110 drivers/gpu/drm/drm_fbdev_shmem.c:39
 cw_putcs_aligned drivers/video/fbdev/core/fbcon_cw.c:110 [inline]
 cw_putcs+0x15af/0x1880 drivers/video/fbdev/core/fbcon_cw.c:158
 fbcon_putcs+0x3bd/0x590 drivers/video/fbdev/core/fbcon.c:1343
 do_update_region+0x3a1/0x460 drivers/tty/vt/vt.c:628
 redraw_screen+0x91a/0xe90 drivers/tty/vt/vt.c:980
 fbcon_rotate drivers/video/fbdev/core/fbcon.c:231 [inline]
 rotate_store+0x33f/0x510 drivers/video/fbdev/core/fbcon.c:3237
 kernfs_fop_write_iter+0x3af/0x540 fs/kernfs/file.c:352
 new_sync_write fs/read_write.c:593 [inline]
 vfs_write+0x5d2/0xb40 fs/read_write.c:686
 ksys_write+0x149/0x250 fs/read_write.c:738
 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
 do_syscall_64+0xe8/0xf80 arch/x86/entry/syscall_64.c:94
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f1d3b3d97c9
Code: c0 79 93 eb d5 48 8d 7c 1d 00 eb 99 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 d8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffcf358dea8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f1d3b3d97c9
RDX: 0000000000000007 RSI: 0000000000000000 RDI: 0000000000000003
RBP: 00007ffcf358def0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007f1d3b43e4bc
R13: 00007f1d3b43e424 R14: 00007ffcf358def0 R15: 0000000000000003
 </TASK>

The buggy address belongs to a vmalloc virtual mapping
Memory state around the buggy address:
 ffffc90003812280: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
 ffffc90003812300: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
>ffffc90003812380: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
                                           ^
 ffffc90003812400: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
 ffffc90003812480: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
==================================================================

The root cause of this crash is a missing error check for `vc_resize()` in `fbcon_modechanged()`, which leads to an integer underflow in `cw_putcs()` (Clockwise rotation blit function) when the console resizing fails.

Here is the step-by-step detailed explanation:

1.  **Context**: The crash occurs when `fbcon` is rotated (e.g., 90 degrees clockwise) via `rotate_store`. This triggers `fbcon_modechanged()`.
2.  **Logic Flaw in `fbcon_modechanged`**:
    *   When the screen is rotated, the logical dimensions (rows and columns) of the console change. For example, if rotating a "portrait" screen (Height > Width) to 90 degrees, the number of text rows must decrease (since the new width is the old height, which is smaller).
    *   `fbcon_modechanged()` calculates the new dimensions (`cols` and `rows`) and calls `vc_resize(vc, cols, rows)` to resize the console buffer.
    *   **The bug:** `fbcon_modechanged()` **ignores the return value** of `vc_resize()`.
    *   If `vc_resize()` fails (e.g., due to a temporary memory allocation failure), the console (`vc`) remains with its *old* dimensions. In our example, `vc->vc_rows` remains large.
    *   Despite the failure, the function proceeds to call `update_screen(vc)`, which triggers a redraw.

3.  **The Crash in `cw_putcs`**:
    *   `update_screen()` calls `redraw_screen()`, which calls `do_update_region()`.
    *   `do_update_region()` iterates through the console buffer using the *current* `vc->vc_rows`. Since the resize failed, it uses the **old (large)** row count.
    *   It calls `fbcon_putcs()`, which delegates to `cw_putcs()` (because rotation is active).
    *   `cw_putcs()` calculates the destination X coordinate (`image.dx`) in the framebuffer:
        ```c
        image.dx = vxres - ((yy + 1) * vc->vc_font.height);
        ```
        *   `vxres` is the physical screen width (which is small in this scenario).
        *   `yy` is the row index passed from `do_update_region()`.
    *   Because `vc` was not resized, `yy` can be large (valid for the old dimensions but too large for the new rotated orientation).
    *   Eventually, `(yy + 1) * height` becomes larger than `vxres`.
    *   The subtraction `vxres - (large_value)` causes an **unsigned integer underflow**, resulting in a massive value for `image.dx`.

4.  **Memory Corruption**:
    *   `cw_putcs()` calls `cw_putcs_aligned()`, which calls `fb_imageblit()`.
    *   `fb_imageblit()` calculates the memory address to write to:
        ```c
        dst = base + image.dy * stride + image.dx * bpp;
        ```
    *   Since `image.dx` is huge due to the underflow, `dst` points far outside the allocated framebuffer (vmalloc area).
    *   `fb_bitmap_2ppw()` (called by `fb_imageblit`) attempts to write to this invalid address via `fb_write_offset()`, triggering the **KASAN: vmalloc-out-of-bounds** crash.

**Summary**: The kernel crashes because `fbcon` attempts to draw rows that don't fit on the rotated screen after a console resize failure. The fix is to check the return value of `vc_resize()` in `fbcon_modechanged()` and abort the screen update if it fails.

gemini-3-pro-preview

							input: 4406
							output: 27
							thoughts: 619
						

**Kernel Crash Analysis: vmalloc-out-of-bounds in framebuffer rotation**

Alright, let's break this down. I've got a `KASAN: vmalloc-out-of-bounds` crash report, specifically a write access violation, which screams memory corruption in the virtual memory mapping. The key here is the `fb_write_offset` function, right there in `drivers/video/fbdev/core/sysmem.h:30`. The stack trace is pretty illuminating: it's showing the crash originating in framebuffer operations.

From the call stack, I'm seeing `fb_write_offset` called, which leads down a chain of inlined functions: `fb_bitmap_2ppw`, `fb_bitmap_imageblit`, `fb_imageblit`, then `sys_imageblit` in `sysimgblt.c`, which is the user-space image blitting interface. This leads to `drm_fbdev_shmem_defio_imageblit` and then something called `cw_putcs`, seemingly related to "ClockWise putcs", which is part of screen rotation in `fbcon_rotate`. So, the crash is triggered during a screen redraw, specifically, it appears to be tied to a rotation operation – probably from a `sysfs` write to `rotate`. The KASAN report specifies an "Write of size 8 at addr ffffc900038123c0" and inaccessible memory, `f8`.

My initial hypothesis is that somewhere in the calculations within `sys_imageblit` or one of its inlined helpers, an address is being calculated that goes beyond the allocated framebuffer memory. This is likely happening during the rotation, where dimensions might be swapped or incorrectly interpreted, causing an overflow when indexing into the framebuffer.

Here's my plan to investigate this further:

1.  First, I'll dive into `drivers/video/fbdev/core/sysmem.h` and take a close look around line 30 at `fb_write_offset`. I need to understand how it's calculating offsets and addresses within the framebuffer.
2.  Then, I'll examine `drivers/video/fbdev/core/fb_imageblit.h`, specifically, the implementation of `fb_bitmap_2ppw` and `fb_imageblit` as those are inlined. These are the likely culprits for memory calculations.
3.  Next, I will look through `drivers/video/fbdev/core/sysimgblt.c`, as this is the user-space interface that is ultimately crashing.
4.  Since `cw_putcs` and rotation are involved, I need to get familiar with `drivers/video/fbdev/core/fbcon_cw.c` to see how rotation is implemented and how `cw_putcs` fits into the picture.
5.  Finally, I'll need to understand how the framebuffer dimensions are handled during rotation. The crash is probably due to a miscalculation related to the change in dimensions or stride. I'll need to find where the dimensions are set/updated and how they're used by the `imageblit` functions.

I will start by listing all the files so that I can make sure I have the correct source paths. Let's dig in.

map[Dir:drivers/video/fbdev/core/]

map[Files:[bitblit.c cfbcopyarea.c cfbfillrect.c cfbimgblt.c cfbmem.h fb_backlight.c fb_chrdev.c fb_cmdline.c fb_copyarea.h fb_ddc.c fb_defio.c fb_draw.h fb_fillrect.h fb_imageblit.h fb_info.c fb_internal.h fb_io_fops.c fb_logo.c fb_notify.c fb_procfs.c fb_sys_fops.c fbcmap.c fbcon.c fbcon.h fbcon_ccw.c fbcon_cw.c fbcon_rotate.c fbcon_rotate.h fbcon_ud.c fbcvt.c fbmem.c fbmon.c fbsysfs.c modedb.c softcursor.c svgalib.c syscopyarea.c sysfillrect.c sysimgblt.c sysmem.h tileblit.c] Subdirs:<nil>]

gemini-3-pro-preview

							input: 6127
							output: 28
							thoughts: 36
						

Okay, here we go.

**Navigating the Framebuffer Code**

Alright, the initial verification is complete - the target files are present, which is good. Now, I need to delve into the core of how this framebuffer memory management works. My first step is to examine `drivers/video/fbdev/core/sysmem.h`. I'm specifically looking for the declaration of `fb_write_offset`. This function or macro will likely provide crucial information regarding how the system calculates the memory offset when writing to the framebuffer. Understanding how this offset is determined is key to grasping the overall memory organization and any potential issues or optimizations related to it. I expect it will involve details related to the screen's resolution, bpp, and potentially some addressing calculations. Let's see what we can find.

map[File:drivers/video/fbdev/core/sysmem.h]

map[Contents:/* SPDX-License-Identifier: GPL-2.0-only
 *
 *	Virtual memory framebuffer access for drawing routines
 *
 *	Copyright (C) 2025 Zsolt Kajtar (soci@c64.rulez.org)
 */

/* keeps track of a bit address in framebuffer memory */
struct fb_address {
	void *address;
	int bits;
};

/* initialize the bit address pointer to the beginning of the frame buffer */
static inline struct fb_address fb_address_init(struct fb_info *p)
{
	void *base = p->screen_buffer;
	struct fb_address ptr;

	ptr.address = PTR_ALIGN_DOWN(base, BITS_PER_LONG / BITS_PER_BYTE);
	ptr.bits = (base - ptr.address) * BITS_PER_BYTE;
	return ptr;
}

/* framebuffer write access */
static inline void fb_write_offset(unsigned long val, int offset, const struct fb_address *dst)
{
	unsigned long *mem = dst->address;

	mem[offset] = val;
}

/* framebuffer read access */
static inline unsigned long fb_read_offset(int offset, const struct fb_address *src)
{
	unsigned long *mem = src->address;

	return mem[offset];
}
]

gemini-3-pro-preview

							input: 6681
							output: 30
							thoughts: 200
						

**Debugging a Suspected Out-of-Bounds Write**

Okay, so I'm looking at this `fb_write_offset` function, and it's pretty straightforward. It's essentially a convenience function for writing a value (`val`) to a memory location. That location is determined by taking a base address (`dst->address`), which is cast as an `unsigned long *`, and adding an offset to it. That offset is most likely an index into an array of `unsigned long` values, which on x86-64 means it's working in 64-bit chunks.

The crash report is screaming `KASAN: vmalloc-out-of-bounds`, which tells me, unequivocally, that we're writing *past* the end of the allocated memory pointed to by `dst->address`. This isn't a maybe, it's a definite problem.

My next move is to understand where `fb_write_offset` is being called from. I need to trace back from here, now, to the `fb_bitmap_2ppw` and `fb_imageblit` functions within `drivers/video/fbdev/core/fb_imageblit.h`. I've got to figure out *what* offset is being calculated, and *why* it's leading to this out-of-bounds access. The problem is almost certainly a miscalculation or improper usage of `offset` somewhere upstream, or a mismatch between the allocated size of the `dst->address` buffer and the index `offset` within it. Time to dig into those functions.

map[File:drivers/video/fbdev/core/fb_imageblit.h]

map[Contents:/* SPDX-License-Identifier: GPL-2.0-only
 *
 *  Generic bitmap / 8 bpp image bitstreamer for packed pixel framebuffers
 *
 *	Rewritten by:
 *	Copyright (C)  2025 Zsolt Kajtar (soci@c64.rulez.org)
 *
 *	Based on previous work of:
 *	Copyright (C)  June 1999 James Simmons
 *	Anton Vorontsov <avorontsov@ru.mvista.com>
 *	Pavel Pisa <pisa@cmp.felk.cvut.cz>
 *	Antonino A. Daplas <adaplas@gmail.com>
 *	and others
 *
 * NOTES:
 *
 * Handles native and foreign byte order on both endians, standard and
 * reverse pixel order in a byte (<8 BPP), word length of 32/64 bits,
 * bits per pixel from 1 to the word length. Handles line lengths at byte
 * granularity while maintaining aligned accesses.
 *
 * Optimized routines for word aligned 1, 2, 4 pixel per word for high
 * bpp modes and 4 pixel at a time operation for low bpp.
 *
 * The color image is expected to be one byte per pixel, and values should
 * not exceed the bitdepth or the pseudo palette (if used).
 */
#include "fb_draw.h"

/* bitmap image iterator, one pixel at a time */
struct fb_bitmap_iter {
	const u8 *data;
	unsigned long colors[2];
	int width, i;
};

static bool fb_bitmap_image(void *iterator, unsigned long *pixels, int *bits)
{
	struct fb_bitmap_iter *iter = iterator;

	if (iter->i < iter->width) {
		int bit = ~iter->i & (BITS_PER_BYTE-1);
		int byte = iter->i++ / BITS_PER_BYTE;

		*pixels = iter->colors[(iter->data[byte] >> bit) & 1];
		return true;
	}
	iter->data += BITS_TO_BYTES(iter->width);
	iter->i = 0;
	return false;
}

/* color image iterator, one pixel at a time */
struct fb_color_iter {
	const u8 *data;
	const u32 *palette;
	struct fb_reverse reverse;
	int shift;
	int width, i;
};

static bool fb_color_image(void *iterator, unsigned long *pixels, int *bits)
{
	struct fb_color_iter *iter = iterator;

	if (iter->i < iter->width) {
		unsigned long color = iter->data[iter->i++];

		if (iter->palette)
			color = iter->palette[color];
		*pixels = color << iter->shift;
		if (iter->reverse.pixel)
			*pixels = fb_reverse_bits_long(*pixels);
		return true;
	}
	iter->data += iter->width;
	iter->i = 0;
	return false;
}

/* bitmap image iterator, 4 pixels at a time */
struct fb_bitmap4x_iter {
	const u8 *data;
	u32 fgxcolor, bgcolor;
	int width, i;
	const u32 *expand;
	int bpp;
	bool top;
};

static bool fb_bitmap4x_image(void *iterator, unsigned long *pixels, int *bits)
{
	struct fb_bitmap4x_iter *iter = iterator;
	u8 data;

	if (iter->i >= BITS_PER_BYTE/2) {
		iter->i -= BITS_PER_BYTE/2;
		iter->top = !iter->top;
		if (iter->top)
			data = *iter->data++ >> BITS_PER_BYTE/2;
		else
			data = iter->data[-1] & ((1 << BITS_PER_BYTE/2)-1);
	} else if (iter->i != 0) {
		*bits = iter->bpp * iter->i;
		if (iter->top)
			data = iter->data[-1] & ((1 << BITS_PER_BYTE/2)-1);
		else
			data = *iter->data++ >> BITS_PER_BYTE/2;
#ifndef __LITTLE_ENDIAN
		data >>= BITS_PER_BYTE/2 - iter->i;
#endif
		iter->i = 0;
	} else {
		*bits = iter->bpp * BITS_PER_BYTE/2;
		iter->i = iter->width;
		iter->top = false;
		return false;
	}
	*pixels = (iter->fgxcolor & iter->expand[data]) ^ iter->bgcolor;
#ifndef __LITTLE_ENDIAN
	*pixels <<= BITS_PER_LONG - *bits;
#endif
	return true;
}

/* draw a line a group of pixels at a time */
static __always_inline void fb_bitblit(bool (*get)(void *iter, unsigned long *pixels,
						   int *bits),
				       void *iter, int bits, struct fb_address *dst,
				       struct fb_reverse reverse)
{
	unsigned long pixels, val, mask, old;
	int offset = 0;
	int shift = dst->bits;

	if (shift) {
		old = fb_read_offset(0, dst);
		val = fb_reverse_long(old, reverse);
		val &= ~fb_right(~0UL, shift);
	} else {
		old = 0;
		val = 0;
	}

	while (get(iter, &pixels, &bits)) {
		val |= fb_right(pixels, shift);
		shift += bits;

		if (shift < BITS_PER_LONG)
			continue;

		val = fb_reverse_long(val, reverse);
		fb_write_offset(val, offset++, dst);
		shift &= BITS_PER_LONG - 1;
		val = !shift ? 0 : fb_left(pixels, bits - shift);
	}

	if (shift) {
		mask = ~fb_pixel_mask(shift, reverse);
		val = fb_reverse_long(val, reverse);
		if (offset || !dst->bits)
			old = fb_read_offset(offset, dst);
		fb_write_offset(fb_comp(val, old, mask), offset, dst);
	}
}

/* draw a color image a pixel at a time */
static inline void fb_color_imageblit(const struct fb_image *image, struct fb_address *dst,
				      unsigned int bits_per_line, const u32 *palette, int bpp,
				      struct fb_reverse reverse)
{
	struct fb_color_iter iter;
	u32 height;

	iter.data = (const u8 *)image->data;
	iter.palette = palette;
	iter.reverse = reverse;
#ifdef __LITTLE_ENDIAN
	if (reverse.pixel)
		iter.shift = BITS_PER_BYTE - bpp;
	else
		iter.shift = 0;
#else
	if (reverse.pixel)
		iter.shift = BITS_PER_LONG - BITS_PER_BYTE;
	else
		iter.shift = BITS_PER_LONG - bpp;
#endif
	iter.width = image->width;
	iter.i = 0;

	height = image->height;
	while (height--) {
		fb_bitblit(fb_color_image, &iter, bpp, dst, reverse);
		fb_address_forward(dst, bits_per_line);
	}
}

#ifdef __LITTLE_ENDIAN
#define FB_GEN(a, b) (((a)/8+(((a)&4)<<((b)-2)) \
		       +(((a)&2)<<((b)*2-1))+(((a)&1u)<<((b)*3)))*((1<<(b))-1))
#define FB_GEN1(a) ((a)/8+((a)&4)/2+((a)&2)*2+((a)&1)*8)
#else
#define FB_GEN(a, b) (((((a)/8)<<((b)*3))+(((a)&4)<<((b)*2-2)) \
		       +(((a)&2)<<(b-1))+((a)&1u))*((1<<(b))-1))
#define FB_GEN1(a) (a)
#endif

#define FB_GENx(a) { FB_GEN(0, (a)), FB_GEN(1, (a)), FB_GEN(2, (a)), FB_GEN(3, (a)),	\
	FB_GEN(4, (a)), FB_GEN(5, (a)), FB_GEN(6, (a)), FB_GEN(7, (a)),			\
	FB_GEN(8, (a)), FB_GEN(9, (a)), FB_GEN(10, (a)), FB_GEN(11, (a)),		\
	FB_GEN(12, (a)), FB_GEN(13, (a)), FB_GEN(14, (a)), FB_GEN(15, (a)) }

/* draw a 2 color image four pixels at a time (for 1-8 bpp only) */
static inline void fb_bitmap4x_imageblit(const struct fb_image *image, struct fb_address *dst,
					 unsigned long fgcolor, unsigned long bgcolor, int bpp,
					 unsigned int bits_per_line, struct fb_reverse reverse)
{
	static const u32 mul[BITS_PER_BYTE] = {
		0xf, 0x55, 0x249, 0x1111, 0x8421, 0x41041, 0x204081, 0x01010101
	};
	static const u32 expand[BITS_PER_BYTE][1 << 4] = {
		{
			FB_GEN1(0), FB_GEN1(1), FB_GEN1(2), FB_GEN1(3),
			FB_GEN1(4), FB_GEN1(5), FB_GEN1(6), FB_GEN1(7),
			FB_GEN1(8), FB_GEN1(9), FB_GEN1(10), FB_GEN1(11),
			FB_GEN1(12), FB_GEN1(13), FB_GEN1(14), FB_GEN1(15)
		},
		FB_GENx(2), FB_GENx(3), FB_GENx(4),
		FB_GENx(5), FB_GENx(6), FB_GENx(7), FB_GENx(8),
	};
	struct fb_bitmap4x_iter iter;
	u32 height;

	iter.data = (const u8 *)image->data;
	if (reverse.pixel) {
		fgcolor = fb_reverse_bits_long(fgcolor << (BITS_PER_BYTE - bpp));
		bgcolor = fb_reverse_bits_long(bgcolor << (BITS_PER_BYTE - bpp));
	}
	iter.fgxcolor = (fgcolor ^ bgcolor) * mul[bpp-1];
	iter.bgcolor = bgcolor * mul[bpp-1];
	iter.width = image->width;
	iter.i = image->width;
	iter.expand = expand[bpp-1];
	iter.bpp = bpp;
	iter.top = false;

	height = image->height;
	while (height--) {
		fb_bitblit(fb_bitmap4x_image, &iter, bpp * BITS_PER_BYTE/2, dst, reverse);
		fb_address_forward(dst, bits_per_line);
	}
}

/* draw a bitmap image 1 pixel at a time (for >8 bpp) */
static inline void fb_bitmap1x_imageblit(const struct fb_image *image, struct fb_address *dst,
					 unsigned long fgcolor, unsigned long bgcolor, int bpp,
					 unsigned int bits_per_line, struct fb_reverse reverse)
{
	struct fb_bitmap_iter iter;
	u32 height;

	iter.colors[0] = bgcolor;
	iter.colors[1] = fgcolor;
#ifndef __LITTLE_ENDIAN
	iter.colors[0] <<= BITS_PER_LONG - bpp;
	iter.colors[1] <<= BITS_PER_LONG - bpp;
#endif
	iter.data = (const u8 *)image->data;
	iter.width = image->width;
	iter.i = 0;

	height = image->height;
	while (height--) {
		fb_bitblit(fb_bitmap_image, &iter, bpp, dst, reverse);
		fb_address_forward(dst, bits_per_line);
	}
}

/* one pixel per word, 64/32 bpp blitting */
static inline void fb_bitmap_1ppw(const struct fb_image *image, struct fb_address *dst,
				  unsigned long fgcolor, unsigned long bgcolor,
				  int words_per_line, struct fb_reverse reverse)
{
	unsigned long tab[2];
	const u8 *src = (u8 *)image->data;
	int width = image->width;
	int offset;
	u32 height;

	if (reverse.byte) {
		tab[0] = swab_long(bgcolor);
		tab[1] = swab_long(fgcolor);
	} else {
		tab[0] = bgcolor;
		tab[1] = fgcolor;
	}

	height = image->height;
	while (height--) {
		for (offset = 0; offset + 8 <= width; offset += 8) {
			unsigned int srcbyte = *src++;

			fb_write_offset(tab[(srcbyte >> 7) & 1], offset + 0, dst);
			fb_write_offset(tab[(srcbyte >> 6) & 1], offset + 1, dst);
			fb_write_offset(tab[(srcbyte >> 5) & 1], offset + 2, dst);
			fb_write_offset(tab[(srcbyte >> 4) & 1], offset + 3, dst);
			fb_write_offset(tab[(srcbyte >> 3) & 1], offset + 4, dst);
			fb_write_offset(tab[(srcbyte >> 2) & 1], offset + 5, dst);
			fb_write_offset(tab[(srcbyte >> 1) & 1], offset + 6, dst);
			fb_write_offset(tab[(srcbyte >> 0) & 1], offset + 7, dst);
		}

		if (offset < width) {
			unsigned int srcbyte = *src++;

			while (offset < width) {
				fb_write_offset(tab[(srcbyte >> 7) & 1], offset, dst);
				srcbyte <<= 1;
				offset++;
			}
		}
		fb_address_move_long(dst, words_per_line);
	}
}

static inline unsigned long fb_pack(unsigned long left, unsigned long right, int bits)
{
#ifdef __LITTLE_ENDIAN
	return left | right << bits;
#else
	return right | left << bits;
#endif
}

/* aligned 32/16 bpp blitting */
static inline void fb_bitmap_2ppw(const struct fb_image *image, struct fb_address *dst,
				  unsigned long fgcolor, unsigned long bgcolor,
				  int words_per_line, struct fb_reverse reverse)
{
	unsigned long tab[4];
	const u8 *src = (u8 *)image->data;
	int width = image->width / 2;
	int offset;
	u32 height;

	tab[0] = fb_pack(bgcolor, bgcolor, BITS_PER_LONG/2);
	tab[1] = fb_pack(bgcolor, fgcolor, BITS_PER_LONG/2);
	tab[2] = fb_pack(fgcolor, bgcolor, BITS_PER_LONG/2);
	tab[3] = fb_pack(fgcolor, fgcolor, BITS_PER_LONG/2);

	if (reverse.byte) {
		tab[0] = swab_long(tab[0]);
		tab[1] = swab_long(tab[1]);
		tab[2] = swab_long(tab[2]);
		tab[3] = swab_long(tab[3]);
	}

	height = image->height;
	while (height--) {
		for (offset = 0; offset + 4 <= width; offset += 4) {
			unsigned int srcbyte = *src++;

			fb_write_offset(tab[(srcbyte >> 6) & 3], offset + 0, dst);
			fb_write_offset(tab[(srcbyte >> 4) & 3], offset + 1, dst);
			fb_write_offset(tab[(srcbyte >> 2) & 3], offset + 2, dst);
			fb_write_offset(tab[(srcbyte >> 0) & 3], offset + 3, dst);
		}

		if (offset < width) {
			unsigned int srcbyte = *src++;

			while (offset < width) {
				fb_write_offset(tab[(srcbyte >> 6) & 3], offset, dst);
				srcbyte <<= 2;
				offset++;
			}
		}
		fb_address_move_long(dst, words_per_line);
	}
}

#define FB_PATP(a, b) (((a)<<((b)*BITS_PER_LONG/4))*((1UL<<BITS_PER_LONG/4)-1UL))
#define FB_PAT4(a) (FB_PATP((a)&1, 0)|FB_PATP(((a)&2)/2, 1)| \
	FB_PATP(((a)&4)/4, 2)|FB_PATP(((a)&8)/8, 3))

/* aligned 16/8 bpp blitting */
static inline void fb_bitmap_4ppw(const struct fb_image *image, struct fb_address *dst,
				  unsigned long fgcolor, unsigned long bgcolor,
				  int words_per_line, struct fb_reverse reverse)
{
	static const unsigned long tab16_be[] = {
		0, FB_PAT4(1UL), FB_PAT4(2UL), FB_PAT4(3UL),
		FB_PAT4(4UL), FB_PAT4(5UL), FB_PAT4(6UL), FB_PAT4(7UL),
		FB_PAT4(8UL), FB_PAT4(9UL), FB_PAT4(10UL), FB_PAT4(11UL),
		FB_PAT4(12UL), FB_PAT4(13UL), FB_PAT4(14UL), ~0UL
	};
	static const unsigned long tab16_le[] = {
		0, FB_PAT4(8UL), FB_PAT4(4UL), FB_PAT4(12UL),
		FB_PAT4(2UL), FB_PAT4(10UL), FB_PAT4(6UL), FB_PAT4(14UL),
		FB_PAT4(1UL), FB_PAT4(9UL), FB_PAT4(5UL), FB_PAT4(13UL),
		FB_PAT4(3UL), FB_PAT4(11UL), FB_PAT4(7UL), ~0UL
	};
	const unsigned long *tab;
	const u8 *src = (u8 *)image->data;
	int width = image->width / 4;
	int offset;
	u32 height;

	fgcolor = fgcolor | fgcolor << BITS_PER_LONG/4;
	bgcolor = bgcolor | bgcolor << BITS_PER_LONG/4;
	fgcolor = fgcolor | fgcolor << BITS_PER_LONG/2;
	bgcolor = bgcolor | bgcolor << BITS_PER_LONG/2;
	fgcolor ^= bgcolor;

	if (BITS_PER_LONG/4 > BITS_PER_BYTE && reverse.byte) {
		fgcolor = swab_long(fgcolor);
		bgcolor = swab_long(bgcolor);
	}

#ifdef __LITTLE_ENDIAN
	tab = reverse.byte ? tab16_be : tab16_le;
#else
	tab = reverse.byte ? tab16_le : tab16_be;
#endif

	height = image->height;
	while (height--) {
		for (offset = 0; offset + 2 <= width; offset += 2, src++) {
			fb_write_offset((fgcolor & tab[*src >> 4]) ^ bgcolor, offset + 0, dst);
			fb_write_offset((fgcolor & tab[*src & 0xf]) ^ bgcolor, offset + 1, dst);
		}

		if (offset < width)
			fb_write_offset((fgcolor & tab[*src++ >> 4]) ^ bgcolor, offset, dst);

		fb_address_move_long(dst, words_per_line);
	}
}

static inline void fb_bitmap_imageblit(const struct fb_image *image, struct fb_address *dst,
				       unsigned int bits_per_line, const u32 *palette, int bpp,
				       struct fb_reverse reverse)
{
	unsigned long fgcolor, bgcolor;

	if (palette) {
		fgcolor = palette[image->fg_color];
		bgcolor = palette[image->bg_color];
	} else {
		fgcolor = image->fg_color;
		bgcolor = image->bg_color;
	}

	if (!dst->bits && !(bits_per_line & (BITS_PER_LONG-1))) {
		if (bpp == BITS_PER_LONG && BITS_PER_LONG == 32) {
			fb_bitmap_1ppw(image, dst, fgcolor, bgcolor,
				       bits_per_line / BITS_PER_LONG, reverse);
			return;
		}
		if (bpp == BITS_PER_LONG/2 && !(image->width & 1)) {
			fb_bitmap_2ppw(image, dst, fgcolor, bgcolor,
				       bits_per_line / BITS_PER_LONG, reverse);
			return;
		}
		if (bpp == BITS_PER_LONG/4 && !(image->width & 3)) {
			fb_bitmap_4ppw(image, dst, fgcolor, bgcolor,
				       bits_per_line / BITS_PER_LONG, reverse);
			return;
		}
	}

	if (bpp > 0 && bpp <= BITS_PER_BYTE)
		fb_bitmap4x_imageblit(image, dst, fgcolor, bgcolor, bpp,
				     bits_per_line, reverse);
	else if (bpp > BITS_PER_BYTE && bpp <= BITS_PER_LONG)
		fb_bitmap1x_imageblit(image, dst, fgcolor, bgcolor, bpp,
				     bits_per_line, reverse);
}

static inline void fb_imageblit(struct fb_info *p, const struct fb_image *image)
{
	int bpp = p->var.bits_per_pixel;
	unsigned int bits_per_line = BYTES_TO_BITS(p->fix.line_length);
	struct fb_address dst = fb_address_init(p);
	struct fb_reverse reverse = fb_reverse_init(p);
	const u32 *palette = fb_palette(p);

	fb_address_forward(&dst, image->dy * bits_per_line + image->dx * bpp);

	if (image->depth == 1)
		fb_bitmap_imageblit(image, &dst, bits_per_line, palette, bpp, reverse);
	else
		fb_color_imageblit(image, &dst, bits_per_line, palette, bpp, reverse);
}
]

gemini-3-pro-preview

							input: 13060
							output: 30
							thoughts: 1251
						

**Analyzing the Pixel Blitting Path and Potential Out-of-Bounds Write**

Okay, so I've traced this back from the crash in `fb_write_offset` within `fb_bitmap_2ppw`, and it's clear what's happening. The backtrace nailed it – `fb_bitmap_2ppw` is the culprit. This function is specialized for 32-bit (BITS_PER_LONG/2) and 2 pixels per word (2ppw) blitting, optimized for aligned memory writes. The key is in the conditions that *trigger* this optimized path: the destination (`dst`) must be long-aligned (`dst->bits == 0`), `bits_per_line` must be a multiple of the long size, the bytes per pixel (`bpp`) is equal to 32 bits, and the image width is even. Since `bpp` is 32, one pixel takes 32 bits which means two pixels fit in one 64-bit long. The `width` variable is actually the number of longs to write per line, thus width = `image->width` / 2.

The core logic within `fb_bitmap_2ppw` is a nested loop. The outer loop iterates through the image height, and the inner loop processes each line, essentially doing the following: It effectively iterates the memory range associated with the image drawing pixels.

```
base = initial_dst + image->dy * line_length + image->dx * bpp / 8
for y from 0 to height-1:
    for x_word from 0 to width-1:
        base[x_word] = ...
    base += words_per_line
```

The `fb_write_offset` is where the memory writes occur, which, with the offset relative to the base address (`dst->address`), combined with the width of the image being even, translates to the code writing memory. The crucial part is that `dst->address` is updated at the end of each line by adding `words_per_line` * `sizeof(long)` bytes.

So, if the calculated destination address plus the size of the blitted image (derived from `image->dx`, `image->dy`, `image->width`, `image->height`, and `line_length`) exceeds the bounds of the framebuffer, it leads to an out-of-bounds write. Now, the call chain confirms this: the crash originates from `sys_imageblit`, which comes from `drm_fbdev_shmem_defio_imageblit`, which then comes from `cw_putcs`, which comes from Clockwise rotation from `drivers/video/fbdev/core/fbcon_cw.c`.
I need to investigate that `cw_putcs` function and the parameters it receives more closely. It looks like the issue is related to how the rotation is implemented in the framebuffer console.

map[File:drivers/video/fbdev/core/fbcon_cw.c]

map[Contents:/*
 *  linux/drivers/video/console/fbcon_ud.c -- Software Rotation - 90 degrees
 *
 *      Copyright (C) 2005 Antonino Daplas <adaplas @pol.net>
 *
 *  This file is subject to the terms and conditions of the GNU General Public
 *  License.  See the file COPYING in the main directory of this archive for
 *  more details.
 */

#include <linux/module.h>
#include <linux/slab.h>
#include <linux/string.h>
#include <linux/fb.h>
#include <linux/vt_kern.h>
#include <linux/console.h>
#include <asm/types.h>
#include "fbcon.h"
#include "fbcon_rotate.h"

/*
 * Rotation 90 degrees
 */

static void cw_update_attr(u8 *dst, u8 *src, int attribute,
				  struct vc_data *vc)
{
	int i, j, offset = (vc->vc_font.height < 10) ? 1 : 2;
	int width = (vc->vc_font.height + 7) >> 3;
	u8 c, msk = ~(0xff >> offset);

	for (i = 0; i < vc->vc_font.width; i++) {
		for (j = 0; j < width; j++) {
			c = *src;
			if (attribute & FBCON_ATTRIBUTE_UNDERLINE && !j)
				c |= msk;
			if (attribute & FBCON_ATTRIBUTE_BOLD && i)
				c |= *(src-width);
			if (attribute & FBCON_ATTRIBUTE_REVERSE)
				c = ~c;
			src++;
			*dst++ = c;
		}
	}
}


static void cw_bmove(struct vc_data *vc, struct fb_info *info, int sy,
		     int sx, int dy, int dx, int height, int width)
{
	struct fbcon_par *par = info->fbcon_par;
	struct fb_copyarea area;
	u32 vxres = GETVXRES(par->p, info);

	area.sx = vxres - ((sy + height) * vc->vc_font.height);
	area.sy = sx * vc->vc_font.width;
	area.dx = vxres - ((dy + height) * vc->vc_font.height);
	area.dy = dx * vc->vc_font.width;
	area.width = height * vc->vc_font.height;
	area.height  = width * vc->vc_font.width;

	info->fbops->fb_copyarea(info, &area);
}

static void cw_clear(struct vc_data *vc, struct fb_info *info, int sy,
		     int sx, int height, int width, int fg, int bg)
{
	struct fbcon_par *par = info->fbcon_par;
	struct fb_fillrect region;
	u32 vxres = GETVXRES(par->p, info);

	region.color = bg;
	region.dx = vxres - ((sy + height) * vc->vc_font.height);
	region.dy = sx *  vc->vc_font.width;
	region.height = width * vc->vc_font.width;
	region.width = height * vc->vc_font.height;
	region.rop = ROP_COPY;

	info->fbops->fb_fillrect(info, &region);
}

static inline void cw_putcs_aligned(struct vc_data *vc, struct fb_info *info,
				    const u16 *s, u32 attr, u32 cnt,
				    u32 d_pitch, u32 s_pitch, u32 cellsize,
				    struct fb_image *image, u8 *buf, u8 *dst)
{
	struct fbcon_par *par = info->fbcon_par;
	u16 charmask = vc->vc_hi_font_mask ? 0x1ff : 0xff;
	u32 idx = (vc->vc_font.height + 7) >> 3;
	u8 *src;

	while (cnt--) {
		src = par->fontbuffer + (scr_readw(s++) & charmask) * cellsize;

		if (attr) {
			cw_update_attr(buf, src, attr, vc);
			src = buf;
		}

		if (likely(idx == 1))
			__fb_pad_aligned_buffer(dst, d_pitch, src, idx,
						vc->vc_font.width);
		else
			fb_pad_aligned_buffer(dst, d_pitch, src, idx,
					      vc->vc_font.width);

		dst += d_pitch * vc->vc_font.width;
	}

	info->fbops->fb_imageblit(info, image);
}

static void cw_putcs(struct vc_data *vc, struct fb_info *info,
		      const unsigned short *s, int count, int yy, int xx,
		      int fg, int bg)
{
	struct fb_image image;
	struct fbcon_par *par = info->fbcon_par;
	u32 width = (vc->vc_font.height + 7)/8;
	u32 cellsize = width * vc->vc_font.width;
	u32 maxcnt = info->pixmap.size/cellsize;
	u32 scan_align = info->pixmap.scan_align - 1;
	u32 buf_align = info->pixmap.buf_align - 1;
	u32 cnt, pitch, size;
	u32 attribute = get_attribute(info, scr_readw(s));
	u8 *dst, *buf = NULL;
	u32 vxres = GETVXRES(par->p, info);

	if (!par->fontbuffer)
		return;

	image.fg_color = fg;
	image.bg_color = bg;
	image.dx = vxres - ((yy + 1) * vc->vc_font.height);
	image.dy = xx * vc->vc_font.width;
	image.width = vc->vc_font.height;
	image.depth = 1;

	if (attribute) {
		buf = kmalloc(cellsize, GFP_KERNEL);
		if (!buf)
			return;
	}

	while (count) {
		if (count > maxcnt)
			cnt = maxcnt;
		else
			cnt = count;

		image.height = vc->vc_font.width * cnt;
		pitch = ((image.width + 7) >> 3) + scan_align;
		pitch &= ~scan_align;
		size = pitch * image.height + buf_align;
		size &= ~buf_align;
		dst = fb_get_buffer_offset(info, &info->pixmap, size);
		image.data = dst;
		cw_putcs_aligned(vc, info, s, attribute, cnt, pitch,
				 width, cellsize, &image, buf, dst);
		image.dy += image.height;
		count -= cnt;
		s += cnt;
	}

	/* buf is always NULL except when in monochrome mode, so in this case
	   it's a gain to check buf against NULL even though kfree() handles
	   NULL pointers just fine */
	if (unlikely(buf))
		kfree(buf);

}

static void cw_clear_margins(struct vc_data *vc, struct fb_info *info,
			     int color, int bottom_only)
{
	unsigned int cw = vc->vc_font.width;
	unsigned int ch = vc->vc_font.height;
	unsigned int rw = info->var.yres - (vc->vc_cols*cw);
	unsigned int bh = info->var.xres - (vc->vc_rows*ch);
	unsigned int rs = info->var.yres - rw;
	struct fb_fillrect region;

	region.color = color;
	region.rop = ROP_COPY;

	if ((int) rw > 0 && !bottom_only) {
		region.dx = 0;
		region.dy = info->var.yoffset + rs;
		region.height = rw;
		region.width = info->var.xres_virtual;
		info->fbops->fb_fillrect(info, &region);
	}

	if ((int) bh > 0) {
		region.dx = info->var.xoffset;
		region.dy = info->var.yoffset;
                region.height = info->var.yres;
                region.width = bh;
		info->fbops->fb_fillrect(info, &region);
	}
}

static void cw_cursor(struct vc_data *vc, struct fb_info *info, bool enable,
		      int fg, int bg)
{
	struct fb_cursor cursor;
	struct fbcon_par *par = info->fbcon_par;
	unsigned short charmask = vc->vc_hi_font_mask ? 0x1ff : 0xff;
	int w = (vc->vc_font.height + 7) >> 3, c;
	int y = real_y(par->p, vc->state.y);
	int attribute, use_sw = vc->vc_cursor_type & CUR_SW;
	int err = 1, dx, dy;
	char *src;
	u32 vxres = GETVXRES(par->p, info);

	if (!par->fontbuffer)
		return;

	cursor.set = 0;

 	c = scr_readw((u16 *) vc->vc_pos);
	attribute = get_attribute(info, c);
	src = par->fontbuffer + ((c & charmask) * (w * vc->vc_font.width));

	if (par->cursor_state.image.data != src ||
	    par->cursor_reset) {
		par->cursor_state.image.data = src;
		cursor.set |= FB_CUR_SETIMAGE;
	}

	if (attribute) {
		u8 *dst;

		dst = kmalloc_array(w, vc->vc_font.width, GFP_ATOMIC);
		if (!dst)
			return;
		kfree(par->cursor_data);
		par->cursor_data = dst;
		cw_update_attr(dst, src, attribute, vc);
		src = dst;
	}

	if (par->cursor_state.image.fg_color != fg ||
	    par->cursor_state.image.bg_color != bg ||
	    par->cursor_reset) {
		par->cursor_state.image.fg_color = fg;
		par->cursor_state.image.bg_color = bg;
		cursor.set |= FB_CUR_SETCMAP;
	}

	if (par->cursor_state.image.height != vc->vc_font.width ||
	    par->cursor_state.image.width != vc->vc_font.height ||
	    par->cursor_reset) {
		par->cursor_state.image.height = vc->vc_font.width;
		par->cursor_state.image.width = vc->vc_font.height;
		cursor.set |= FB_CUR_SETSIZE;
	}

	dx = vxres - ((y * vc->vc_font.height) + vc->vc_font.height);
	dy = vc->state.x * vc->vc_font.width;

	if (par->cursor_state.image.dx != dx ||
	    par->cursor_state.image.dy != dy ||
	    par->cursor_reset) {
		par->cursor_state.image.dx = dx;
		par->cursor_state.image.dy = dy;
		cursor.set |= FB_CUR_SETPOS;
	}

	if (par->cursor_state.hot.x || par->cursor_state.hot.y ||
	    par->cursor_reset) {
		par->cursor_state.hot.x = cursor.hot.y = 0;
		cursor.set |= FB_CUR_SETHOT;
	}

	if (cursor.set & FB_CUR_SETSIZE ||
	    vc->vc_cursor_type != par->p->cursor_shape ||
	    par->cursor_state.mask == NULL ||
	    par->cursor_reset) {
		char *tmp, *mask = kmalloc_array(w, vc->vc_font.width,
						 GFP_ATOMIC);
		int cur_height, size, i = 0;
		int width = (vc->vc_font.width + 7)/8;

		if (!mask)
			return;

		tmp = kmalloc_array(width, vc->vc_font.height, GFP_ATOMIC);

		if (!tmp) {
			kfree(mask);
			return;
		}

		kfree(par->cursor_state.mask);
		par->cursor_state.mask = mask;

		par->p->cursor_shape = vc->vc_cursor_type;
		cursor.set |= FB_CUR_SETSHAPE;

		switch (CUR_SIZE(par->p->cursor_shape)) {
		case CUR_NONE:
			cur_height = 0;
			break;
		case CUR_UNDERLINE:
			cur_height = (vc->vc_font.height < 10) ? 1 : 2;
			break;
		case CUR_LOWER_THIRD:
			cur_height = vc->vc_font.height/3;
			break;
		case CUR_LOWER_HALF:
			cur_height = vc->vc_font.height >> 1;
			break;
		case CUR_TWO_THIRDS:
			cur_height = (vc->vc_font.height << 1)/3;
			break;
		case CUR_BLOCK:
		default:
			cur_height = vc->vc_font.height;
			break;
		}

		size = (vc->vc_font.height - cur_height) * width;
		while (size--)
			tmp[i++] = 0;
		size = cur_height * width;
		while (size--)
			tmp[i++] = 0xff;
		memset(mask, 0, w * vc->vc_font.width);
		rotate_cw(tmp, mask, vc->vc_font.width, vc->vc_font.height);
		kfree(tmp);
	}

	par->cursor_state.enable = enable && !use_sw;

	cursor.image.data = src;
	cursor.image.fg_color = par->cursor_state.image.fg_color;
	cursor.image.bg_color = par->cursor_state.image.bg_color;
	cursor.image.dx = par->cursor_state.image.dx;
	cursor.image.dy = par->cursor_state.image.dy;
	cursor.image.height = par->cursor_state.image.height;
	cursor.image.width = par->cursor_state.image.width;
	cursor.hot.x = par->cursor_state.hot.x;
	cursor.hot.y = par->cursor_state.hot.y;
	cursor.mask = par->cursor_state.mask;
	cursor.enable = par->cursor_state.enable;
	cursor.image.depth = 1;
	cursor.rop = ROP_XOR;

	if (info->fbops->fb_cursor)
		err = info->fbops->fb_cursor(info, &cursor);

	if (err)
		soft_cursor(info, &cursor);

	par->cursor_reset = 0;
}

static int cw_update_start(struct fb_info *info)
{
	struct fbcon_par *par = info->fbcon_par;
	u32 vxres = GETVXRES(par->p, info);
	u32 xoffset;
	int err;

	xoffset = vxres - (info->var.xres + par->var.yoffset);
	par->var.yoffset = par->var.xoffset;
	par->var.xoffset = xoffset;
	err = fb_pan_display(info, &par->var);
	par->var.xoffset = info->var.xoffset;
	par->var.yoffset = info->var.yoffset;
	par->var.vmode = info->var.vmode;
	return err;
}

static const struct fbcon_bitops cw_fbcon_bitops = {
	.bmove = cw_bmove,
	.clear = cw_clear,
	.putcs = cw_putcs,
	.clear_margins = cw_clear_margins,
	.cursor = cw_cursor,
	.update_start = cw_update_start,
	.rotate_font = fbcon_rotate_font,
};

void fbcon_set_bitops_cw(struct fbcon_par *par)
{
	par->bitops = &cw_fbcon_bitops;
}
]

gemini-3-pro-preview

							input: 19110
							output: 39
							thoughts: 1230
						

**Analyzing the `cw_putcs` Function and Potential Underflow Issues in Clockwise Rotation**

Alright, let's break down this `cw_putcs` function. It seems to be the core of handling character output in a 90-degree clockwise rotated framebuffer. My initial observation is that it sets up an `fb_image` structure and then calls `cw_putcs_aligned`, which eventually calls the framebuffer's `fb_imageblit` function.

The key lies in how the `fb_image` structure is populated for the rotated scenario. We're dealing with a logical-to-physical coordinate transformation. In the standard, unrotated mode, the `dx` and `dy` members of `fb_image` would directly correspond to the x and y coordinates on the display, with `width` and `height` being the dimensions of the image being blitted.

In this rotated context, `cw_putcs` cleverly maps the logical screen coordinates (yy, xx) to the physical coordinates (dx, dy).  Specifically, the code indicates:

-   `image.dx` (physical x) is computed from `yy` (logical row),  which is `vxres - ((yy + 1) * vc->vc_font.height)`.
-   `image.dy` (physical y) is computed from `xx` (logical column) , which is `xx * vc->vc_font.width`.

This directly reflects the 90-degree rotation: the logical rows map to physical columns, and logical columns map to physical rows. The dimensions are also consistent with the rotation: the blitted image width becomes the font height, and the height becomes the count of characters multiplied by the font width.

Now, the concerning part... Looking at the `image.dx` calculation, `vxres - ((yy + 1) * vc->vc_font.height)`, a possible issue is apparent:  what if `(yy + 1) * vc->vc_font.height` exceeds `vxres`? Given that `yy` is signed and `vxres` is unsigned, there's a risk of integer underflow. If this underflow occurs, `image.dx` will become a very large value.

In `fb_imageblit`, `dst.address` is advanced by `image->dy * bits_per_line + image->dx * bpp`. If `image.dx` becomes huge due to the underflow, the destination address `dst` will point far out of bounds. This would explain the observed crash during the write operation at `ffffc900038123c0`, given that the adjacent memory content, `f8`, suggests a write out of the allocated region.

The problem arises in `fb_write_offset`, called from `fb_bitmap_2ppw`, inside of `sys_imageblit`, which is calling `fb_imageblit`.  `fb_bitmap_2ppw` iterates over `height` and `width`. If the dimensions are correct and given that `image.dx` is the issue, then the problem isn't with font size. The issue is with the location.

To confirm this hypothesis, I need to verify that `fb_image`'s `dx` member is indeed an unsigned integer (`u32` usually). Also need to determine the maximum value `vxres`, `vc->vc_font.height`, and `yy` in that call to see how large the underflow may become.

map[ContextFile:include/linux/fb.h IncludeLines:false Name:fb_image]

map[SourceCode:struct fb_image {
	__u32 dx;		/* Where to place image */
	__u32 dy;
	__u32 width;		/* Size of image */
	__u32 height;
	__u32 fg_color;		/* Only used when a mono bitmap */
	__u32 bg_color;
	__u8  depth;		/* Depth of the image */
	const char *data;	/* Pointer to image data */
	struct fb_cmap cmap;	/* color map info */
};
 SourceFile:include/uapi/linux/fb.h]

gemini-3-pro-preview

							input: 21287
							output: 45
							thoughts: 1043
						

**Analyzing a Potential Integer Overflow in Framebuffer Rotation**

Okay, so I'm digging into this framebuffer issue, and it looks like a classic integer overflow is the culprit. We've got `dx` and `dy` as `__u32` (unsigned 32-bit integers), which is the key here. The problem seems to be originating in `cw_putcs`, specifically this line:

`image.dx = vxres - ((yy + 1) * vc->vc_font.height);`

where if `(yy + 1) * vc->vc_font.height` exceeds `vxres` that the resulting calculation causes a wrap around, giving a huge value for `image.dx`.

`yy` is the row index and is passed as an argument to `cw_putcs`. Now, `vxres` is supposed to represent the screen's "width" or, rather, the horizontal resolution *before* rotation, because this code is used with rotated consoles. I need to keep straight here. The physical screen dimensions (`info->var.xres`, `info->var.yres`) usually stay fixed. When the console rotates, `fbcon` reinterprets rows and columns but those values are often not updated. So, a rotation affects the logical coordinates (how we *think* the screen is laid out) but doesn't necessarily change the physical dimensions reported by `info->var`.

Thinking this through, if we have a 90-degree clockwise rotation:

*   Logical (0,0) maps to Physical (Width, 0)
*   Logical (0,1) maps to Physical (Width, 1) and so on...

So increasing `yy` (logical row) means we're moving "left" on the physical screen (decreasing physical X), so `vxres - (yy * height)` makes sense.

The use of `(yy + 1) * height` is meant to account for this. If `yy = 0`, then `dx = vxres - height`, which is what we would expect if the character is 'height' pixels wide. The critical part is if `yy` is the last row, should `(yy + 1) * height` be close to `0` or `vxres`? It should be closer to `0`. If `(yy + 1) * height` is larger than `vxres`, `image.dx` underflows wrapping around into a very large value. This would imply `yy` is too large for the current `vxres` value.

The crucial point here is that `vc->vc_rows` is critical. It should be calculated according to the screen dimensions and it is important that `fbcon` updates it to match the new orientation when rotation is enabled. If `fbcon` thinks the screen is larger than it actually is, then we'll have this problem. Likewise, if the reported `vxres` is less than expected we'll also have a problem.

The KASAN report points the finger at `fb_imageblit`, called from `sys_imageblit`. It's a write operation that's failing. A huge `image.dx` will cause `dst` in `fb_imageblit` to be advanced way too much:

`fb_address_forward(&dst, image->dy * bits_per_line + image->dx * bpp);`

Specifically, the problem seems to be the `dst.address` plus the offset calculation that results in memory outside the accessible region. The address `ffffc900038123c0` is a vmalloc address. The adjacent memory is inaccessible, which suggests either the base address is valid but the offset makes the write out of bounds, or `dst.address` is already invalid. However, because this is a vmalloc address, it's more likely that a large `image.dx` is creating an out-of-bounds write within the vmalloc area, rather than a totally random address. If `dst.address` were a huge, wrap-around address, it'd likely lead to a GPF or access to user-space addresses. It's likely that `image.dx` isn't *completely* huge (e.g., `0xFFFFFFFF`), but it's large enough to overflow the bounds of the valid memory. It's also possible that `image.dx` is small negative (when interpreted as a signed integer), which is huge when interpreted as an unsigned integer.

I need to look closer at `GETVXRES`, it's a macro, and figure out where that's defined, which likely is in a `fbcon_rotate.h` or something of that ilk. Also, I'll check `drivers/video/fbdev/core

requested entity does not exist

map[ContextFile:drivers/video/fbdev/core/fbcon.h IncludeLines:false Name:GETVXRES]

map[SourceCode: SourceFile:]

gemini-3-pro-preview

							input: 23421
							output: 30
							thoughts: 69
						

**Investigating the fbcon_cw.c File**

Okay, so I'm looking at this `fbcon_cw.c` file, specifically within the `drivers/video/fbdev/core` directory. I see it includes `fbcon.h` and, interestingly, `fbcon_rotate.h`. Hmm, that suggests this code is related to console rotation within the framebuffer device. Given that context, I'm going to shift my focus towards that `fbcon_rotate.h` file. I need to understand how the rotation functionality is implemented here, and that header file is my obvious first point of investigation. Let's see what's in there.

map[File:drivers/video/fbdev/core/fbcon_rotate.h]

map[Contents:/*
 *  linux/drivers/video/console/fbcon_rotate.h -- Software Display Rotation
 *
 *	Copyright (C) 2005 Antonino Daplas <adaplas@pol.net>
 *
 *  This file is subject to the terms and conditions of the GNU General Public
 *  License.  See the file COPYING in the main directory of this archive
 *  for more details.
 */

#ifndef _FBCON_ROTATE_H
#define _FBCON_ROTATE_H

#define GETVYRES(s,i) ({                           \
        (fb_scrollmode(s) == SCROLL_REDRAW || fb_scrollmode(s) == SCROLL_MOVE) ? \
        (i)->var.yres : (i)->var.yres_virtual; })

#define GETVXRES(s,i) ({                           \
        (fb_scrollmode(s) == SCROLL_REDRAW || fb_scrollmode(s) == SCROLL_MOVE || !(i)->fix.xpanstep) ? \
        (i)->var.xres : (i)->var.xres_virtual; })


static inline int pattern_test_bit(u32 x, u32 y, u32 pitch, const char *pat)
{
	u32 tmp = (y * pitch) + x, index = tmp / 8,  bit = tmp % 8;

	pat +=index;
	return (*pat) & (0x80 >> bit);
}

static inline void pattern_set_bit(u32 x, u32 y, u32 pitch, char *pat)
{
	u32 tmp = (y * pitch) + x, index = tmp / 8, bit = tmp % 8;

	pat += index;

	(*pat) |= 0x80 >> bit;
}

static inline void rotate_ud(const char *in, char *out, u32 width, u32 height)
{
	int i, j;
	int shift = (8 - (width % 8)) & 7;

	width = (width + 7) & ~7;

	for (i = 0; i < height; i++) {
		for (j = 0; j < width - shift; j++) {
			if (pattern_test_bit(j, i, width, in))
				pattern_set_bit(width - (1 + j + shift),
						height - (1 + i),
						width, out);
		}

	}
}

static inline void rotate_cw(const char *in, char *out, u32 width, u32 height)
{
	int i, j, h = height, w = width;
	int shift = (8 - (height % 8)) & 7;

	width = (width + 7) & ~7;
	height = (height + 7) & ~7;

	for (i = 0; i < h; i++) {
		for (j = 0; j < w; j++) {
			if (pattern_test_bit(j, i, width, in))
				pattern_set_bit(height - 1 - i - shift, j,
						height, out);

		}
	}
}

static inline void rotate_ccw(const char *in, char *out, u32 width, u32 height)
{
	int i, j, h = height, w = width;
	int shift = (8 - (width % 8)) & 7;

	width = (width + 7) & ~7;
	height = (height + 7) & ~7;

	for (i = 0; i < h; i++) {
		for (j = 0; j < w; j++) {
			if (pattern_test_bit(j, i, width, in))
				pattern_set_bit(i, width - 1 - j - shift,
						height, out);
		}
	}
}

int fbcon_rotate_font(struct fb_info *info, struct vc_data *vc);

#if defined(CONFIG_FRAMEBUFFER_CONSOLE_ROTATION)
void fbcon_set_bitops_cw(struct fbcon_par *par);
void fbcon_set_bitops_ud(struct fbcon_par *par);
void fbcon_set_bitops_ccw(struct fbcon_par *par);
#else
static inline void fbcon_set_bitops_cw(struct fbcon_par *par)
{ }
static inline void fbcon_set_bitops_ud(struct fbcon_par *par)
{ }
static inline void fbcon_set_bitops_ccw(struct fbcon_par *par)
{ }
#endif

#endif
]

gemini-3-pro-preview

							input: 24795
							output: 46
							thoughts: 440
						

**Analyzing a Graphics Driver Crash During Rotation**

Okay, so I'm looking at this crash and it seems to be happening during a rotation operation, specifically when the user is likely changing the rotation mode at runtime through sysfs. The code snippet I'm seeing relates to calculating a horizontal coordinate or offset, let's call it `vxres`. It's defined to return either the physical horizontal resolution (`xres`) or the virtual horizontal resolution (`xres_virtual`), depending on the framebuffer's scrolling mode and whether horizontal panning is enabled. The core of the issue seems to be how these resolutions are handled and potentially, how they are updated during rotation.

The key line of code that's problematic is calculating `image.dx` which involves `vxres`. The calculation is: `vxres - ((yy + 1) * vc->vc_font.height)`. My initial thought is that if the row index `yy` refers to a row beyond what `vxres` understands as the current screen bounds, the calculation will result in `dx` becoming significantly large, potentially leading to an integer underflow.

The console subsystem (`vt`) is responsible for managing the number of rows (`vc->vc_rows`) and columns (`vc->vc_cols`). When rotation happens, `fbcon` informs `vt` about the updated dimensions. This is where things get interesting. From what I recall, If the framebuffer rotates 90 degrees, the roles of rows and columns are effectively swapped. The new `cols` value is tied to the physical height (`yres`), and the new `rows` value corresponds to the physical width (`xres`).

The initial setup for these dimensions is handled by either `cw_startup` (if present) or `fbcon_startup`. The crash trace points to a sequence involving `rotate_store` followed by `fbcon_rotate`. This suggests that the issue might stem from how the framebuffer's virtual dimensions are being updated during this rotation. It looks like the system is in the middle of reconfiguring the console.

My hunch is that `fbcon_rotate` is the crucial function, and it's likely responsible for updating `info->var` and also calls some other functions that are responsible to redraw the screen (like `fbcon_set_all_vcs(info)` or `redraw_screen`). I need to dig deeper into the code within `fbcon_rotate` and, more importantly, its interactions with `fbcon_set_all_vcs()` or `redraw_screen()` to see how it adjusts the virtual framebuffer dimensions and how these changes impact the subsequent calculations of `vxres`. I need to verify that `vt` is properly informed of the new `rows` and `cols` after the rotation.

map[ContextFile:drivers/video/fbdev/core/fbcon.c IncludeLines:true Name:fbcon_rotate]

map[SourceCode: 213:	static void fbcon_rotate(struct fb_info *info, u32 rotate)
 214:	{
 215:		struct fbcon_par *par = info->fbcon_par;
 216:		struct fb_info *fb_info;
 217:	
 218:		if (!par || par->currcon == -1)
 219:			return;
 220:	
 221:		fb_info = fbcon_info_from_console(par->currcon);
 222:	
 223:		if (info == fb_info) {
 224:			struct fbcon_display *p = &fb_display[par->currcon];
 225:	
 226:			if (rotate < 4)
 227:				p->con_rotate = rotate;
 228:			else
 229:				p->con_rotate = 0;
 230:	
 231:			fbcon_modechanged(info);
 232:		}
 233:	}
 SourceFile:drivers/video/fbdev/core/fbcon.c]

gemini-3-pro-preview

							input: 26181
							output: 47
							thoughts: 126
						

**Investigating the fbcon_rotate Call**

Okay, so I requested a trace of `fbcon_rotate`, but the trace is showing that `rotate_store` is *calling* `fbcon_rotate`. That's a little unexpected, but not unheard of given the structure I'm familiar with in the framebuffer console drivers. Let me confirm the relationship.

Looking at the trace output again:

```
 fbcon_rotate drivers/video/fbdev/core/fbcon.c:231 [inline]
 rotate_store+0x33f/0x510 drivers/video/fbdev/core/fbcon.c:3237
```

Right, confirming that `fbcon_rotate` is indeed inlined and called from within `rotate_store`.  And that line 231 of `fbcon.c`, which is where `fbcon_rotate` is defined, calls `fbcon_modechanged(info)`. Given the context, this is starting to paint a clearer picture of the control flow related to mode changes.

I need to understand what's happening within `fbcon_modechanged` and how it relates to this rotation mechanism. Let's delve into that function definition next.

map[ContextFile:drivers/video/fbdev/core/fbcon.c IncludeLines:true Name:fbcon_modechanged]

map[SourceCode:2692:	static void fbcon_modechanged(struct fb_info *info)
2693:	{
2694:		struct fbcon_par *par = info->fbcon_par;
2695:		struct vc_data *vc;
2696:		struct fbcon_display *p;
2697:		int rows, cols;
2698:	
2699:		if (!par || par->currcon < 0)
2700:			return;
2701:		vc = vc_cons[par->currcon].d;
2702:		if (vc->vc_mode != KD_TEXT ||
2703:		    fbcon_info_from_console(par->currcon) != info)
2704:			return;
2705:	
2706:		p = &fb_display[vc->vc_num];
2707:		set_blitting_type(vc, info);
2708:	
2709:		if (con_is_visible(vc)) {
2710:			var_to_display(p, &info->var, info);
2711:			cols = FBCON_SWAP(par->rotate, info->var.xres, info->var.yres);
2712:			rows = FBCON_SWAP(par->rotate, info->var.yres, info->var.xres);
2713:			cols /= vc->vc_font.width;
2714:			rows /= vc->vc_font.height;
2715:			vc_resize(vc, cols, rows);
2716:			updatescrollmode(p, info, vc);
2717:			scrollback_max = 0;
2718:			scrollback_current = 0;
2719:	
2720:			if (fbcon_is_active(vc, info)) {
2721:				par->var.xoffset = par->var.yoffset = p->yscroll = 0;
2722:				par->bitops->update_start(info);
2723:			}
2724:	
2725:			fbcon_set_palette(vc, color_table);
2726:			update_screen(vc);
2727:		}
2728:	}
 SourceFile:drivers/video/fbdev/core/fbcon.c]

gemini-3-pro-preview

							input: 27256
							output: 48
							thoughts: 336
						

**Analyzing the Framebuffer Console Mode Change**

Okay, so I'm digging into how the framebuffer console handles mode changes, and it's got me thinking. `fbcon_modechanged` kicks things off, right? It calls `vc_resize(vc, cols, rows)` and then `update_screen(vc)`. No surprises there. `update_screen` then calls `redraw_screen`, which is where the magic (or potential problems) happen, eventually leading to `do_update_region` and finally `fbcon_putcs`. Standard stuff, so far.

But the crux of the matter is *how* the dimensions are calculated within `fbcon_modechanged`:

```c
			cols = FBCON_SWAP(par->rotate, info->var.xres, info->var.yres);
			rows = FBCON_SWAP(par->rotate, info->var.yres, info->var.xres);
			cols /= vc->vc_font.width;
			rows /= vc->vc_font.height;
			vc_resize(vc, cols, rows);
```

Specifically, it uses `par->rotate`, and that's the key. This value is used within the swap function to determine how `xres` and `yres` are handled and ultimately affects the new console dimensions. The question that bugs me is, *where* is `par->rotate` updated? It *should* be set *before* `fbcon_modechanged` is even called, right?

Looking further back, there's `fbcon_rotate`, where it looks like this update *should* happen:

```c
			if (rotate < 4)
				p->con_rotate = rotate;
            // ...
			fbcon_modechanged(info);
```

It updates `p->con_rotate`. And `p` is defined as `&fb_display[par->currcon]`. So, is `par->rotate` updated from `p->con_rotate` *somewhere*? Or is `par->rotate` directly modified elsewhere? If the data isn't being flowed correctly, we could have a nasty rotation bug on our hands.

I need to confirm if `par->rotate` is a direct copy from `p->con_rotate`, or some other value.

Let's trace it and see how it is handled in `set_blitting_type` called at line 2707. That is where I will start my digging.

map[ContextFile:drivers/video/fbdev/core/fbcon.c IncludeLines:true Name:set_blitting_type]

map[SourceCode: 692:	static void set_blitting_type(struct vc_data *vc, struct fb_info *info)
 693:	{
 694:		struct fbcon_par *par = info->fbcon_par;
 695:	
 696:		par->p = &fb_display[vc->vc_num];
 697:	
 698:		if ((info->flags & FBINFO_MISC_TILEBLITTING))
 699:			fbcon_set_tileops(vc, info);
 700:		else {
 701:			fbcon_set_rotation(info);
 702:			fbcon_set_bitops(par);
 703:		}
 704:	}
 SourceFile:drivers/video/fbdev/core/fbcon.c]

gemini-3-pro-preview

							input: 28418
							output: 48
							thoughts: 10
						

Okay, here's my attempt at summarizing that thought process, tailored for an expert audience, and written in the first person:

**Analyzing `fbcon_set_rotation`**

Right, so `fbcon_set_rotation`. That function immediately piques my interest. It’s a core piece within the frame buffer console (fbcon) system, which means it’s responsible for manipulating the visual output directly on the physical console, likely without relying on a full X server or Wayland compositor. My initial thought is to dive into *why* rotation is being handled here. Is this for device-specific orientation control on embedded systems? Are we talking about a general feature for desktop consoles? The context is crucial.

I'm starting to think about the underlying framebuffer structure. Is it using a packed pixel format? How are color depths handled? I'd need to consider how `fbcon_set_rotation` interacts with the framebuffer's internal data layout. Are we talking about a simple 90-degree rotation, or are we dealing with arbitrary angles, which would involve more complex transformations and interpolation?

The implementation itself will be key. I'd be looking for potential performance bottlenecks. Memory copy operations are the obvious suspects. If the rotation is complex, are there optimizations using SIMD instructions or hardware acceleration available? Are there any potential issues with double-buffering or synchronization to avoid screen tearing during the rotation process? A quick look at the surrounding code to determine how it is triggered and the available options would be the next logical step.

I need to understand the function’s error handling, too. What happens if the requested rotation is not supported? Does it return an error code, log a warning, or fall back to a default orientation? How does it behave if it's called with invalid arguments, like an unsupported rotation angle? Error handling is critical in this type of low-level code, as it sets up a robust system that can gracefully manage unexpected conditions. I'll need to check the associated `ioctl` interface if applicable.

map[ContextFile:drivers/video/fbdev/core/fbcon.c IncludeLines:true Name:fbcon_set_rotation]

map[SourceCode: 202:	static inline void fbcon_set_rotation(struct fb_info *info)
 203:	{
 204:		struct fbcon_par *par = info->fbcon_par;
 205:	
 206:		if (!(info->flags & FBINFO_MISC_TILEBLITTING) &&
 207:		    par->p->con_rotate < 4)
 208:			par->rotate = par->p->con_rotate;
 209:		else
 210:			par->rotate = 0;
 211:	}
 SourceFile:drivers/video/fbdev/core/fbcon.c]

gemini-3-pro-preview

							input: 29090
							output: 869
							thoughts: 19514
						

**Analysis of a KASAN: vmalloc-out-of-bounds Crash in the Linux Framebuffer Console (fbcon)**

Okay, here's my breakdown of this KASAN crash, and I'll walk you through my thought process. The crash report indicates an out-of-bounds write occurring within the framebuffer console driver, specifically during a rotation operation. The trace points to `fb_write_offset`, which is a low-level function that's part of the core framebuffer handling, the crash occurs when trying to write an 8-byte value. Let's dig in.

The call stack reveals `fb_write_offset` is ultimately reached through `cw_putcs`, a function I recognize as the put character routine optimized for clockwise (CW) 90-degree rotations in `fbcon`. That's important. This function is called by `do_update_region`, which is part of screen redrawing. It appears after a rotation operation, which calls `fbcon_modechanged`, after a `rotate_store` sysfs write. This is the scenario.

So, `cw_putcs` must be involved. This code, and the use of the `GETVXRES` macro is suspicious. We're dealing with a rotated framebuffer where the visual representation is remapped to physical coordinates. Within `cw_putcs`, the `image.dx` coordinate calculation is key: `vxres - ((yy + 1) * vc->vc_font.height)`. Here, `vxres` likely represents the "virtual" horizontal resolution, `yy` is the row index, and `vc->vc_font.height` is the font height.

If the calculated `dx` is negative (underflows), a huge value (close to the maximum value of a 32-bit unsigned integer) is then computed by wrapping around on `dx` with this formula. Then by adding `dx` and `bpp` this causes an out of bounds address. The question becomes, when does `dx` become negative? The calculation requires that `(yy + 1) * height` exceed `vxres` which is derived using `GETVXRES`, for all practical purposes, `xres`.

Okay, the calculation of `yy` is also critical. Now, here's where the problem lies. When a rotation occurs, the console's logical dimensions (rows and columns) change, and we need to handle that change during the resizing phase. The `fbcon_modechanged` function is supposed to resize the console's buffer via `vc_resize` after `rows` and `cols` are computed. `vc_resize` will call `vc_do_resize`, if `cols`, `rows` are different from `vc->vc_cols`, `vc->vc_rows`. But importantly, `vc_resize` can fail, typically because of memory allocation problems (OOM) within the kernel. If it fails, that is the root of the issue!

The bug, I think, is that `fbcon_modechanged` *ignores* the return value of `vc_resize`. It calls `vc_resize` but doesn't check for errors. If `vc_resize` fails, the VC is not resized, and it keeps its old, larger row value which is `vc->vc_rows` or even `vc_rows`, which were passed into `vc_resize` as input. Next, `update_screen` gets called as part of the normal rotation path, the `redraw_screen` function calls `do_update_region` where the row coordinate `yy` is part of `putcs` call. In that context, when `cw_putcs` is called, it still uses the original, large value of rows which the screen can no longer support, hence `(yy + 1) * font_height > vxres` becomes true, and `dx` underflows, and the OOB memory access happens as it writes a character that can no longer fit. The bug then has the effect of an oversized draw.

In summary, the specific steps of the crash are as follows:
1.  Rotation triggers `fbcon_modechanged` which calls `vc_resize` with new, rotated dimensions.
2.  `vc_resize` *fails* for whatever reason (typically OOM if resolution changes too much, it is related to the resolution changes and OOM conditions).
3.  `fbcon_modechanged` proceeds, ignoring the failure.
4.  Subsequent screen updates call `do_update_region` and eventually `cw_putcs` using the *old*, large console row count (`vc->vc_rows`).
5.  Within `cw_putcs`, the `dx` calculation underflows because `yy`, derived from the old row count,

map[KernelScratchSrc:/usr/local/google/home/dvyukov/syzkaller/agent/workdir/cache/tmp/tmp445743226]

gemini-3-pro-preview

You are an experienced Linux kernel developer tasked with creating a fix for a kernel bug.
Use the codeedit tool to do code edits.
Note: you will not see your changes when looking at the code using codesearch tools.
Your final reply should contain explanation of what you did in the patch and why.



Prefer calling several tools at the same time to save round-trips.

The crash that corresponds to the bug is:

R10: 0000000000000000 R11: 0000000000000246 R12: 00007f1d3b43e4bc
R13: 00007f1d3b43e424 R14: 00007ffcf358def0 R15: 0000000000000003
 </TASK>
==================================================================
BUG: KASAN: vmalloc-out-of-bounds in fb_write_offset drivers/video/fbdev/core/sysmem.h:30 [inline]
BUG: KASAN: vmalloc-out-of-bounds in fb_bitmap_2ppw drivers/video/fbdev/core/fb_imageblit.h:364 [inline]
BUG: KASAN: vmalloc-out-of-bounds in fb_bitmap_imageblit drivers/video/fbdev/core/fb_imageblit.h:462 [inline]
BUG: KASAN: vmalloc-out-of-bounds in fb_imageblit drivers/video/fbdev/core/fb_imageblit.h:492 [inline]
BUG: KASAN: vmalloc-out-of-bounds in sys_imageblit+0xfed/0x1ea0 drivers/video/fbdev/core/sysimgblt.c:24
Write of size 8 at addr ffffc900038123c0 by task syz-executor388/6015

CPU: 0 UID: 0 PID: 6015 Comm: syz-executor388 Not tainted syzkaller #1 PREEMPT(full) 
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:94 [inline]
 dump_stack_lvl+0x10e/0x190 lib/dump_stack.c:120
 print_address_description mm/kasan/report.c:378 [inline]
 print_report+0x17e/0x810 mm/kasan/report.c:482
 kasan_report+0x147/0x180 mm/kasan/report.c:595
 fb_write_offset drivers/video/fbdev/core/sysmem.h:30 [inline]
 fb_bitmap_2ppw drivers/video/fbdev/core/fb_imageblit.h:364 [inline]
 fb_bitmap_imageblit drivers/video/fbdev/core/fb_imageblit.h:462 [inline]
 fb_imageblit drivers/video/fbdev/core/fb_imageblit.h:492 [inline]
 sys_imageblit+0xfed/0x1ea0 drivers/video/fbdev/core/sysimgblt.c:24
 drm_fbdev_shmem_defio_imageblit+0x2c/0x110 drivers/gpu/drm/drm_fbdev_shmem.c:39
 cw_putcs_aligned drivers/video/fbdev/core/fbcon_cw.c:110 [inline]
 cw_putcs+0x15af/0x1880 drivers/video/fbdev/core/fbcon_cw.c:158
 fbcon_putcs+0x3bd/0x590 drivers/video/fbdev/core/fbcon.c:1343
 do_update_region+0x3a1/0x460 drivers/tty/vt/vt.c:628
 redraw_screen+0x91a/0xe90 drivers/tty/vt/vt.c:980
 fbcon_rotate drivers/video/fbdev/core/fbcon.c:231 [inline]
 rotate_store+0x33f/0x510 drivers/video/fbdev/core/fbcon.c:3237
 kernfs_fop_write_iter+0x3af/0x540 fs/kernfs/file.c:352
 new_sync_write fs/read_write.c:593 [inline]
 vfs_write+0x5d2/0xb40 fs/read_write.c:686
 ksys_write+0x149/0x250 fs/read_write.c:738
 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
 do_syscall_64+0xe8/0xf80 arch/x86/entry/syscall_64.c:94
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f1d3b3d97c9
Code: c0 79 93 eb d5 48 8d 7c 1d 00 eb 99 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 d8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffcf358dea8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f1d3b3d97c9
RDX: 0000000000000007 RSI: 0000000000000000 RDI: 0000000000000003
RBP: 00007ffcf358def0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007f1d3b43e4bc
R13: 00007f1d3b43e424 R14: 00007ffcf358def0 R15: 0000000000000003
 </TASK>

The buggy address belongs to a vmalloc virtual mapping
Memory state around the buggy address:
 ffffc90003812280: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
 ffffc90003812300: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
>ffffc90003812380: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
                                           ^
 ffffc90003812400: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
 ffffc90003812480: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
==================================================================


The explanation of the root cause of the bug is:

The root cause of this crash is a missing error check for `vc_resize()` in `fbcon_modechanged()`, which leads to an integer underflow in `cw_putcs()` (Clockwise rotation blit function) when the console resizing fails.

Here is the step-by-step detailed explanation:

1.  **Context**: The crash occurs when `fbcon` is rotated (e.g., 90 degrees clockwise) via `rotate_store`. This triggers `fbcon_modechanged()`.
2.  **Logic Flaw in `fbcon_modechanged`**:
    *   When the screen is rotated, the logical dimensions (rows and columns) of the console change. For example, if rotating a "portrait" screen (Height > Width) to 90 degrees, the number of text rows must decrease (since the new width is the old height, which is smaller).
    *   `fbcon_modechanged()` calculates the new dimensions (`cols` and `rows`) and calls `vc_resize(vc, cols, rows)` to resize the console buffer.
    *   **The bug:** `fbcon_modechanged()` **ignores the return value** of `vc_resize()`.
    *   If `vc_resize()` fails (e.g., due to a temporary memory allocation failure), the console (`vc`) remains with its *old* dimensions. In our example, `vc->vc_rows` remains large.
    *   Despite the failure, the function proceeds to call `update_screen(vc)`, which triggers a redraw.

3.  **The Crash in `cw_putcs`**:
    *   `update_screen()` calls `redraw_screen()`, which calls `do_update_region()`.
    *   `do_update_region()` iterates through the console buffer using the *current* `vc->vc_rows`. Since the resize failed, it uses the **old (large)** row count.
    *   It calls `fbcon_putcs()`, which delegates to `cw_putcs()` (because rotation is active).
    *   `cw_putcs()` calculates the destination X coordinate (`image.dx`) in the framebuffer:
        ```c
        image.dx = vxres - ((yy + 1) * vc->vc_font.height);
        ```
        *   `vxres` is the physical screen width (which is small in this scenario).
        *   `yy` is the row index passed from `do_update_region()`.
    *   Because `vc` was not resized, `yy` can be large (valid for the old dimensions but too large for the new rotated orientation).
    *   Eventually, `(yy + 1) * height` becomes larger than `vxres`.
    *   The subtraction `vxres - (large_value)` causes an **unsigned integer underflow**, resulting in a massive value for `image.dx`.

4.  **Memory Corruption**:
    *   `cw_putcs()` calls `cw_putcs_aligned()`, which calls `fb_imageblit()`.
    *   `fb_imageblit()` calculates the memory address to write to:
        ```c
        dst = base + image.dy * stride + image.dx * bpp;
        ```
    *   Since `image.dx` is huge due to the underflow, `dst` points far outside the allocated framebuffer (vmalloc area).
    *   `fb_bitmap_2ppw()` (called by `fb_imageblit`) attempts to write to this invalid address via `fb_write_offset()`, triggering the **KASAN: vmalloc-out-of-bounds** crash.

**Summary**: The kernel crashes because `fbcon` attempts to draw rows that don't fit on the rotated screen after a console resize failure. The fix is to check the return value of `vc_resize()` in `fbcon_modechanged()` and abort the screen update if it fails.

The bug is a KASAN vmalloc-out-of-bounds write in `cw_putcs`, which is invoked during a screen update after a framebuffer mode change (specifically, rotation).

When the framebuffer mode changes (e.g., rotation), `fbcon_modechanged` calculates the new number of columns and rows required for the console and calls `vc_resize` to resize the console buffer. The original code failed to check the return value of `vc_resize`.

If `vc_resize` fails (e.g., due to memory allocation failure), the `vc` structure retains its old dimensions (e.g., a large number of rows for a portrait mode). However, the function proceeds to call `update_screen`, which triggers a redraw. The redraw logic (`cw_putcs` for clockwise rotation) uses the *old* (large) row count from `vc` but the *new* (small) screen width/height from `info`.

In `cw_putcs`, the destination offset is calculated as `vxres - ((yy + 1) * font_height)`. If `yy` (the current row) corresponds to a row index that fits in the old dimensions but exceeds the new rotated height, the subtraction results in an unsigned integer underflow (since `(yy + 1) * font_height` > `vxres`). This underflow produces a huge memory offset, causing the out-of-bounds write.

The fix is to verify the return value of `vc_resize` in `fbcon_modechanged`. If resizing fails, the function should return immediately to avoid calling `update_screen` with inconsistent dimensions, thus preventing the crash.

I applied the following change to `drivers/video/fbdev/core/fbcon.c`:

```c
<<<<
		cols /= vc->vc_font.width;
		rows /= vc->vc_font.height;
		vc_resize(vc, cols, rows);
		updatescrollmode(p, info, vc);
====
		cols /= vc->vc_font.width;
		rows /= vc->vc_font.height;
		if (vc_resize(vc, cols, rows))
			return;
		updatescrollmode(p, info, vc);
>>>>
```

gemini-3-pro-preview

							input: 5519
							output: 27
							thoughts: 202
						

**Fixing a Kernel Bug in fbcon.c**

Alright, let's dive into this. I've got a kernel bug to squash in `drivers/video/fbdev/core/fbcon.c`. Apparently, there's a missing error check after a call to `vc_resize()` within `fbcon_modechanged()`. The problem is, if `vc_resize()` fails, the console dimensions aren't updated correctly, and that leads to an integer underflow later on in `cw_putcs()` when `update_screen()` is called with some bad values. Classic integer overflow!

So, the fix is straightforward: check the return value of `vc_resize()` and bail out of the update process if it fails. Easy peasy. But, I'm not going to rush this. I need to get this right.

First, I need to make sure I'm in the right directory, so let's get a file listing in that `drivers/video/fbdev/core/` directory. Double-checking paths is always a good habit. Once I'm certain, I need to pinpoint the location of `fbcon_modechanged` in `fbcon.c` and truly understand how `vc_resize` is called, and, more importantly, *where* `update_screen` is called. That context is crucial for ensuring the fix is correctly placed and doesn't introduce any new regressions.

Once I've got a solid grasp of the surrounding code, I'll then carefully implement the check. Careful and precise, because kernel work demands it. This seems like a relatively contained issue, so, hopefully, I can get this done quickly.

map[Dir:drivers/video/fbdev/core/]

map[Files:[bitblit.c cfbcopyarea.c cfbfillrect.c cfbimgblt.c cfbmem.h fb_backlight.c fb_chrdev.c fb_cmdline.c fb_copyarea.h fb_ddc.c fb_defio.c fb_draw.h fb_fillrect.h fb_imageblit.h fb_info.c fb_internal.h fb_io_fops.c fb_logo.c fb_notify.c fb_procfs.c fb_sys_fops.c fbcmap.c fbcon.c fbcon.h fbcon_ccw.c fbcon_cw.c fbcon_rotate.c fbcon_rotate.h fbcon_ud.c fbcvt.c fbmem.c fbmon.c fbsysfs.c modedb.c softcursor.c svgalib.c syscopyarea.c sysfillrect.c sysimgblt.c sysmem.h tileblit.c] Subdirs:<nil>]

gemini-3-pro-preview

							input: 6469
							output: 47
							thoughts: 46
						

Here's my take on the thought process:

**Diving into the Framebuffer Console Core**

Alright, the framebuffer console core code is where I need to be. Specifically, `drivers/video/fbdev/core/fbcon.c` seems to be the right place to start. Now, I need to get my bearings and understand how mode changes are handled. I'm going to jump into `fbcon_modechanged` within that file and see how the console reacts to changes in video mode. I'll focus on how this function adapts the console to a new screen resolution, color depth, or refresh rate. Understanding this should give me a solid grasp of how the low-level console interacts with the framebuffer device after a resolution change.

map[ContextFile:drivers/video/fbdev/core/fbcon.c IncludeLines:true Name:fbcon_modechanged]

map[SourceCode:2692:	static void fbcon_modechanged(struct fb_info *info)
2693:	{
2694:		struct fbcon_par *par = info->fbcon_par;
2695:		struct vc_data *vc;
2696:		struct fbcon_display *p;
2697:		int rows, cols;
2698:	
2699:		if (!par || par->currcon < 0)
2700:			return;
2701:		vc = vc_cons[par->currcon].d;
2702:		if (vc->vc_mode != KD_TEXT ||
2703:		    fbcon_info_from_console(par->currcon) != info)
2704:			return;
2705:	
2706:		p = &fb_display[vc->vc_num];
2707:		set_blitting_type(vc, info);
2708:	
2709:		if (con_is_visible(vc)) {
2710:			var_to_display(p, &info->var, info);
2711:			cols = FBCON_SWAP(par->rotate, info->var.xres, info->var.yres);
2712:			rows = FBCON_SWAP(par->rotate, info->var.yres, info->var.xres);
2713:			cols /= vc->vc_font.width;
2714:			rows /= vc->vc_font.height;
2715:			vc_resize(vc, cols, rows);
2716:			updatescrollmode(p, info, vc);
2717:			scrollback_max = 0;
2718:			scrollback_current = 0;
2719:	
2720:			if (fbcon_is_active(vc, info)) {
2721:				par->var.xoffset = par->var.yoffset = p->yscroll = 0;
2722:				par->bitops->update_start(info);
2723:			}
2724:	
2725:			fbcon_set_palette(vc, color_table);
2726:			update_screen(vc);
2727:		}
2728:	}
 SourceFile:drivers/video/fbdev/core/fbcon.c]

gemini-3-pro-preview

							input: 7353
							output: 36
							thoughts: 177
						

**Analyzing the Screen Resize Routine**

Okay, so I'm looking at this snippet of C code and something immediately jumps out. We've got a resize operation (`vc_resize`), followed by some screen updates. The problem is, `vc_resize` returns an integer, which likely signifies success or failure. But the code doesn't *check* that return value! If `vc_resize` fails – say, it encounters some memory allocation issue or an unsupported resolution – it's going to return an error, presumably non-zero. And yet, the code merrily barrels on to `updatescrollmode` and, critically, `update_screen`, assuming `vc` has been resized. That's a disaster waiting to happen. We're going to be operating on potentially invalid screen dimensions.

My immediate thought is, we need to capture the return from `vc_resize`. If it's anything other than zero (or whatever constant represents success), we *have* to bail. Returning early from the routine altogether would probably be the safest approach. At the very least, we should skip calling `update_screen` and any other functions that depend on the new dimensions. This is crucial for avoiding any segfaults or corrupted display behavior.

To be absolutely sure about the expected behavior, I should probably dig into the definition of `vc_resize` to see exactly what values it returns in different scenarios. That will help nail down exactly what signals an error condition. We need to handle that gracefully.

map[ContextFile:drivers/tty/vt/vt.c Name:vc_resize]

map[Comment: Kind:function]

gemini-3-pro-preview

							input: 7901
							output: 41
							thoughts: 29
						

Okay, here's my take on those thoughts, framed as an expert's internal monologue:

**Investigating the Unexplained**

Alright, no comment, eh? That's always a bit of a red flag. Let's dig in and see if we can suss out the purpose behind this code. The signature and return values should provide some critical context. It's time to check the source, specifically in `drivers/tty/vt/vt.c`. That file is the most likely location, given that it's dealing with virtual terminals. Understanding how this function is *used* is the key to filling in that missing comment. Knowing the input and expected output from the function will quickly make the purpose and behaviour more obvious. Let's see what we can find!

map[ContextFile:drivers/tty/vt/vt.c IncludeLines:false Name:vc_resize]

map[SourceCode:static inline int vc_resize(struct vc_data *vc, unsigned int cols,
			    unsigned int lines)
{
	return __vc_resize(vc, cols, lines, false);
}
 SourceFile:include/linux/vt_kern.h]

gemini-3-pro-preview

							input: 8212
							output: 140
							thoughts: 5970
						

**My Analysis of the `fbcon_modechanged` Function**

Okay, so I'm looking at this `fbcon_modechanged` function, which seems to be critical for handling screen mode changes in the framebuffer console. My initial thought is that it's calling `vc_resize` to adjust the virtual console's dimensions to match the new screen dimensions. If `vc_resize` fails, it seems likely that the virtual console (`vc`) is left with its *old* dimensions, while the framebuffer info (`info`) reflects the *new* dimensions, which could be a recipe for disaster.

The code calculates the new number of columns and rows based on the new resolution (`info->var.xres`, `info->var.yres`) and the font size. Then, it calls `vc_resize`. The problem is, if `vc_resize` fails (which I see is possible by its non-zero return code), the function continues, and the console drawing (`update_screen`) will use the *old* rows from the vc in drawing functions like `cw_putcs`, which uses information in the `vc` combined with the geometry in `info`, leading to an out-of-bounds access. The crash is caused by the calculation `image.dx = vxres - ((yy + 1) * vc->vc_font.height)`, where `yy` is the row and if `yy` is large because the `vc` rows are too big, but the screen width `vxres` is small, it leads to an underflow, potentially triggering a crash.

My first impression is that if `vc_resize` fails, it's essential that we bail out of this function *immediately* to prevent this crash. Returning early seems like the only sane option, since continuing would lead to a likely kernel panic. The simplest fix is to add a check for the return value of `vc_resize` and return if it's not zero (meaning an error).

However, I'm also thinking about the side effects of this return. `updatescrollmode`, `fbcon_set_palette`, etc. are skipped. But since `updatescrollmode` relies on a matching `vc` and `info`, leaving them in a inconsistent state is not safe. The failure of `vc_resize` essentially means the resize failed and now the sizes are inconsistent. Therefore skipping `update_scroll_mode`, etc. seems correct.

I'm confident the proposed fix is sound:

```c
			if (vc_resize(vc, cols, rows))
				return;
```

This prevents the immediate crash, even though it leaves the console in an inconsistent state. The best we can hope for is that the system can recover. The user would need to take action to get a display with the correct sizes.

Now, I'll go ahead and apply that change to the file.

map[CurrentCode:		cols /= vc->vc_font.width;
		rows /= vc->vc_font.height;
		vc_resize(vc, cols, rows);
		updatescrollmode(p, info, vc); NewCode:		cols /= vc->vc_font.width;
		rows /= vc->vc_font.height;
		if (vc_resize(vc, cols, rows))
			return;
		updatescrollmode(p, info, vc); SourceFile:drivers/video/fbdev/core/fbcon.c]

map[]

gemini-3-pro-preview

							input: 14939
							output: 47
							thoughts: 21
						

Okay, here's my summary, presented as a stream of thought:

**Verification in Progress**

Alright, the fix is in place. Now, before I move on, I need to verify that everything's as it should be. I'll meticulously check the surrounding code, the lines immediately before and after the modification. I'm looking for any unexpected side effects, any remnants of the old code, anything that suggests the fix didn't fully take or introduced new, subtle issues. A quick glance won't cut it; I need a detailed inspection to ensure the fix seamlessly integrates and functions as intended within the context of the larger file. I need to be absolutely certain that this doesn't create any new problems that could be more complex later.

map[ContextFile:drivers/video/fbdev/core/fbcon.c IncludeLines:true Name:fbcon_modechanged]

map[SourceCode:2692:	static void fbcon_modechanged(struct fb_info *info)
2693:	{
2694:		struct fbcon_par *par = info->fbcon_par;
2695:		struct vc_data *vc;
2696:		struct fbcon_display *p;
2697:		int rows, cols;
2698:	
2699:		if (!par || par->currcon < 0)
2700:			return;
2701:		vc = vc_cons[par->currcon].d;
2702:		if (vc->vc_mode != KD_TEXT ||
2703:		    fbcon_info_from_console(par->currcon) != info)
2704:			return;
2705:	
2706:		p = &fb_display[vc->vc_num];
2707:		set_blitting_type(vc, info);
2708:	
2709:		if (con_is_visible(vc)) {
2710:			var_to_display(p, &info->var, info);
2711:			cols = FBCON_SWAP(par->rotate, info->var.xres, info->var.yres);
2712:			rows = FBCON_SWAP(par->rotate, info->var.yres, info->var.xres);
2713:			cols /= vc->vc_font.width;
2714:			rows /= vc->vc_font.height;
2715:			vc_resize(vc, cols, rows);
2716:			updatescrollmode(p, info, vc);
2717:			scrollback_max = 0;
2718:			scrollback_current = 0;
2719:	
2720:			if (fbcon_is_active(vc, info)) {
2721:				par->var.xoffset = par->var.yoffset = p->yscroll = 0;
2722:				par->bitops->update_start(info);
2723:			}
2724:	
2725:			fbcon_set_palette(vc, color_table);
2726:			update_screen(vc);
2727:		}
2728:	}
 SourceFile:drivers/video/fbdev/core/fbcon.c]

gemini-3-pro-preview

							input: 15798
							output: 490
							thoughts: 1041
						

**Analyzing Code Modification and Ensuring Correctness**

Okay, first things first, I need to verify if the code change I just made with `codeedit` was actually applied. I'm wondering if `codesearch-definition-source` returns the original source or the modified version. Normally, it should show the state on disk, and `codeedit` directly modifies the file. However, the output I'm seeing is identical to the previous one, and line 2715 still shows the original code: `vc_resize(vc, cols, rows);`.

Oh, right! I need to remember that I can't see the changes with codesearch. I forgot about the note, but I can trust `codeedit`'s success. An empty response object usually means it worked, while an error would be returned otherwise.

I *did* use `codeedit` correctly, I'm sure of it. The JSON shows the `CurrentCode` and `NewCode`. I just need to verify that it didn't fail to apply the patch.

The tabs/spaces in `CurrentCode` might have presented an issue, but the source output should use tabs. Let's check a previous output to confirm. Okay, it uses tabs!

The tool's documentation explicitly says to provide full lines of code, including newlines. So, I'm pretty sure I provided the right input to `codeedit`.

I provided:
```c
		cols /= vc->vc_font.width;
		rows /= vc->vc_font.height;
		vc_resize(vc, cols, rows);
		updatescrollmode(p, info, vc);
```
and I believe it should be interpreted as tabs. `codeedit` is smart enough, or it should fail.

The previous `read-file` or `definition-source` output looked like this:
```c
2713:			cols /= vc->vc_font.width;
2714:			rows /= vc->vc_font.height;
2715:			vc_resize(vc, cols, rows);
2716:			updatescrollmode(p, info, vc);
```
Those are tabs.

I'm confident the change was applied because the `codeedit` response was `{}`.

Now, about the underlying issue: The crash is due to an integer underflow in `cw_putcs` (clockwise rotation blit). It happens during `fbcon_modechanged`, when the console dimensions (rows/cols) are recalculated, and `vc_resize()` is called to adjust the console buffer size. The original code didn't check `vc_resize()`'s return value. If it fails (e.g., memory allocation), the console (`vc`) retains its old dimensions, but `fbcon_modechanged` proceeds to call `update_screen()`. `update_screen()` iterates over the *current* `vc` rows. If `vc` has many rows (old state), but the screen is now rotated (new state, reduced height), `cw_putcs` gets called with a row index `yy` that's valid for `vc` but too large for the new screen geometry.

`cw_putcs` calculates the destination offset: `vxres - ((yy + 1) * height)`. If `(yy+1)*height` > `vxres`, the subtraction underflows (unsigned), leading to a massive offset. Writing to this offset triggers the KASAN vmalloc-out-of-bounds error.

My fix involved checking `vc_resize()`'s return value. If it fails, I return immediately from `fbcon_modechanged()`, skipping `update_screen()` and other operations that rely on a successful resize. This directly prevents the crash.

My patch:
```c
-		vc_resize(vc, cols, rows);
+		if (vc_resize(vc, cols, rows))
+			return;
```

Effectively, this stops the update chain if resizing fails.

map[PatchDiff:diff --git a/drivers/video/fbdev/core/fbcon.c b/drivers/video/fbdev/core/fbcon.c
index 7be9e8653..b9e418747 100644
--- a/drivers/video/fbdev/core/fbcon.c
+++ b/drivers/video/fbdev/core/fbcon.c
@@ -2712,7 +2712,8 @@ static void fbcon_modechanged(struct fb_info *info)
 		rows = FBCON_SWAP(par->rotate, info->var.yres, info->var.xres);
 		cols /= vc->vc_font.width;
 		rows /= vc->vc_font.height;
-		vc_resize(vc, cols, rows);
+		if (vc_resize(vc, cols, rows))
+			return;
 		updatescrollmode(p, info, vc);
 		scrollback_max = 0;
 		scrollback_current = 0;
 TestError:]

map[Recipients:[map[Email:deller@gmx.de Name:Helge Deller To:true] map[Email:dri-devel@lists.freedesktop.org Name: To:true] map[Email:linux-fbdev@vger.kernel.org Name: To:true] map[Email:linux-kernel@vger.kernel.org Name: To:false] map[Email:simona@ffwll.ch Name:Simona Vetter To:true]]]

gemini-3-pro-preview

You are an experienced Linux kernel developer tasked with writing a commit description for
a kernel bug fixing commit. The description should start with a one-line summary,
and then include description of the bug being fixed, and how it's fixed by the provided patch.
Your final reply should contain only the text of the commit description.
Phrase the one-line summary so that it is not longer than 72 characters.
The rest of the description must be word-wrapped at 72 characters.

Prefer calling several tools at the same time to save round-trips.

The crash that corresponds to the bug is:

R10: 0000000000000000 R11: 0000000000000246 R12: 00007f1d3b43e4bc
R13: 00007f1d3b43e424 R14: 00007ffcf358def0 R15: 0000000000000003
 </TASK>
==================================================================
BUG: KASAN: vmalloc-out-of-bounds in fb_write_offset drivers/video/fbdev/core/sysmem.h:30 [inline]
BUG: KASAN: vmalloc-out-of-bounds in fb_bitmap_2ppw drivers/video/fbdev/core/fb_imageblit.h:364 [inline]
BUG: KASAN: vmalloc-out-of-bounds in fb_bitmap_imageblit drivers/video/fbdev/core/fb_imageblit.h:462 [inline]
BUG: KASAN: vmalloc-out-of-bounds in fb_imageblit drivers/video/fbdev/core/fb_imageblit.h:492 [inline]
BUG: KASAN: vmalloc-out-of-bounds in sys_imageblit+0xfed/0x1ea0 drivers/video/fbdev/core/sysimgblt.c:24
Write of size 8 at addr ffffc900038123c0 by task syz-executor388/6015

CPU: 0 UID: 0 PID: 6015 Comm: syz-executor388 Not tainted syzkaller #1 PREEMPT(full) 
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:94 [inline]
 dump_stack_lvl+0x10e/0x190 lib/dump_stack.c:120
 print_address_description mm/kasan/report.c:378 [inline]
 print_report+0x17e/0x810 mm/kasan/report.c:482
 kasan_report+0x147/0x180 mm/kasan/report.c:595
 fb_write_offset drivers/video/fbdev/core/sysmem.h:30 [inline]
 fb_bitmap_2ppw drivers/video/fbdev/core/fb_imageblit.h:364 [inline]
 fb_bitmap_imageblit drivers/video/fbdev/core/fb_imageblit.h:462 [inline]
 fb_imageblit drivers/video/fbdev/core/fb_imageblit.h:492 [inline]
 sys_imageblit+0xfed/0x1ea0 drivers/video/fbdev/core/sysimgblt.c:24
 drm_fbdev_shmem_defio_imageblit+0x2c/0x110 drivers/gpu/drm/drm_fbdev_shmem.c:39
 cw_putcs_aligned drivers/video/fbdev/core/fbcon_cw.c:110 [inline]
 cw_putcs+0x15af/0x1880 drivers/video/fbdev/core/fbcon_cw.c:158
 fbcon_putcs+0x3bd/0x590 drivers/video/fbdev/core/fbcon.c:1343
 do_update_region+0x3a1/0x460 drivers/tty/vt/vt.c:628
 redraw_screen+0x91a/0xe90 drivers/tty/vt/vt.c:980
 fbcon_rotate drivers/video/fbdev/core/fbcon.c:231 [inline]
 rotate_store+0x33f/0x510 drivers/video/fbdev/core/fbcon.c:3237
 kernfs_fop_write_iter+0x3af/0x540 fs/kernfs/file.c:352
 new_sync_write fs/read_write.c:593 [inline]
 vfs_write+0x5d2/0xb40 fs/read_write.c:686
 ksys_write+0x149/0x250 fs/read_write.c:738
 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
 do_syscall_64+0xe8/0xf80 arch/x86/entry/syscall_64.c:94
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f1d3b3d97c9
Code: c0 79 93 eb d5 48 8d 7c 1d 00 eb 99 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 d8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffcf358dea8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f1d3b3d97c9
RDX: 0000000000000007 RSI: 0000000000000000 RDI: 0000000000000003
RBP: 00007ffcf358def0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00007f1d3b43e4bc
R13: 00007f1d3b43e424 R14: 00007ffcf358def0 R15: 0000000000000003
 </TASK>

The buggy address belongs to a vmalloc virtual mapping
Memory state around the buggy address:
 ffffc90003812280: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
 ffffc90003812300: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
>ffffc90003812380: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
                                           ^
 ffffc90003812400: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
 ffffc90003812480: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
==================================================================


The explanation of the root cause of the bug is:

The root cause of this crash is a missing error check for `vc_resize()` in `fbcon_modechanged()`, which leads to an integer underflow in `cw_putcs()` (Clockwise rotation blit function) when the console resizing fails.

Here is the step-by-step detailed explanation:

1.  **Context**: The crash occurs when `fbcon` is rotated (e.g., 90 degrees clockwise) via `rotate_store`. This triggers `fbcon_modechanged()`.
2.  **Logic Flaw in `fbcon_modechanged`**:
    *   When the screen is rotated, the logical dimensions (rows and columns) of the console change. For example, if rotating a "portrait" screen (Height > Width) to 90 degrees, the number of text rows must decrease (since the new width is the old height, which is smaller).
    *   `fbcon_modechanged()` calculates the new dimensions (`cols` and `rows`) and calls `vc_resize(vc, cols, rows)` to resize the console buffer.
    *   **The bug:** `fbcon_modechanged()` **ignores the return value** of `vc_resize()`.
    *   If `vc_resize()` fails (e.g., due to a temporary memory allocation failure), the console (`vc`) remains with its *old* dimensions. In our example, `vc->vc_rows` remains large.
    *   Despite the failure, the function proceeds to call `update_screen(vc)`, which triggers a redraw.

3.  **The Crash in `cw_putcs`**:
    *   `update_screen()` calls `redraw_screen()`, which calls `do_update_region()`.
    *   `do_update_region()` iterates through the console buffer using the *current* `vc->vc_rows`. Since the resize failed, it uses the **old (large)** row count.
    *   It calls `fbcon_putcs()`, which delegates to `cw_putcs()` (because rotation is active).
    *   `cw_putcs()` calculates the destination X coordinate (`image.dx`) in the framebuffer:
        ```c
        image.dx = vxres - ((yy + 1) * vc->vc_font.height);
        ```
        *   `vxres` is the physical screen width (which is small in this scenario).
        *   `yy` is the row index passed from `do_update_region()`.
    *   Because `vc` was not resized, `yy` can be large (valid for the old dimensions but too large for the new rotated orientation).
    *   Eventually, `(yy + 1) * height` becomes larger than `vxres`.
    *   The subtraction `vxres - (large_value)` causes an **unsigned integer underflow**, resulting in a massive value for `image.dx`.

4.  **Memory Corruption**:
    *   `cw_putcs()` calls `cw_putcs_aligned()`, which calls `fb_imageblit()`.
    *   `fb_imageblit()` calculates the memory address to write to:
        ```c
        dst = base + image.dy * stride + image.dx * bpp;
        ```
    *   Since `image.dx` is huge due to the underflow, `dst` points far outside the allocated framebuffer (vmalloc area).
    *   `fb_bitmap_2ppw()` (called by `fb_imageblit`) attempts to write to this invalid address via `fb_write_offset()`, triggering the **KASAN: vmalloc-out-of-bounds** crash.

**Summary**: The kernel crashes because `fbcon` attempts to draw rows that don't fit on the rotated screen after a console resize failure. The fix is to check the return value of `vc_resize()` in `fbcon_modechanged()` and abort the screen update if it fails.

The diff of the bug fix is:

diff --git a/drivers/video/fbdev/core/fbcon.c b/drivers/video/fbdev/core/fbcon.c
index 7be9e8653..b9e418747 100644
--- a/drivers/video/fbdev/core/fbcon.c
+++ b/drivers/video/fbdev/core/fbcon.c
@@ -2712,7 +2712,8 @@ static void fbcon_modechanged(struct fb_info *info)
 		rows = FBCON_SWAP(par->rotate, info->var.yres, info->var.xres);
 		cols /= vc->vc_font.width;
 		rows /= vc->vc_font.height;
-		vc_resize(vc, cols, rows);
+		if (vc_resize(vc, cols, rows))
+			return;
 		updatescrollmode(p, info, vc);
 		scrollback_max = 0;
 		scrollback_current = 0;


Additional description of the patch:

The bug is a KASAN vmalloc-out-of-bounds write in `cw_putcs`, which is invoked during a screen update after a framebuffer mode change (specifically, rotation).

When the framebuffer mode changes (e.g., rotation), `fbcon_modechanged` calculates the new number of columns and rows required for the console and calls `vc_resize` to resize the console buffer. The original code failed to check the return value of `vc_resize`.

If `vc_resize` fails (e.g., due to memory allocation failure), the `vc` structure retains its old dimensions (e.g., a large number of rows for a portrait mode). However, the function proceeds to call `update_screen`, which triggers a redraw. The redraw logic (`cw_putcs` for clockwise rotation) uses the *old* (large) row count from `vc` but the *new* (small) screen width/height from `info`.

In `cw_putcs`, the destination offset is calculated as `vxres - ((yy + 1) * font_height)`. If `yy` (the current row) corresponds to a row index that fits in the old dimensions but exceeds the new rotated height, the subtraction results in an unsigned integer underflow (since `(yy + 1) * font_height` > `vxres`). This underflow produces a huge memory offset, causing the out-of-bounds write.

The fix is to verify the return value of `vc_resize` in `fbcon_modechanged`. If resizing fails, the function should return immediately to avoid calling `update_screen` with inconsistent dimensions, thus preventing the crash.

I applied the following change to `drivers/video/fbdev/core/fbcon.c`:

```c
<<<<
		cols /= vc->vc_font.width;
		rows /= vc->vc_font.height;
		vc_resize(vc, cols, rows);
		updatescrollmode(p, info, vc);
====
		cols /= vc->vc_font.width;
		rows /= vc->vc_font.height;
		if (vc_resize(vc, cols, rows))
			return;
		updatescrollmode(p, info, vc);
>>>>
```

fbcon: Fix OOB write in cw_putcs() if vc_resize() fails

Syzkaller reported a KASAN vmalloc-out-of-bounds write in
fb_write_offset(). The crash occurs during console rotation when
vc_resize() fails.

When the console is rotated, fbcon_modechanged() calls vc_resize() to
update the console dimensions. However, it ignores the return value. If
vc_resize() fails (e.g., due to memory allocation failure), the console
state (vc) retains the old dimensions, while the framebuffer info
reflects the new dimensions.

If the rotation effectively reduces the screen size (e.g., rotating from
portrait to landscape on a portrait screen), the old row count in `vc`
may exceed the new limits. Subsequently, update_screen() invokes
cw_putcs(), which calculates the destination column using:

  image.dx = vxres - ((yy + 1) * vc->vc_font.height);

Here, `vxres` is the new (smaller) width, but `yy` iterates up to the
old (larger) row count. This causes the subtraction to underflow,
producing a huge `image.dx` value. This invalid coordinate is passed to
fb_imageblit(), causing the out-of-bounds write.

Fix this by checking the return value of vc_resize() in
fbcon_modechanged(). If resizing fails, return immediately to prevent
inconsistent state and the subsequent crash.

gemini-3-pro-preview

							input: 6099
							output: 48
							thoughts: 2594
						

**Bug Summary:**

Okay, here's my thought process on crafting this commit message.  I've got the crash report, the root cause, the diff, and extra context. I need to make this clear and concise for the commit log.

First, the title:  `fbcon: Fix OOB write in cw_putcs() if vc_resize() fails`. That's clean, direct, and accurately highlights the core issue. It's concise enough to be under the 72-character limit. It focuses on `cw_putcs` and the failure of `vc_resize`, which are the critical pieces.

Now, the description.  I need to explain *why* this happens. Syzkaller reported a KASAN out-of-bounds write in `fb_write_offset`.  The fundamental problem is related to the console rotation code. When you rotate the console (`fbcon_rotate` triggers `fbcon_modechanged`),  `fbcon_modechanged` calculates the new console dimensions and calls `vc_resize()`. This is where it goes wrong.  `vc_resize` *can* fail. For instance, it could be a memory allocation issue. But the code *ignores* the failure.  Instead of checking the return code, it continues and attempts to update the screen. This is a crucial detail that needs to be communicated.

Because `vc_resize` fails, the console's internal state (`vc`) doesn't reflect the new dimensions, specifically the new number of rows and columns.  However, the framebuffer info *does* get updated with the new dimensions. So, you have an inconsistency between the expected and actual console size. This is a classic recipe for disaster.

When the system continues on to `update_screen`, the code calls `cw_putcs` (because of the clockwise rotation). `cw_putcs` is critical to our story because it's where the bad math happens.  `cw_putcs` calculates the target X coordinate for where to draw text using this equation: `image.dx = vxres - ((yy + 1) * vc->vc_font.height)`. The description needs to highlight the variables and what they represent.
- `vxres` is the *new* screen width.
- `yy` is the row number, which iterates according to the *old* (larger) row count from the failure.
- `vc->vc_font.height` is the font height.

The underflow occurs in the `image.dx` calculation.  If you rotate the display to a narrower width, the old `yy` value is large, and `vxres` is now small.  Because of this combination, subtracting the larger row count's y coordinate value from the smaller width results in an underflow. This generates a very large (and invalid) value for the X coordinate. This is the key.

This large, bad `image.dx` value is then fed to `fb_imageblit`, causing the kernel to try to write outside the bounds of the framebuffer, which triggers the KASAN report.

The fix is simple, but crucial: Check the return value of `vc_resize()` in `fbcon_modechanged()`. If it fails, `return` early. Don't proceed. The crash is easy to avoid if it bails out and does not try to update the framebuffer.

Here's the final commit message:

```
fbcon: Fix OOB write in cw_putcs() if vc_resize() fails

Syzkaller reported a KASAN vmalloc-out-of-bounds write in
fb_write_offset(). The crash occurs during console rotation when
vc_resize() fails.

When the console is rotated, fbcon_modechanged() calls vc_resize() to
update the console dimensions. However, it ignores the return value. If
vc_resize() fails (e.g., due to memory allocation failure), the console
state (vc) retains the old dimensions, while the framebuffer info
reflects the new dimensions.

If the rotation effectively reduces the screen size (e.g., rotating to a
narrower width), the old row count in `vc` may exceed the new limits.
Subsequently, update_screen() invokes cw_putcs(), which calculates the
destination column using:

  image.dx = vxres - ((yy + 1) * vc->vc_font.height);

Here, `vxres` is the new (smaller) width, but `yy` iterates up to the
old (larger) row count. This causes the subtraction to underflow,
producing a huge `image.dx` value. This invalid coordinate is passed to
fb_imageblit(), causing the out-of-bounds write.

Fix this by checking the

map[ContextFile:drivers/video/fbdev/core/fbcon_cw.c IncludeLines:true Name:cw_putcs]

map[SourceCode: 113:	static void cw_putcs(struct vc_data *vc, struct fb_info *info,
 114:			      const unsigned short *s, int count, int yy, int xx,
 115:			      int fg, int bg)
 116:	{
 117:		struct fb_image image;
 118:		struct fbcon_par *par = info->fbcon_par;
 119:		u32 width = (vc->vc_font.height + 7)/8;
 120:		u32 cellsize = width * vc->vc_font.width;
 121:		u32 maxcnt = info->pixmap.size/cellsize;
 122:		u32 scan_align = info->pixmap.scan_align - 1;
 123:		u32 buf_align = info->pixmap.buf_align - 1;
 124:		u32 cnt, pitch, size;
 125:		u32 attribute = get_attribute(info, scr_readw(s));
 126:		u8 *dst, *buf = NULL;
 127:		u32 vxres = GETVXRES(par->p, info);
 128:	
 129:		if (!par->fontbuffer)
 130:			return;
 131:	
 132:		image.fg_color = fg;
 133:		image.bg_color = bg;
 134:		image.dx = vxres - ((yy + 1) * vc->vc_font.height);
 135:		image.dy = xx * vc->vc_font.width;
 136:		image.width = vc->vc_font.height;
 137:		image.depth = 1;
 138:	
 139:		if (attribute) {
 140:			buf = kmalloc(cellsize, GFP_KERNEL);
 141:			if (!buf)
 142:				return;
 143:		}
 144:	
 145:		while (count) {
 146:			if (count > maxcnt)
 147:				cnt = maxcnt;
 148:			else
 149:				cnt = count;
 150:	
 151:			image.height = vc->vc_font.width * cnt;
 152:			pitch = ((image.width + 7) >> 3) + scan_align;
 153:			pitch &= ~scan_align;
 154:			size = pitch * image.height + buf_align;
 155:			size &= ~buf_align;
 156:			dst = fb_get_buffer_offset(info, &info->pixmap, size);
 157:			image.data = dst;
 158:			cw_putcs_aligned(vc, info, s, attribute, cnt, pitch,
 159:					 width, cellsize, &image, buf, dst);
 160:			image.dy += image.height;
 161:			count -= cnt;
 162:			s += cnt;
 163:		}
 164:	
 165:		/* buf is always NULL except when in monochrome mode, so in this case
 166:		   it's a gain to check buf against NULL even though kfree() handles
 167:		   NULL pointers just fine */
 168:		if (unlikely(buf))
 169:			kfree(buf);
 170:	
 171:	}
 SourceFile:drivers/video/fbdev/core/fbcon_cw.c]

gemini-3-pro-preview

							input: 10735
							output: 326
							thoughts: 2760
						

**fbcon: Analyzing a KASAN OOB Write During Console Rotation with Failed vc_resize()**

Okay, here's my understanding: Syzkaller flagged a KASAN out-of-bounds write within `fb_write_offset()`. This happens during console rotation when `vc_resize()` fails, which is crucial. When a rotation occurs, `fbcon_modechanged()` kicks off `vc_resize()` to adjust the console dimensions. However, there's a problem: the code doesn't check `vc_resize()`'s return value. If `vc_resize()` fails (like, maybe a memory allocation issue), the console's state (`vc`) keeps its old dimensions, but the framebuffer info reflects the *new* dimensions.

Now, here's where it gets interesting. If we have a rotation that *reduces* the apparent screen size (think rotating from portrait to a narrower landscape), the old row count stored in `vc` can exceed what's actually valid for the new dimensions.  `update_screen()` then calls `cw_putcs()`, which calculates the destination column. It uses the following line: `image.dx = vxres - ((yy + 1) * vc->vc_font.height);`.  `vxres` is the *new* smaller width, but `yy` is still iterating up to the *old*, larger row count. This causes an underflow, yielding a massive `image.dx` value. This invalid coordinate then gets fed to `fb_imageblit()`, resulting in the out-of-bounds write.

The fix is simple: check the return value of `vc_resize()` in `fbcon_modechanged()`. If resizing fails, we return immediately. This prevents the inconsistent state and the subsequent crash. The analysis is accurate. I'm satisfied.