randr: do full transform when checking SetScreenSize size

When validating the size passed to SetScreenSize, the CRTC mode size
needs to be applied the full CRTC transform, otherwise the check may
bogusly fail.

Do a full transform on the CRTC mode size when checking the
SetScreenSize request size instead of the current code that only
manually handles rotation.

Signed-off-by: Icenowy Zheng <uwu@icenowy.me>
Part-of: <https://gitlab.freedesktop.org/xorg/xserver/-/merge_requests/1971>
This commit is contained in:
Icenowy Zheng
2026-03-14 20:44:04 +02:00
committed by Enrico Weigelt
parent f0a679cfb2
commit e2550d065b

View File

@@ -271,17 +271,14 @@ ProcRRSetScreenSize(ClientPtr client)
RRModePtr mode = crtc->mode;
if (!RRCrtcIsLeased(crtc) && mode) {
int source_width = mode->mode.width;
int source_height = mode->mode.height;
Rotation rotation = crtc->rotation;
struct pixman_box16 display_box = {
crtc->x, crtc->y,
crtc->x + mode->mode.width,
crtc->y + mode->mode.height
};
pixman_f_transform_bounds(&crtc->f_transform, &display_box);
if (rotation & (RR_Rotate_90 | RR_Rotate_270)) {
source_width = mode->mode.height;
source_height = mode->mode.width;
}
if (crtc->x + source_width > stuff->width ||
crtc->y + source_height > stuff->height)
if (display_box.x2 > stuff->width || display_box.y2 > stuff->height)
return BadMatch;
}
}