perspective divide and clipping

Question

EDIT3:

Hi, The question might not be super clear, that is because I'm not sure I completely understand the problem myself. I think it seems to be "negative w" causing the issue. Other posts seem to point to a "negative w" requires clipping. Look at this for example: Clip matrix for 3D Perspective Projection

However, my problem is NOT similar, so I don't understand how to clip, or even what to clip, or what to do with these negative w values. It sais in that post that negative w occurs when a point is "behind the camera", but I don't have a camera, I only have a 3x3 transform matrix, and no concept of camera. (The 3x3 matrix btw is obtained by mapping 4 points into another 4 points, so there is no "camera" or notion of "depth" etc. It's like perspective transform in photoshop)

EDIT: Restated question. If you look at the first image in this original question, the problem might be restated as: "what coordinates of the image are at the corners of the axis-aligned bounding box?" The way I do that now, is that I transform each corner point with the inverse of the matrix. And the transform function is provided below (with w divide since it's a 3x3 matrix). This works most of the time, however, when the matrix transform is more extreme (however it doesnt really have to be that extreme, something "wraps over", and I'm not sure if it's a bug, or if this is just what to be expected and I need to do clipping

EDIT2: After further testing, it looks like when the "wrap around" happens, is at the same time as the "w" in the function becomes negative. This does seem to be something I read other places, and has something to do with some clipping needed, but I'm not sure how to do this...

Original question: I'll try to describe my problem (sorry if it's a bit long):

I make an image editor, which uses 3x3 matrix for its transformations, and it supports using the entire 3x3 matrix so that it can also do simple perspective transformations

(BTW: This is in javascript and I'm using gl-matrix for matrix math)

I'm also using this code to transform a 2d point with a 3x3 matrix (note the division by w)

vec2.transformMat3Div = function(out, a, m)
{
    const x = a[0],
      y = a[1];

    const w = (m[2]*x + m[5]*y + m[8]) || 1;

    out[0] = (m[0]*x + m[3]*y + m[6]) / w;
    out[1] = (m[1]*x + m[4]*y + m[7]) / w;
    return out;
};

I sometimes need to find "rectangle of the image that I need to cover an area", what I mean, is, for example in the image above. If I were to cover the axis-aligned rectangle, what are the image bbox that I need to fill the axis-aligned outer blue box? In this case, there will be negative values and also values that go outside the image (and that's fine).

What I do to accomplish this, is I take the inverse of the matrix, and transform the axis-aligned bbox with the inverse matrix and find a new bbox. Basically something like:

        const imatrix3 = matrix3.invert();
        const newBBox = bbox.transform(bbox, imatrix3);//min-max of transforming the points in bbox

While this works most of the time, there are cases where it doesn't.

See how the image has been cut at the bottom because the above calculation (The bbox that I've calculated for "how much of the image I need" is wrong).

Ok.. in the following example, perhaps this will clarify (or not) I've checked a button that will use the inverse of the matrix

In the picture below, I'm only slightly moving one of the points, so that the image extends even more down and to the right, but in my calculating the bounding box of that, my calculations "wrap around" see how the outer bbox (it's the light blue lines, a bit hard to see) now instead of going more bottom-right, has wrapped around and goes top-left. In a way, it makes sense, because the image goes more and more towards a larger number, HOWEver, we're not talking about that large numbers. If I had shown how far the image extends we're only talking about 30000 or so, hardly a number that is high enough to cause double to overflow...

I had a thought, and it's that there's a clipping issue. Just like when using 4x4 matrices, you need to clip before doing divide by w, maybe I need to do some clipping before dividing by w in the above javascript code? I don't actually know how to do that, And I'm not sure I understand, because it's not the drawing that goes wrong, its me calculating a bbox of the transformed points, I'm not sure what it is that I should be clipping and against what. I also don't feel this is true 3d-space I'm working in (no 4x4 matrices, no 3d-coordinates, just 2d-coordinates multipled by 3x3 matrix)

Answer 1

It looks like you are dealing with issues related to perspective transformations using a 3x3 matrix and encountering problems when calculating bounding boxes after applying the inverse matrix. The problem is most likely related to points having negative w values, causing unexpected behavior.

When dealing with 3x3 matrices in computer graphics, especially in a 2D context with perspective transformations, it's common to encounter issues with the perspective divide and negative w values. Negative w values can occur when a point is behind the viewer or in a space not well-handled by simple perspective transformations.

Here are a few suggestions to address the issues you're facing:

1. Handling Negative w Values: When you perform the perspective transformation, check for negative w values. If w is negative, it means the point is behind the viewer, and you may need to handle it differently. You might choose to skip such points or apply additional transformations to bring them into the visible space.

2. Clipping: Clipping involves removing parts of the scene that fall outside the view frustum. In your case, it may involve clipping points that result in negative w values or points that are too far away. Implementing a simple clipping algorithm can help ensure that points are within a reasonable range.

3. Bounding Box Calculation: When calculating the bounding box after applying the inverse matrix, consider excluding points with negative w values or points that have been clipped. This may prevent the bounding box from wrapping around unexpectedly.

4. Normalization: Normalize your transformed points before calculating the bounding box. This involves dividing the x and y coordinates by the homogeneous coordinate (w). Normalizing ensures that the coordinates are within a reasonable range and may help with unexpected behavior.

Here's a modified version of your vec2.transformMat3Div function to include handling of negative w values and normalization:

Javascript:

vec2.transformMat3Div = function(out, a, m) {
const x = a[0];
const y = a[1];

const w = (m[2] * x + m[5] * y + m[8]) || 1;

if (w < 0) {
// Handle negative w value (skip or apply additional transformations)
// You may also consider clipping or excluding such points
    return out;
}
out[0] = (m[0] * x + m[3] * y + m[6]) / w;
out[1] = (m[1] * x + m[4] * y + m[7]) / w;
// Normalize the coordinates
out[0] /= w;
out[1] /= w;
return out;
};

Remember that handling perspective transformations and negative w values can be complex, and the specific solution may depend on the requirements of your application. Additionally, testing and visualizing intermediate results can help identify the root cause of unexpected behavior.