Intersect TriangleMesh from screen coordinates in JavaFX

Question

I have a TriangleMesh in javafx and a 2D point in screen space and i want to check whether any triangles in the mesh intersect with that point(basically same as clicking a point in the mesh except that i already have a predefined screen coordinate)

Note that i don't care where the intersection happened etc but only the fact whether it intersected or not.

I did try googling this and found some useful answers, especially one by José Pereda here: https://stackoverflow.com/a/27612786/14999427

However the links are down

Edit: i did find a working link to it, however it just hardcodes the origin/target and im not sure how to compute these from the 2d screen point

Another idea i had was to copy the implementation from openjfx but after some research i figured that i'd have to copy a lot of internals and even then i wasn't sure if i would get it working so i scrapped that idea.

Goal: i want to use my mouse to create a rectangle at an arbitrary position in the mesh and then find all triangles that are inside that rectangle (i believe it's usually called rectangle selection in 3d modeling software)

Update: converting each triangle to 2d screen coordinates using Node#localToScreen and then doing a 2D point inside triangle test works perfectly however it also selects faces that are culled.

Update 2: after also doing the culling myself it works quite well (it's not perfect but it's mostly accurate) Current code:

Point2D v1Screen = view.localToScreen(mesh.getPoints().get(0), mesh.getPoints()
        .get(1), mesh.getPoints().get(2));
Point2D v2Screen = view.localToScreen(mesh.getPoints().get(3), mesh.getPoints()
        .get(4), mesh.getPoints().get(5));
Point2D v3Screen = view.localToScreen(mesh.getPoints().get(6), mesh.getPoints()
        .get(7), mesh.getPoints().get(8));
Point2D point = new Point2D(mouseX, mouseY);
boolean inTriangle = pointInTriangle(point, v1Screen, v2Screen, v3Screen);
// Back-face culling
double dxAB = v1Screen.getX() - v2Screen.getX();
double dyAB = v1Screen.getY() - v2Screen.getY();
double dxCB = v3Screen.getX() - v2Screen.getX();
double dyCB = v3Screen.getY() - v2Screen.getY();
boolean culled = ((dxAB * dyCB) - (dyAB * dxCB)) <= 0;
if (inTriangle && !culled) {
    view.setMaterial(new PhongMaterial(Color.BLUE));
    System.out.println("Intersected");
}

Answer 1

I think the answer to this question may help you. How to correctly obtain screen coordinates of 3D shape after rotation If you can compute the screen coordinates of a 3D shape (triangle) the hit test should then be simple.