MacOS BigSur auto-scales Java16/Swing app window for 5K Retina display

Question

Our interactive math-ed app is designed using Swing JFrames, and was developed for 2560p (Mac) and 1920p (Win) displays, i.e., using native pixel-coordinates for graphics output and mouse-input-processing.

iMac Retina 5K testing -- BIG SURPRISE! The app window's normal width is 1/2 screen width. I was fully expecting a problem when running on a hi-rez (5120p) iMac -- 1/4 screen-width. But NO, it renders 1/2 screen wide.

Looking into where the program polls the system for display device bounds:

    GraphicsDevice[] deviceList = getLocalGraphicsEnvironment.getScreenDevices()

    GraphicsConfiguration[] gc = deviceList[0].getConfigurations();

    Rectangle deviceBounds = gc[0].getBounds(); 

I see that it comes back as [ 0, 0, 2560, 1440 ] even though the MacOS System Report says [5120 x 2880 ].

Futhermore, the MouseEvent coordinates fed as input to the app are "halved" so as to agree with the deviceBounds.

I can't find any explanation for these test results. Needless to say, I'm elated that the app runs perfectly on the 5K Retina iMac without modifications.

However, I really need to understand who is auto-scaling-down the native graphics environment. Is this something the MacOS does? Something Java16 does for Swing apps? Or, something the Java16 MacOS JRE does for Swing apps? 4th possibility?

Answer 1

tl;dr

You asked:

I really need to understand who is auto-scaling-down the native graphics environment.

• The macOS scales the screen, providing a logical versus physical screen resolution to apps.
• Java 2D, Java AWT, and Java Swing respect and leverage that scaling.

Details

Caveat: I am not an expert in how Swing graphics work, but I may be able to give you a basic picture (pun intended!) of what is going on.

---

To simplify the story, you can think of there being logical pixels versus physical pixels.

In the old days, such as the original Mac, 72 pixels were distributed across one inch of actual screen space. This number of 72 PPI was chosen because it nearly aligns with the typographical unit of a point, with approximately 72 points to the inch. (Actually 1 point is equal to 0.013836 inch, so 72 points are 0.996264 inch.)

Over the decades, screens grew larger, and pixel density increased with more and more smaller pixels crammed together. Eventually we had an embarrassment of riches, enough pixels that we could “waste” them on finer detailing, filling in the jaggies for smoother fonts and lines. Apple refers to their implementation of this as Retina screens; other OSes have similar schemes (of debatable quality). Nowadays, displays have from over 200 to almost 500 pixels per inch.

To set aside the extra physical pixels for more smoothing, we need logical pixels, imaginary pixels, that the apps think they are painting. Apple used to use 72 pixels per inch (PPI) for this, for compatibility with older software unaware of the extra smoothing issues. That may still be the case, or perhaps nowadays the PPI is variable (I don't know). But the exact number is beside the point. The point is the logical pixels-per-inch is much less than the physical.

➥ And germane to your Question, modern Java Swing graphics technology respects those logical pixels while allowing for extra smoothing by the extra physical pixels.

As I vaguely recall, this may not have been the case in the early transition period, where the extra pixels were wasted, not used for smoothing/detailing. So, for a while, back then, a Swing app would have appeared "chunky"/"jaggy" while on-screen next to other apps that were Retina-savvy. Eventually Java 2D & Java Swing were enhanced to be Retina-savvy. And, voilà, modern Swing apps look great on high-res screens.

(By the way, in Java 17 the Java graphics technology is being reworked to integrate with Apple’s Metal framework rather than OpenGL, for faster and richer graphics.)

The ratio of how many extra physical pixels are reserved for extra smoothing is controlled by the user. On macOS you control a setting in System Preferences > Display > Resolution. There you determine the logical pixel count.

Here is a screenshot of my own system, a MacBook Pro (13-inch, M1, 2020) with an external BenQ 4K monitor attached (which I highly recommend by the way, certified as truly flicker-free).

Notice how the monitor is physically built with 3,840 pixels across horizontally and 2,160 pixels high vertically. This 3840 × 2160 is commonly known as 4K display.

But I have Retina scaling engaged. So the OS and apps act as if I had only 3,008 by 1,692 pixels, the logical pixel count. I engage this smoothing so that (a) content appears larger, and more importantly (b) content appears smoother with greater detail filled-in.

In the upper-left corner is a white app window from a custom Java program I wrote. This app reports the screens as seen by Java Swing. The green call-out lines are for my internal monitor, while the orange lines are for my external 4K monitor discussed above.

➥ Notice how Java Swing reports the same logical pixel resolution as we set in scaling. So if you size your Java Swing JFrame window to be half the width of the screen, you would in this case be sizing to half of 3008 rather than half of 3840.

You can experiment with this app. Change the scaling factor of your Mac, then click the Update screen info button to see fresh numbers.

➥ Given that you seem to be surprised by this behavior, I suspect you had disengaged the Retina scaling on your previous Mac. Without scaling engaged, your software was using “native” physical pixels rather than logical pixels. You could say that as the physical pixels equalled the virtual pixels, mapped one-to-one.

Here is the source-code for that app.

package work.basil.screen;

import javax.swing.*;
import java.awt.*;
import java.awt.event.ActionEvent;
import java.util.ArrayList;
import java.util.List;

// Example app to show current logical screen resolutions.
// By Basil Bourque. http://www.Basil.work/
public class Screener extends JFrame
{
    public static void main ( String[] args )
    {
        Screener frame = new Screener();
        frame.setVisible( true );
    }

    // Constructor
    public Screener ()
    {
        // Layout
        this.setDefaultCloseOperation( JFrame.DISPOSE_ON_CLOSE );
        this.setPreferredSize( new Dimension( 600 , 200 ) );
        this.setLayout( new BorderLayout() );

        // Widgets
        JTextArea report = new JTextArea( "report goes here" );
        report.setLineWrap( true );
        report.setWrapStyleWord( true );

        JButton update = new JButton( "Update screen info" );
        update.addActionListener( ( ActionEvent e ) -> report.setText( reportScreenInfo() ) );

        // Arrange
        this.add( report , BorderLayout.CENTER );
        this.add( update , BorderLayout.PAGE_END );
        this.pack();
    }

    private String reportScreenInfo ()
    {
        List < String > reportItems = new ArrayList <>();

        GraphicsDevice[] deviceList = GraphicsEnvironment.getLocalGraphicsEnvironment().getScreenDevices();
        for ( GraphicsDevice graphicsDevice : deviceList )
        {
            reportItems.add( "graphicsDevice ID: " + graphicsDevice.getIDstring() );
            GraphicsConfiguration[] graphicsConfigurations = graphicsDevice.getConfigurations();
            for ( GraphicsConfiguration gc : graphicsConfigurations )
            {
                reportItems.add( "bounds: " + gc.getBounds() );
            }
        }

        String result = String.join( System.lineSeparator() , reportItems );
        return result;
    }
}