UTF-8 encoding for output from Console to JavaFX TextArea

Question

I want to redirect the output in Console to JavaFX TextArea, and I follow a suggestion here: JavaFX: Redirect console output to TextArea that is created in SceneBuilder

I tried to set charset to UTF-8 in PrintStream(), but it does not look so well. Setting the charset to UTF-16 improves it a bit, but it is still illegible.

In Eclipse IDE, the supposed text output in Console turns out fine:

KHA khởi đầu phiên giao dịch sáng nay ở mức 23600 điểm, khối lượng giao dịch trong ngày đạt 765 cổ phiếu, tương đương khoảng 18054000 đồng.

Controller.java

public class Controller {
    @FXML
    private Button button;

    public Button getButton() {
        return button;
    }

    @FXML
    private TextArea textArea;

    public TextArea getTextArea() {
        return textArea;
    }

    private PrintStream printStream;

    public PrintStream getPrintStream() {
        return printStream;
    }

    public void initialize() {
        textArea.setWrapText(true);
        printStream = new PrintStream(new UITextOutput(textArea), true, StandardCharsets.UTF_8);
    } // Encoding set to UTF-8

    public class UITextOutput extends OutputStream {
        private TextArea text;

        public UITextOutput(TextArea text) {
            this.text = text;
        }

        public void appendText(String valueOf) {
            Platform.runLater(() -> text.appendText(valueOf));
        }

        public void write(int b) throws IOException {
            appendText(String.valueOf((char) b));
        }
    }
}

UI.java

public class UI extends Application {
    @Override
    public void start(Stage stage) {
        try {
            FXMLLoader loader = new FXMLLoader(getClass().getResource("Sample.fxml"));
            Parent root = loader.load();
            Controller control = loader.getController();

            stage.setTitle("Title");
            stage.setScene(new Scene(root));
            stage.show();

            control.getButton().setOnAction(new EventHandler<ActionEvent>() {
                public void handle(ActionEvent event) {
                    try {
                        System.setOut(control.getPrintStream());
                        System.setErr(control.getPrintStream());
                        System.out.println(
                                "KHA khởi đầu phiên giao dịch sáng nay ở mức 23600 điểm, khối lượng giao dịch trong ngày đạt 765 cổ phiếu, tương đương khoảng 18054000 đồng.");
                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                }
            });
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public static void main(String[] args) {
        launch(args);
    }
}

Sample.fxml

<?xml version="1.0" encoding="UTF-8"?>

<?import javafx.scene.control.Button?>
<?import javafx.scene.control.TextArea?>
<?import javafx.scene.layout.BorderPane?>


<BorderPane prefHeight="339.0" prefWidth="468.0" xmlns:fx="http://javafx.com/fxml/1" xmlns="http://javafx.com/javafx/11.0.1" fx:controller="main.Controller">
   <center>
      <TextArea fx:id="textArea" prefHeight="200.0" prefWidth="200.0" BorderPane.alignment="CENTER" />
   </center>
   <right>
      <Button fx:id="button" mnemonicParsing="false" onAction="#getButton" text="Button" BorderPane.alignment="CENTER" />
   </right>
</BorderPane>

I'm still new to Java so I'm unfamiliar to how exactly PrintStream or OutputStream works. Please excuse my ignorance.

Every suggestion is appreciated.

Answer 1

I believe your problem is caused by this code:

public void write(int b) throws IOException {
    appendText(String.valueOf((char) b));
}

This is converting each individual byte into a character. In other words, it's assuming each character is represented by a single byte. That's not necessarily true. Some encodings, such as UTF-8, may use multiple bytes to represent a single character. They have to if they want to be able to represent more than 256 characters.

You'll need to properly decode the incoming bytes. Rather than trying to do this yourself it would be better to find a way to use something like BufferedReader. Luckily that's possible with PipedInputStream and PipedOutputStream. For example:

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.PipedInputStream;
import java.io.PipedOutputStream;
import java.io.PrintStream;
import java.io.UncheckedIOException;
import javafx.application.Application;
import javafx.application.Platform;
import javafx.scene.Scene;
import javafx.scene.control.TextArea;
import javafx.stage.Stage;

import static java.nio.charset.StandardCharsets.UTF_8;

public class Main extends Application {

  @Override
  public void start(Stage primaryStage) {
    TextArea area = new TextArea();
    area.setWrapText(true);

    redirectStandardOut(area);

    primaryStage.setScene(new Scene(area, 800, 600));
    primaryStage.show();

    System.out.println(
        "KHA khởi đầu phiên giao dịch sáng nay ở mức 23600 điểm, khối lượng giao dịch trong ngày đạt 765 cổ phiếu, tương đương khoảng 18054000 đồng.");
  }

  private void redirectStandardOut(TextArea area) {
    try {
      PipedInputStream in = new PipedInputStream();
      System.setOut(new PrintStream(new PipedOutputStream(in), true, UTF_8));

      Thread thread = new Thread(new StreamReader(in, area));
      thread.setDaemon(true);
      thread.start();
    } catch (IOException ex) {
      throw new UncheckedIOException(ex);
    }
  }

  private static class StreamReader implements Runnable {

    private final StringBuilder buffer = new StringBuilder();
    private boolean notify = true;

    private final BufferedReader reader;
    private final TextArea textArea;

    StreamReader(InputStream input, TextArea textArea) {
      this.reader = new BufferedReader(new InputStreamReader(input, UTF_8));
      this.textArea = textArea;
    }

    @Override
    public void run() {
      try (reader) {
        int charAsInt;
        while ((charAsInt = reader.read()) != -1) {
          synchronized (buffer) {
            buffer.append((char) charAsInt);
            if (notify) {
              notify = false;
              Platform.runLater(this::appendTextToTextArea);
            }
          }
        }
      } catch (IOException ex) {
        throw new UncheckedIOException(ex);
      }
    }

    private void appendTextToTextArea() {
      synchronized (buffer) {
        textArea.appendText(buffer.toString());
        buffer.delete(0, buffer.length());
        notify = true;
      }
    }
  }
}

The use of buffer above is an attempt to avoid flooding the JavaFX Application Thread with tasks.

Some other things you need to take into consideration:

• Since you're using a string literal, make sure you're both saving the source file with UTF-8 and compiling the code with -encoding UTF-8.
• Make sure the font you use with the TextArea can represent all the characters you want it to.
• It's possible you also need to run the application with -Dfile.encoding=UTF-8 but I'm not sure. I did not and it still worked for me.

Answer 2

Try to set your default JVM encoding to UTF-8.

java -Dfile.encoding=UTF-8 -jar YourJarfile.jar

For more details look at this thread: Setting the default Java character encoding

If you don't want to export your file, go into your Eclipse Preferences > General > Workspace and set the Text file encoding to UTF-8 (or the encoding you'd like to have).

There are a few more details: How to change default text file encoding in Eclipse

Answer 3

To add a bit more from _ the above answer using PipedInputStream _

Here is how you can use with a TeeOutputStream to output to both the Console & TextArea::

• vv

  • (the code should work, at least in my case)

  • (I dont think too many synchronized is needed, since the JavaFx Thread is a Single Thread?)

• Update (misc)

  • //debug <strike> sysout_ori.println(str); this is removed

  • The code before relies on only 1 Thread TR -- the Thread that is Reading from the BufferReader
    -- to do the TextArea update
    -> this is problematic, cuz TR can block on reader.read()
    & the batch of bytes remained in StringBuffer will not be updated to TextArea until next .read()
    to fix that, a new Thread Executors.newSingleThreadScheduledExecutor() is introduced & thats why synchronized (sb) { is required now.

  // ############

  private static final int max_TextAllowed_InTextArea_ToAvoidLag = 8000;
  private static final int amount_TextPreserve_InTextArea_WhenMaxTextReached = 3000;

  private static void setup_OutputStream(TextArea textArea_info) {
    PipedInputStream pipe_in = new PipedInputStream();
    PipedOutputStream pipe_out = null;
    try {
      pipe_out = new PipedOutputStream(pipe_in);
    } catch (IOException e) {
      e.printStackTrace();
    }
    final PrintStream sysout_ori = System.out;
    TeeOutputStream teeOutputStream = new TeeOutputStream(sysout_ori, pipe_out);
    System.setOut(new PrintStream(teeOutputStream, true, StandardCharsets.UTF_8));
    System.setErr(new PrintStream(teeOutputStream, true, StandardCharsets.UTF_8));

    Thread thread_OutputStreamRedirectToJavafx = new Thread()
      {
        private final StringBuffer sb = new StringBuffer();

        private final BufferedReader reader = new BufferedReader(new InputStreamReader(pipe_in, StandardCharsets.UTF_8));

        private final ScheduledExecutorService executor_ExecIdleBatch = Executors.newSingleThreadScheduledExecutor();

        {
          executor_ExecIdleBatch.scheduleWithFixedDelay(
                                                        () -> {
                                                          if (sb.length() != 0) {
                                                            final String str;
                                                            synchronized (sb) {
                                                              str = sb.toString();
                                                              sb.setLength(0);
                                                            }
                                                            Platform.runLater(() -> {
                                                              String content_existing = textArea_info.getText();
                                                              int length = content_existing.length();
                                                              if (length > max_TextAllowed_InTextArea_ToAvoidLag) {
                                                                textArea_info.setText((content_existing + str).substring(length - amount_TextPreserve_InTextArea_WhenMaxTextReached));
                                                              }
                                                              else {
                                                                textArea_info.appendText(str);
                                                              }
                                                            });
                                                          }

                                                        }, 0, 200, TimeUnit.MILLISECONDS);
        }

        @Override
        public void run() {
          int b;
          while (true) {
            try {
              b = reader.read();
            } catch (IOException e) {
              throw new Error(e);
            }
            if (b == -1) {
              executor_ExecIdleBatch.shutdown();
              break;
            }
            else {
              synchronized (sb) {
                sb.append((char) b);
              }
            }
          }
        }

      };
    thread_OutputStreamRedirectToJavafx.start();

  }

---

---

Update (misc) [Pipe broken]

• An error java.io.IOException: Pipe broken / java.io.IOException: Write end dead happened in my case.
  
  Due to the use of
  1. event.consume(); // consume the WindowEvent.WINDOW_CLOSE_REQUEST to prevent close (so that have time to properly handle the shutdown first) +
  2. stage.close() // close the stage when shutdown handle is completed
  inside window.addEventFilter(WindowEvent.WINDOW_CLOSE_REQUEST, (event) -> { for Shutdown handling
  
  it was fixed by not using stage.close(), but re-dispatching the consumed event by window.fireEvent(new WindowEvent(window, WindowEvent.WINDOW_CLOSE_REQUEST));
  
  Idk why, but guessing: the Javafx Thread is writing to the System.out -> the PipedOutputStream is linked to the Javafx Thread? --> the Javafx Thread is dead due to stage.close()? --> Write end dead + Pipe broken?