Java: Asynchronous concurrent writes to disk

Question

I have a function in my Main thread which will write some data to disk. I don't want my Main thread to stuck (High Latency of Disk I/O) and creating a new thread just to write is an overkill. I have decided to use ExecutorService.

ExecutorService executorService = Executors.newFixedThreadPool(3);

   Future future = executorService.submit(new Callable<Boolean>() {
    public Boolean call() throws Exception {
      logger.log(Level.INFO, "Writing data to disk");
      return writeToDisk();
    }
  });

writeToDisk is the function which will write to disk

Is it a nice way to do? Could somebody refer better approach if any?

UPDATE: Data size will be greater than 100 MB. Disk bandwidth is 40 MBps, so the write operation could take couple of seconds. I don't want calling function to stuck as It has to do other jobs, So, I am looking for a way to schedule Disk I/O asynchronous to the execution of the calling thread.

I need to delegate the task and forget about it!

Answer 1

There are two approaches that I am aware of, spin up a thread (or use a pool of threads as you have) or memory map a file and let the OS manage it. Both are good approaches, memory mapping a file can be as much as 10x faster than using Java writers/streams and it does not require a separate thread so I often bias towards that when performance is key.

Either way, as a few tips to optimize disk writing try to preallocate the file where possible. Resizing a file is expensive. Spinning disks do not like seeking, and SSDs do not like mutating data that has been previously written.

I wrote some benchmarks to help me explore this area awhile back, feel free to run the benchmarks yourself. Amongst them is an example of memory mapping a file.

Answer 2

Your code looks good anyways I've used AsynchronousFileChannel from new non-blocking IO. The implementation uses MappedByteBuffer through FileChannel. It might give you the performance which @Chris stated. Below is the simple example:

public static void main(String[] args) {
    String filePath = "D:\\tmp\\async_file_write.txt";
    Path file = Paths.get(filePath);
    try(AsynchronousFileChannel asyncFile = AsynchronousFileChannel.open(file,
                        StandardOpenOption.WRITE,
                        StandardOpenOption.CREATE)) {

        asyncFile.write(ByteBuffer.wrap("Some text to be written".getBytes()), 0);
    } catch (IOException e) {
        e.printStackTrace();
    }
}

Answer 3

I would agree with Õzbek to use a non-blocking approach. Yet, as pointed by Dean Hiller, we cannot close the AsynchronousFileChannel before-hand using a try with resources because it may close the channel before the write has completed.

Thus, I would add a CompletionHandler on asyncFile.write(…,new CompletionHandler< >(){…} to track completion and close the underlying AsynchronousFileChannel after conclusion of write operation. To simplify the use I turned the CompletionHandler into a CompletableFuture which we can easily chain with a continuation to do whatever we want after write completion. The final auxiliary method is CompletableFuture<Integer> write(ByteBuffer bytes) which returns a CompletableFuture of the final file index after the completion of the corresponding write operation. I placed all this logic in an auxiliary class AsyncFiles that can be used like this:

Path path = Paths.get("output.txt")
AsyncFiles
    .write(path, bytes)
    .thenAccept(index -> /* called on completion from a background thread */)