java.io.FileNotFoundException (Read-only file system) // Uploading files to S3

Question

I am trying to create csv files from a list of maps and uploading them to S3 bucket through a lambda function. Following is the code:

public void createCSV(List<Map<String, AttributeValue>> changedRecords, Context context, String tableName)
            throws IOException {
        Calendar calendar = Calendar.getInstance();
        SimpleDateFormat formatter = new SimpleDateFormat("yyyyMMddHHmmss");
        String outputName = tableName + "_" + formatter.format(calendar.getTime()) + ".csv";

        List<String> headers = changedRecords.stream().flatMap(map -> map.keySet().stream()).distinct()
                .collect(Collectors.toList());

        try (FileWriter writer = new FileWriter(outputName, true);) {
            for (String string : headers) {
                writer.write(string);
                writer.write(",");
            }
            writer.write("\r\n");
            for (Map<String, AttributeValue> lmap : changedRecords) {
                for (Entry<String, AttributeValue> string2 : lmap.entrySet()) {
                    writer.write(string2.getValue().getS());
                    writer.write(",");
                }
                writer.write("\r\n");
            }
        }
        catch (Exception e) {
            e.printStackTrace();
        }
        s3.putObject(new PutObjectRequest("bucket_name", "data/" + outputName, outputName));
    }

Getting the following fileNotFound exception:

java.io.FileNotFoundException: data_20200227192207.csv (Read-only file system) at java.io.FileOutputStream.open0(Native Method) at java.io.FileOutputStream.open(FileOutputStream.java:270) at java.io.FileOutputStream.(FileOutputStream.java:213) at java.io.FileOutputStream.(FileOutputStream.java:133) at java.io.FileWriter.(FileWriter.java:78) at com.amazonaws.lambda.demo.PLMLambda.createCSV(PLMLambda.java:84) at com.amazonaws.lambda.demo.PLMLambda.handleRequest(PLMLambda.java:54) at com.amazonaws.lambda.demo.PLMLambda.handleRequest(PLMLambda.java:1) at lambdainternal.EventHandlerLoader$PojoHandlerAsStreamHandler.handleRequest(EventHandlerLoader.java:178) at lambdainternal.EventHandlerLoader$2.call(EventHandlerLoader.java:906) at lambdainternal.AWSLambda.startRuntime(AWSLambda.java:341) at lambdainternal.AWSLambda.(AWSLambda.java:63) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at lambdainternal.LambdaRTEntry.main(LambdaRTEntry.java:114)

Answer 1

If your CSV content is not so huge (like 4GB), you can simple use StringWriter instead of FileWriter with AWS Lambda, and then just put directly the string to S3.

Answer 2

Change the line:

try (FileWriter writer = new FileWriter(outputName, true);) {

to

try (FileWriter writer = new FileWriter("/tmp" + outputName, true);) {

In Lambda you can only write to the /tmp directory.