Spring Boot + Azure SDK, extra characters at the end of the file while copying to Azure Storage account

Question

Some extra characters are added at the end of the file after uploading the file into storage account. And there is no issue with 1.33gb file, observed the size difference for 2.22gb file. Below is the code snippet and pom.xml details.

how to resolve it? let me know any details are needed.

Code:

private boolean upload(final MultipartFile file) throws IOException {           
BlobClientBuilder blobClientBuilder = new BlobClientBuilder();
blobClientBuilder.endpoint(STORAGE_URL).connectionString(storageConnectionString);          blobClientBuilder.containerName(CONTAINER_NAME);
BlobClient blobClient = blobClientBuilder.blobName(file.getOriginalFilename()).buildClient();
blobClient.upload(file.getInputStream(), file.getSize());
boolean uploadStatus = blobClient.exists();

pom.xml:

<parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.5.2</version>
        <relativePath /> <!-- lookup parent from repository -->
    </parent>
<dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-core</artifactId>
            <version>1.18.0</version>
        </dependency>
        <dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-storage-blob</artifactId>
            <version>12.12.0</version>
            <exclusions>
                <exclusion>
                    <groupId>io.projectreactor</groupId>
                    <artifactId>reactor-core</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <!-- https://mvnrepository.com/artifact/io.projectreactor/reactor-core -->
        <dependency>
            <groupId>io.projectreactor</groupId>
            <artifactId>reactor-core</artifactId>
            <version>3.4.8</version>
            <!--$NO-MVN-MAN-VER$ -->
            <!-- Please don't remove/degrade the version, possible for compatibility 
                issues -->
        </dependency>

        <!-- https://mvnrepository.com/artifact/io.projectreactor.netty/reactor-netty -->
        <dependency>
            <groupId>io.projectreactor.netty</groupId>
            <artifactId>reactor-netty</artifactId>
            <version>1.0.9</version>
            <!--$NO-MVN-MAN-VER$ -->
            <!-- Please don't remove/degrade the version, possible for compatibility 
                issues -->
        </dependency>

1.33gb file uploaded correctly but 2.22gb shows some extra characters which leads to increase the size of the file in bytes

Answer 1

Instead of uploading large file directly upload them in zip file or chucks

Try with this code

public static void uploadFilesByChunk() {
                String connString = "<conn str>";
                String containerName = "<container name>";
                String blobName = "UploadOne.zip";
                String filePath = "D:/temp/" + blobName;

                BlobServiceClient client = new BlobServiceClientBuilder().connectionString(connString).buildClient();
                BlobClient blobClient = client.getBlobContainerClient(containerName).getBlobClient(blobName);
                long blockSize = 2 * 1024 * 1024; //2MB
                ParallelTransferOptions parallelTransferOptions = new ParallelTransferOptions()
                                .setBlockSizeLong(blockSize).setMaxConcurrency(2)
                                .setProgressReceiver(new ProgressReceiver() {
                                        @Override
                                        public void reportProgress(long bytesTransferred) {
                                                System.out.println("uploaded:" + bytesTransferred);
                                        }
                                });

                BlobHttpHeaders headers = new BlobHttpHeaders().setContentLanguage("en-US").setContentType("binary");

                blobClient.uploadFromFile(filePath, parallelTransferOptions, headers, null, AccessTier.HOT,
                                new BlobRequestConditions(), Duration.ofMinutes(30));
        }

For more details refer this SO Thread

Answer 2

Thanks @ShrutiJoshi-MT for your code snippet.

I am not sure why it is working with 'uploadFromFile' method and having issue with 'upload' method of BlobClient. Below is the final code I am using, it is working for different file extensions. Anyone finds bug or having suggestions for below code please let me know, it helps me a lot.

First copying Multipartfile to local file and then providing the path.

public boolean uploadWithFile(final MultipartFile multipartFile) throws Exception {
        logger.info("uploadWithFile started");
        File file = null;
        try {
            String fileName = multipartFile.getOriginalFilename();
            file = new File(fileName);
            logger.info("uploadWithFile fileName: {}", fileName);
            Path path = Paths.get(fileName);
            logger.debug("Copying from MultipartFile to file");
            try (InputStream inputStream = multipartFile.getInputStream()) {
                Files.copy(inputStream, path, StandardCopyOption.REPLACE_EXISTING);
            }
            logger.debug("Copied from MultipartFile to file");
            String filePath = file.getPath();
            logger.debug("Copied file name: {}", file.getName());
            logger.debug("Copied file Path: {}", filePath);
            logger.debug("Copied file length: {}", file.length());
            
            String containerName = "temp";
            String storageConnectionString = "<primarykey> or <secondarykey>";
            BlobClientBuilder blobClientBuilder = new BlobClientBuilder();
            blobClientBuilder.endpoint(STORAGE_URL).connectionString(storageConnectionString);
            blobClientBuilder.containerName(containerName);
            BlobClient blobClient = blobClientBuilder.blobName(fileName).buildClient();
            logger.debug("uploading to storage account");
            blobClient.uploadFromFile(filePath);
            logger.debug("uploaded to storage account");
            boolean uploadStatus = blobClient.exists();
            logger.debug("uploaded status : {}", uploadStatus);
            logger.info("uploadWithFile ended");
            return uploadStatus;
        } catch (Exception exception) {
            logger.error("uploadWithFile upload failed: {}", exception);
            throw exception;
        } finally {
            if (Objects.nonNull(file) && file.exists()) {
                logger.debug("delete file: {}", file.getName());
                file.delete();
                logger.debug("deleted file: {}", file.getName());
            }
        }
    }```