Azure blob file download failed after downloading a few files

Question

I tried to download blob files using BlobClient.downloadToFile(dest), but it failed after downloading a few files.

package com.blob.download;

import com.azure.storage.blob.BlobContainerClient;
import com.azure.storage.blob.BlobServiceClient;
import com.azure.storage.blob.BlobServiceClientBuilder;
import com.azure.storage.blob.models.BlobItem;
import com.azure.storage.blob.models.ListBlobsOptions;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Iterator;

public class BlobDownload {
    public static void main(String args[]){
        BlobServiceClientBuilder builder = new BlobServiceClientBuilder();
        //Specify storage uri
        String storageBaseUri = "";
        ///Specify connection string
        String connectionString = "";
        //Specify key
        String key = "";

        builder.connectionString(connectionString);
        BlobServiceClient client = builder.buildClient();

        if (!storageBaseUri.startsWith(client.getAccountUrl())) {
            throw new IllegalArgumentException(
                    "The given credential can not be used for the specified container.");
        }
        String containerName = storageBaseUri.replace(client.getAccountUrl() + "/", "");
        BlobContainerClient blobContainerClient = client.getBlobContainerClient(containerName);

        System.out.println("Downloading " + blobContainerClient.getBlobContainerName() + "/" + key);
        Iterator<BlobItem> keyspaceBlobs =
                blobContainerClient.listBlobs(new ListBlobsOptions().setPrefix(key + "/"), null).iterator();

        while (keyspaceBlobs.hasNext()){
            BlobItem blob = keyspaceBlobs.next();

            Path destFile = Paths.get("/opt/data/", blob.getName());


            try {
                Files.createDirectories(destFile.getParent());
            } catch (IOException e) {
                System.out.println(e);
            }

            blobContainerClient
                    .getBlobClient(blob.getName())
                    .downloadToFile(destFile.toString());
            System.out.println("Download file succeeded : " + destFile.toString());
        }

    }
}

We can simply reproduce this issue in the local environment while trying to download a 3GB file from blob storage.

package com.blob.download;

import com.azure.storage.blob.BlobContainerClient;
import com.azure.storage.blob.BlobServiceClient;
import com.azure.storage.blob.BlobServiceClientBuilder;


public class BlobDownload {
    public static void main(String args[]){
        BlobServiceClientBuilder builder = new BlobServiceClientBuilder();

        // specify storage uri
        String storageBaseUri = "";
        //Specify connection string
        String connectionString = "";
        //Specify 3 GB blob file path inazure
        String blobPath=""
        //Specify destination path
        String dest="/opt/data/md-388-big-Index.db"

        builder.connectionString(connectionString);
        BlobServiceClient client = builder.buildClient();

        if (!storageBaseUri.startsWith(client.getAccountUrl())) {
            throw new IllegalArgumentException(
                    "The given credential can not be used for the specified container.");
        }
        String containerName = storageBaseUri.replace(client.getAccountUrl() + "/", "");
        BlobContainerClient blobContainerClient = client.getBlobContainerClient(containerName);

        blobContainerClient
                .getBlobClient(blobPath)
                .downloadToFile(dest);

    }
}

Dependencies

nettyTcNativeVersion = '2.0.7.Final'

compile group: 'com.azure', name: 'azure-storage-blob', version: '12.4.0'
compile group: 'com.azure', name: 'azure-identity', version: '1.0.6'

I have got the following error message

Exception in thread "main" reactor.core.Exceptions$ReactiveException: java.io.IOException: Connection reset by peer
    at reactor.core.Exceptions.propagate(Exceptions.java:393)
    at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:97)
    at reactor.core.publisher.Mono.block(Mono.java:1678)
    at com.azure.storage.common.implementation.StorageImplUtils.blockWithOptionalTimeout(StorageImplUtils.java:99)
    at com.azure.storage.blob.specialized.BlobClientBase.downloadToFileWithResponse(BlobClientBase.java:563)
    at com.azure.storage.blob.specialized.BlobClientBase.downloadToFile(BlobClientBase.java:488)
    at com.azure.storage.blob.specialized.BlobClientBase.downloadToFile(BlobClientBase.java:457)
    at com.blob.download.BlobDownload.main(BlobDownload.java:57)
    Suppressed: java.lang.Exception: #block terminated with an error
        at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:99)
        ... 6 more
Caused by: java.io.IOException: Connection reset by peer
    at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
    at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
    at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
    at sun.nio.ch.IOUtil.read(IOUtil.java:192)
    at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
    at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:247)
    at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1147)
    at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:347)
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:148)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:700)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:635)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:552)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:514)
    at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.lang.Thread.run(Thread.java:748)

Can you please guide me if there is any wrong usage in my implementation?

Answer 1

@Boney Jacob Welcome to Microsoft Q&A platform, Thank you for posting your query here!

Apologies for the delay in response here! I see you have posted the similar thread in the SO forum, Refer to the suggestion mentioned over-there and let me know if you have any question!

Hope this helps!

Kindly let us know if the above helps or you need further assistance on this issue.

---------------------------------------------------------------------------------------------------------------------------------

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

Answer 2

I am also facing the same issue any.
I am using Azure SDK.
Maven dependency as below:

<dependency>
   <groupId>com.azure</groupId>
   <artifactId>azure-storage-blob</artifactId>
   <version>12.9.0-beta.2</version>
</dependency>

Please find below the stack trace:

reactor.core.Exceptions$ReactiveException: io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer at 
reactor.core.Exceptions.propagate(Exceptions.java:393) at 
reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:97) at
 reactor.core.publisher.Mono.block(Mono.java:1678) at com.azure.storage.common.implementation.StorageImplUtils.blockWithOptionalTimeout(StorageImplUtils.java:128) at com.azure.storage.blob.specialized.BlobClientBase.downloadToFileWithResponse(BlobClientBase.java:728) at com.azure.storage.blob.specialized.BlobClientBase.downloadToFileWithResponse(BlobClientBase.java:699) at com.azure.storage.blob.specialized.BlobClientBase.downloadToFile(BlobClientBase.java:630) at

If anyone is having any solution, please let me know.