AzCopy Support for On premise HDFS

Soumitra 1 Reputation point
2020-08-29T04:12:27.9+00:00

Hi,
Can we use AzCopy to transfer on-prem hdfs files to ads gen2 ?

Thanks,
Soumitra.

Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
2,722 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,644 questions
0 comments

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Vaibhav Chaudhari 38,631 Reputation points
    2020-08-29T05:11:14.193+00:00

    Unfortunately I don't see On-premise hdfs support in the document here - storage-ref-azcopy-copy

    But see if below options help i.e. ADF & Azure data box

    https://learn.microsoft.com/en-us/azure/data-factory/data-migration-guidance-hdfs-azure-storage

    https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-migrate-on-premises-hdfs-cluster

    ----------

    Please don't forget to Accept Answer and Up-vote if the response helped -- Vaibhav

    0 comments
  2. Sumarigo-MSFT 43,806 Reputation points Microsoft Employee
    2020-08-31T07:38:38.65+00:00

    @Soumitra Firstly, apologies for the delay in responding here and any inconvenience this issue may have caused.

    so you cannot access the local hdfs as a file-system. Can we use AzCopy to transfer on-prem hdfs files to ads gen2 ? NO

    You may wish to leave your feedback on blocking access based on the session details on Uservoice.

    Workaround: https://learn.microsoft.com/en-us/azure/data-factory/connector-azure-data-lake-storage
    Hope this helps!

    Kindly let us know if the above helps or you need further assistance on this issue.

    --------------------------------------------------------------------------------------------------------------------------------

    Please don’t forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.