question

pankajchaturvedi-6530 avatar image
0 Votes"
pankajchaturvedi-6530 asked AnthonyRyan-6351 commented

Issue while loading data in ADLS GEN2(SINK)/Extracting files from ADLS GEN2(SOURCE)

Hi Team,

I am extracting data from REST API and loading into ADLS and ADLS to SQL DB.

I am encountering the below errors:

activity flow-- copy activity(REST API to ADLS)----->copy activity(ADLS to SQL DB)

step 1- while copying data from rest api to ADLS:

Failure happened on 'Sink' side.

ErrorCode=AdlsGen2OperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=ADLS Gen2 operation failed for: Operation returned an invalid status code

'PreconditionFailed'. Account: 'testadl'. FileSystem: 'INPUT. Path: 'input test /YYYY=2020/MM=11/DD=26/customer_test_2020-11-26.json'. ErrorCode: 'LeaseLost'. Message: 'A lease ID was specified, but the lease for the resource has expired.'. RequestId: 'a9a153a7-401f-007f-739e-c4fe17000000'. TimeStamp: 'Fri, 27 Nov 2020 09:18:26

GMT'..,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.Azure.Storage.Data.Models.ErrorSchemaException,Message=Operation returned an invalid status code 'PreconditionFailed',Source=Microsoft.DataTransfer.ClientLibrary,'

step 2- while copying data from ADLS to SQL DB:

Details
Failure happened on 'Source' side. 'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=ADLS Gen2 operation failed for: Operation returned an invalid status code **'RequestedRangeNotSatisfiable'. Account: 'testadl'.

FileSystem: 'INPUT'. Path: 'input test /YYYY=2020/MM=11/DD=26/customer_testcase_2020-11-26.json'. ErrorCode: 'InvalidRange'. Message: 'The range specified is invalid for the current size of the resource.'.** RequestId: 'aa879dc9-e01f-0097-22c1-c403ec000000'. TimeStamp: 'Fri, 27 Nov 2020 13:28:44

GMT'..,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.Azure.Storage.Data.Models.ErrorSchemaException,Message=Operation returned an invalid status code

'RequestedRangeNotSatisfiable',Source=Microsoft.DataTransfer.ClientLibrary,'


I am unable to get why those errors are generating. Could you please someone help out on this.

Thanks,
pankaj








azure-data-factoryazure-data-lake-storage
· 5
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi Team, Could you please respond here. Thanks!

0 Votes 0 ·

Hi @pankajchaturvedi-6530,

Welcome to Microsoft Q&A Platform. Thanks for posting the query.

The 'LeaseLost' error seems to be related to outage at ADLS side and further investigation is needed to find root cause of both the errors. Please share data factory name, region and the pipeline run ids that are failing with the above errors and I will work with product team to investigate further.

Looking forward for your response!

0 Votes 0 ·

Hi Haritha,

Thanks for your response.

Sorry, i am unable to share ADF name, it is confidential information

pipeline run ID: b18488fd-477f-45a8-a82a-700b12b3e1d1

region :North Europe

Please let me know if you do find any solution.

0 Votes 0 ·

Hi @pankajchaturvedi-6530,

Thanks for sharing the run id and region, I will work with our team to investigate further and if we need confidential information, we will connect with you offline. I will update you as soon as I hear from the internal team.

Thanks for your patience!

0 Votes 0 ·
Show more comments
DeromeEric-0001 avatar image
1 Vote"
DeromeEric-0001 answered AnthonyRyan-6351 commented

Hello,
Is this issue resolved ?
We're getting this error which seems the same :
ErrorCode=AdlsGen2OperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=ADLS Gen2 operation failed for: Operation returned an invalid status code 'PreconditionFailed'. Account: 'povazudevstoragedatalake'. FileSystem: '01-brute'. Path: 'lnkHoraireChrono/2021/08/26/lnkHoraireChrono_20200402.parquet'. ErrorCode: 'LeaseLost'. Message: 'A lease ID was specified, but the lease for the resource has expired.'. RequestId: 'c04302f3-c01f-0013-6598-9a5bbe000000'. TimeStamp: 'Thu, 26 Aug 2021 16:32:59 GMT'..,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.Azure.Storage.Data.Models.ErrorSchemaException,Message=Operation returned an invalid status code 'PreconditionFailed',Source=Microsoft.DataTransfer.ClientLibrary,'", "failureType": "UserError", "target": "CPD_SQL_HoraireChrono_DL_Raw", "details": [] }

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.


The error above : ErrorCode: 'LeaseLost'. Message: 'A lease ID was specified, but the lease for the resource has expired.'. happens in the ForEach loop in a pipeline Activity, in which (inside the ForEach) there is a SELECT FROM in a SQL Server Database and that dataset is going into a parquet file in our Azure DataLake. When the file is being written, that exception seems to happen. What is it with that Lease, is it something the infrastructure manages, because we don't seem to manage it... or have control over it. ??

1 Vote 1 ·

Same issue here. Google took me here. Still trying to find a solution.

0 Votes 0 ·
HarithaMaddi-MSFT avatar image
0 Votes"
HarithaMaddi-MSFT answered HarithaMaddi-MSFT commented

Hi @pankajchaturvedi-6530,

We have not received a response from you. Please suggest if the first error due to concurrent runs is resolved. For the second error, product team suggested that RequestedRangeNotSatisfiable indicates the pipeline is trying to access past the length of the file/blob. From this page, If the offset is invalid and exceeds the blob’s total length, the request will return status code 416 (Requested Range Not Satisfiable). Please check the configuration details and let us know if the issue persists.

Thanks for your patience!



  • Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification.

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @pankajchaturvedi-6530,

We have not received a response from you. Are you still facing the issue? If you found a solution, would you please share it here with the community? Otherwise, let us know and we will continue to engage with you on the issue.

Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members

0 Votes 0 ·

Hi @pankajchaturvedi-6530,

We still have not heard back from you. Are you still facing the issue? If you found a solution, would you please share it here with the community? Otherwise, let us know and we will continue to engage with you on the issue.

Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members

0 Votes 0 ·