question

prasadk-6267 avatar image
0 Votes"
prasadk-6267 asked HimanshuSinha-MSFT commented

Azure data lake gen2 uploading files— throttling issue

uploading files to azure datalake gen2 blob storage.Getting throttling issue while uploading with 2000 files. while doing Parallel.ForEach(zzz.Items, (Item) => {

             DataLakeFileClient fileClient = directoryClient.CreateFile(xxxx + ".json");
         });

So iam getting below issue

One or more errors occurred. (Service request failed. Status: 429 (Too Many Requests) Content: {"type":"https://xxx/errors/too-many-requests","title":"Resource utilization has surpassed the assigned quota","policy":"Total Requests","status":429}

azure-data-lake-storage
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @prasadk-6267 ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .
Thanks
Himanshu

0 Votes 0 ·

Hello @prasadk-6267 ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .
Thanks
Himanshu

0 Votes 0 ·

1 Answer

HimanshuSinha-MSFT avatar image
0 Votes"
HimanshuSinha-MSFT answered

Hello @prasadk-6267 ,
Thanks for the ask and using the Microsoft Q&A platform .

I suggest you to please tweek the batch count and that should help . You can also add a wait activity .

90912-image.png


Please do let me know how it goes .
Thanks
Himanshu
Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members



image.png (13.4 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.