question

NamNguyen-1476 avatar image
0 Votes"
NamNguyen-1476 asked NamNguyen-1476 commented

Azure Data Factory - OutOfMemoryError

Source: Hubspot
Sink: parquet file in Azure blob storage

Even when i adjust Core count to 16 (+16 core), following error still pop up when running pipeline:

![162491-image.png][1]


Pipeline run ID:
50d2a7b0-c483-449d-8bb8-5f846f8e0914


azure-data-factoryazure-data-lake-storageazure-face
image.png (77.0 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

HimanshuSinha-MSFT avatar image
1 Vote"
HimanshuSinha-MSFT answered NamNguyen-1476 commented

Hello @NamNguyen-1476 ,
Thanks for the ask and using Microsoft Q&A platform .
Just wanted to suggest you to try the below options and let us know if that helps .

  1. Please try out the Compute type to "Memory optimatized " and publish the changes and see if that helps .

162624-image.png

2.Also try to update the Azure IR to a higher core

162642-image.png



Please do let me know how it goes .
Thanks
Himanshu


  • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification

  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators



image.png (36.3 KiB)
image.png (73.6 KiB)
· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @NamNguyen-1476 ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .
Thanks
Himanshu

0 Votes 0 ·

Hello @NamNguyen-1476 ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .
Thanks
Himanshu

0 Votes 0 ·
NamNguyen-1476 avatar image NamNguyen-1476 HimanshuSinha-MSFT ·

Hi,
Thank you it works

0 Votes 0 ·