Synapse Analytics - Pipeline - Activity timed out

Justin Moen 1

I am getting an error (Activity timed out) when trying to write around 14k rows (about 10 columns) to the data lake gen 2 using a COPY activity in the data pipeline. (the timeout setting is 30 min) If I limit my results to ~20 rows (completes in less than 1 min), I can get the data to write to the data lake, so I know that it is able to write to the lake (.parquet files), but can't figure out why ~20 rows work and ~14k times out. Also, this is the only job that was running at the time, so it is not competing for resources.

Any thoughts or advice is much appreciated.

AnnuKumari-MSFT 30,601 Reputation points Microsoft Employee

2022-01-24T05:31:56.31+00:00

Hi @Justin Moen ,
Following up to see if the below suggestion was helpful, kindly do click Accept Answer and Up-Vote for the same. In case you have any further query please do let us know.
Justin Moen 1 Reputation point

2022-01-24T15:41:32.397+00:00

Thanks for the response, but unfortunately, these suggestions have not helped us solve our issue. With only trying to write 14k rows to a ADLG2 location, I feel like this should be something that Azure should be able to do easily without much configuration changes.
AnnuKumari-MSFT 30,601 Reputation points Microsoft Employee

2022-01-25T07:01:58.433+00:00

@Justin Moen ,
Could you please help me with further more details on your data source and sink type so that I can repro your scenario and try to help more. Are you trying to copy data from on-premise sql server to ADLS in parquet format ? Please provide more insights like this so that I can recreate your scenario and help more.

1 answer

AnnuKumari-MSFT 30,601 Reputation points Microsoft Employee

2022-01-12T10:41:26.16+00:00
Hi @Justin Moen ,

Welcome to Microsoft Q&A platform. Could you please clarify why have you set the activity timeout for Copy-data activity to 30mins?

By default, the activity timeout is set to 7 days. In order to make the task of copying data more efficient you can increase the DIU(Data integration unit) present in the settings of copy activity . To increase the parallelism , you can increase the DOP(Degree of parallelism) .

Kindly refer to this article for more details on optimization : Copy activity performance optimization features

Hope this will help. Please let us know if any further queries.

------------------------------

Please don't forget to click on or upvote button whenever the information provided helps you.
Original posters help the community find answers faster by identifying the correct answer. Here is how

Want a reminder to come back and check responses? Here is how to subscribe to a notification

If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators
Please sign in to rate this answer.
Justin Moen 1 Reputation point

2022-01-12T14:48:07+00:00

Thank you for your response. We set the activity timeout to 30 min as we are only trying to process 14k rows and believe that this should take only a couple of minutes and don't want the process to run for a long time if it wont finish in a timely manner.

I have looked at the link provided and see the options and unfortunately, the settings here did not help performance.

As I mentioned, we are trying to copy a very small set of data and the do not have other jobs running at the same time. Is this normal and expected performance or is there something else that we should be doing to copy the 14k records from source to the ADLS Gen 2?

Justin Moen 1 Reputation point

2022-01-13T14:27:23.007+00:00

Thanks for the response. I set the timeout to 30 min as I don't believe that it should take 30 min to write 14k rows to the ADLS Gen2. Also, we have tried some of the tips in the link provided, but still the performance is still slow. In looking at the logs, most (90%) of the time that this is running is on the SINK side.

As I mentioned, there is nothing else running in the environment, and with the DIU set to "Auto", is the performance that should be expected?

Thanks!

AnnuKumari-MSFT 30,601 Reputation points Microsoft Employee

2022-01-19T12:19:18.703+00:00

Hi @Justin Moen ,
Performance for copying the data depends on lots of parameters like - Source dataset type, Sink dataset type, Source location, Sink location, Additional settings.
I know you want to copy data from source to ADLS . Could you please through more insights on what source you are using . Also, if possible could you please share screenshot of copy activity for source and sink tabs.

Also, you can enable staging in Settings of Copy activity depending on kind of source data you are using.

Sometimes it takes a while to perform a hybrid data movement (that is, to copy from an on-premises data store to a cloud data store) over a slow network connection. To improve performance, you can use staged copy to compress the data on-premises so that it takes less time to move data to the staging data store in the cloud. Then you can decompress the data in the staging store before you load into the destination data store.

To get more details on how staged copy works , kindly refer this article : Staged copy
If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.
Sign in to comment