question

CatameoFredTaguigCity1634-6104 avatar image
0 Votes"
CatameoFredTaguigCity1634-6104 asked ErnestoMejia-2102 commented

Is there now a new size limit for variables on Azure Data Factory?

We pass in the columns mapping of our Copy data activities thru pipeline parameters.
91827-image.png


But first we get the columns mapping from a Lookup table then store it on a variable which we pass to the pipeline with the Copy data activity.
91854-image.png

This used to work with no problems even in production.
But starting last night, we now encounter an error on our dev ADF that there is now a limit of 1024 bytes for the variable intermittently (some pipelines are still working ok).
91855-image.png


Has there been a recent limit on variable size that has been imposed? Does this 1024 bytes limit also apply to parameters? If so, how can we make the columns mapping for Copy data activity dynamic?


azure-data-factory
image.png (14.3 KiB)
image.png (17.3 KiB)
image.png (33.4 KiB)
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

same here. it started on Monday. Though we manage to circumvent the situation, I still demand answer from ADF team. Is this going to be subscriptions limitation or soon to be permanent limitations.

I also noticed, it's not happening on other data-center. ,Could this be just bug?

ADF team, please respond.

0 Votes 0 ·

1 Answer

CatameoFredTaguigCity1634-6104 avatar image
1 Vote"
CatameoFredTaguigCity1634-6104 answered PRADEEPCHEEKATLA-MSFT commented

Managed to do a workaround fix by not using intermediate variables anymore. Instead, directly passing in the output of the Lookup activity to the Copy data activity parameter. No more error.

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @CatameoFredTaguigCity1634-6104,

Glad to know that your issue has resolved. And thanks for sharing the solution, which might be beneficial to other community members reading this thread.

0 Votes 0 ·