question

MatthiasNohl avatar image
0 Votes"
MatthiasNohl asked ShaikMaheer-MSFT commented

Reading metadata ADLS gen2 GetMedata activitiy?

We are happy to see that we are able to write individual metadata to blobs:
https://docs.microsoft.com/en-us/azure/data-factory/connector-azure-data-lake-storage#azure-data-lake-storage-gen2-as-a-sink-type


This helps us a lot for identifying PII. BUT:
The GetMetadata activity in ADF doesn’t seem to accept individual attributes within the field list. Even dynamic content with an expression is not working118340-adf1.png118356-adf2.png

Any ideas?


azure-data-factory
adf1.png (5.1 KiB)
adf2.png (36.1 KiB)
· 6
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @MatthiasNohl,

Welcome to Microsoft Q&A forum and thanks for reaching out.

I was able to reproduce the problem and reaching out internally to understand the exact use case or when dynamic content is applicable/valid in GetMetadata activity -> Field List

We will get back to you as soon as we have an update on the same.

Thank you for your patience.

1 Vote 1 ·

Regarding UseCase:
We would like to trigger different workflows based on the tags. Since the tags can now be written via the ADF, it would be great if they could also be read out via the ADF.

0 Votes 0 ·

Hi @MatthiasNohl,

Correct me if my understanding is wrong.

It seems you have a variable inside your pipeline called "vMetadataClassification" and based on value of that variable you would like to decide your workflow?

If above statement is true, then in that case you go with IF activity in Azure data factory. To know more about If activity kindly Click here.

Between, using GetMetadata activity we can get below information from data lake storage Gen2. Click Here to know more about same.

  • temName(file/folder)

  • itemType(file/folder)

  • size(file)

  • lastModified(file/folder)

  • childItems(folder)

  • contentMD5(file)

  • structure(file)

  • columnCount(file)

  • exists(file/folder)




0 Votes 0 ·

Since I am now able to write arbitrary tags per blob, I also want to be able to read arbitrary tags. That's what the parameter was for, which of course doesn't contain any fields from the default doc :)
So, yes, you got me wrong.

0 Votes 0 ·

Hi @MatthiasNohl ,

Thank you for quick response :-)

At this moment, GetMetadata Activity can only get below information from ADLS Gen2. I doubt if we can access aerbitrary tags per blob using GetMetaData activity. We will check this with internally and get back to you. Thank you.

  • ItemName(file/folder)

  • itemType(file/folder)

  • size(file)

  • lastModified(file/folder)

  • childItems(folder)

  • contentMD5(file)

  • structure(file)

  • columnCount(file)

  • exists(file/folder)

0 Votes 0 ·
Show more comments

1 Answer

KranthiPakala-MSFT avatar image
0 Votes"
KranthiPakala-MSFT answered ShaikMaheer-MSFT commented

Hi @MatthiasNohl,

Sorry for the delay.

After having conversation with internal team, it is confirmed that reading arbitrary tags/custom metada using Get Metadata activity is not supported currently. As called out by @ShaikMaheer-MSFT the metadata properties mentioned in the public document are only retrievable at the moment by a Get metadata activity.

I would recommend you to please submit a feature request in Azure Data factory user voice forum for the same: https://feedback.azure.com/forums/270578-data-factory
All the feedback shared in this forum are actively monitored and reviewed by ADF product team.

Also please do share the feedback link here once it is posted so that we can share the feedback, along with user voice link to the product team to further review. Submitting a feature request idea also will give an opportunity for others with similar idea to up-vote and comment on your suggestion, which would help increase the priority of feature implementation.

Hope this info helps. Do let us know if you have further query.



  • Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification.



· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @MatthiasNohl,

Following up to see if the above information was helpful. If you have further query, please do let us know.

Than you

0 Votes 0 ·

Hi @MatthiasNohl,

Just Following up to see if the above information was helpful. If you have further query, please do let us know.

Than you

0 Votes 0 ·