question

msftsyn7270 avatar image
0 Votes"
msftsyn7270 asked msftsyn7270 commented

Azure Synapse Analytics - contentMD5

I am wanting to do a quality check on my data to ensure that today's file does not look like the previous day's file. In data factory, you can easily accomplish this with a Get Metada activity and specifying your argument as contentMD5. This functionality is not provided in Azure Synapse. I've tried querying the database to check for duplicate data, but the runtime is extremely long, as I have to go through this process for each file (over 100 files). Is there any workaround that you know of? Or is there a timeline for when the contentMD5 functionality will be available in Synapse? Appreciate the help.

azure-data-factoryazure-synapse-analyticsazure-data-lake-storage
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

MarkKromer-MSFT avatar image
0 Votes"
MarkKromer-MSFT answered msftsyn7270 commented

It is there in Synapse pipelines Get Metadata activity. 102550-synapsemeta.png



synapsemeta.png (57.5 KiB)
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

thank you for your help!

0 Votes 0 ·