question

Pm-3513 avatar image
0 Votes"
Pm-3513 asked PramodValavala-MSFT answered

Send Email when file lands in Azure Data Lake Storage Gen 2

Currently using Synapse to process data, and I have a folder in my Azure Data Lake Storage Gen 2 that is called 'Failed'. When any bad data is processed and placed in 'Failed' I want to send an email notification. I was thinking I could do this with Logic Apps but I'm not seeing that capability. Any suggestions?

Thank you!!

azure-logic-appsazure-data-lake-storage
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

cooldadtx avatar image
0 Votes"
cooldadtx answered cooldadtx commented

You can do this with a durable function which is a specialized Azure Function. Logical apps can also do it. The differences boil down to how much integration and customization you want. Refer to this doc on the official differences.

In both cases you define a storage trigger that monitors the container you care about. When a blob is added or updated in the container then the trigger is fired and Azure spins up your DF or LA.

  1. Create a new DF/LA in Azure. The recommendation is to use consummation model.

  2. The DF/LA will either directly contain the logic to process the request or call out to an API that does.

  3. Add a storage trigger that points to the container you want to monitor. For logical apps you can see the instructions here.




· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@cooldadtx Thank you for the quick response! Do you know if you can customize to where it looks at only one specific directory?

0 Votes 0 ·

This is a common misconception for Azure Storage. There is no such thing as a folder/directory in containers. A container is just a list of blob data with names. A blob name can have slashes in it but that doesn't change anything. For example a container can contain both a blob named 'something\someone' and 'someone'. There are 2 separate blobs in the container and they are flat. Some UIs (such as the Azure Storage Preview) do assume that you're building a "folder-like" structure and therefore will show a directory structure but under the hood there is just the single blob file. That is why you cannot have empty "folders" in a container.

So, to answer your question, yes you can limit it to a "folder" by simply specifying the file mask when defining the trigger. An example is given here where you filter on blobs that have a name starting with original-. What I don't know is if it'll like a pattern that includes slashes in the name (outside the container). But you could try container/folderA/{name} which would in theory match files that begin with the name folderA/.

0 Votes 0 ·

If you cannot get the filtering to work at the trigger level then you can also just use the base trigger to get notified and then in your function activity check for a more complex name and abort if needed. It requires consumption but would still allow the use of functions.

0 Votes 0 ·
PramodValavala-MSFT avatar image
0 Votes"
PramodValavala-MSFT answered

@Pm-3513 Azure Data Lake Storage Gen 2 supports events via Azure Event Grid that can be used to trigger a logic app that sends your email using the outlook connector.

With Azure Event Grid, you can also filter events to the ones that you are interested in.


5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.