question

ewinkiser avatar image
0 Votes"
ewinkiser asked JoshuaAbad-1992 answered

How is a Dataset used in an ADF Web Activity? PLEASE HELP

HELLO!

How is the dataset used in the Web Activity? Can it be used in the Body? Do you know of an example? I searched and did not find one...

I need to pass data within a JSON Blob to items in the Body of a Web Activity (PATCH) and wondered if a dataset could help me.

Thanks!
Mike Kiser

azure-data-factory
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @ewinkiser and thank you for your question.

I am reaching out internally to find out the expected behavior of this feature.

0 Votes 0 ·
MartinJaffer-MSFT avatar image
1 Vote"
MartinJaffer-MSFT answered ewinkiser commented

I got some details of how the dataset / linked service feature in Web Activity works.

If the contents of the body are in a JSON format, AND the a dataset it chosen, then the definition of the dataset and its associated linked service is added to the body. Sadly, this will not help put the content of the blob in the body.

As example I tested by having the web write to blob:

45021-image.png

Then when I checked the blob the web activity URL pointed to:

44915-image.png

This sort of thing might be useful if you wanted to tell another service how to pull data from one of your sources.

This said, I think there is a different solution. Use another web activity to fetch the contents of the JSON blob, and pass the output into the body of your PATCH web activity.



image.png (157.1 KiB)
image.png (221.6 KiB)
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Martin, Please see below...

0 Votes 0 ·
ewinkiser avatar image
0 Votes"
ewinkiser answered RahulShashwat-1202 commented

Wow, Martin!! You have saved me again! I am writing an integration (pipeline) between one of our Vendors and Ceridian. I have a JSON doc like below that I have to pass through the body of a web Activity to the Ceridian REST API to update Employee Status' as follows. After reading your answer several times, I wanted to make sure that I understood. So I can not put the following Body in a Blob as a json file and pass it as a Dataset if I understand correctly? That would be the "easiest" way to go. I am going to try your following solution: (why can't anything be easy LOL)....

"This said, I think there is a different solution. Use another web activity to fetch the contents of the JSON blob, and pass the output into the body of your PATCH web activity."

{
"EmployeeNumber": "010004",
"EffectiveStart": "2020-12-04T04:42:27.193Z",
"EffectiveEnd": "2020-12-04T04:42:27.193Z",
"EmploymentStatus": {
"IsBenefitArrearsEnabled": true,
"XRefCode": "ACTIVE",
"ShortName": "string",
"LongName": "string",
"LastModifiedTimestamp": "2020-12-04T04:42:27.193Z"
},
"EmploymentStatusGroup": {
"XRefCode": "ACTIVE",
"ShortName": "string",
"LongName": "string",
"LastModifiedTimestamp": "2020-12-04T04:42:27.193Z"
},
"PayType": {
"XRefCode": "Salaried",
"ShortName": "string",
"LongName": "string",
"LastModifiedTimestamp": "2020-12-04T04:42:27.193Z"
}

}

You have saved me several times and I can't tell you how much I appreciate it!!!
MIKE KISER

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thank you very much for your feedback MIKE KISER. Much appreciated. Lately I have seen an uptick in similar or related asks. I think the ability to pass a blob as body sound like a great feature ask. While I make a note to mention this in my next meeting, could you suggest/upvote such a thing in the feedback forum?


1 Vote 1 ·

Hi Mike, Martin,

I need to a similar activity which has been done in this thread. I need to send data to a REST API from a Blob Folder. Initially, I used look-up activity to extract data from the Data Folder and pass it in the body of Web Activity. The problem with this approach is that the Look-up activity has a limitation of 4000 Rows and hence not all my data was getting transferred.

I need to try the dataset feature which is there in the Web Activity. But I'm not sure how to use that dataset in the body of the Web Activity.

Any help will be appreciated.

Thanks
Rahul


0 Votes 0 ·
JoshuaAbad-1992 avatar image
0 Votes"
JoshuaAbad-1992 answered

I'm upvoting on this one too dataset link is kind of misleading. Assumption is you can use a dataset to pass a payload in your request

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.