Hello ,
I want to use Web or Rest API as a source in Dataflow transformation and Sink into Azure SQL Database by performing certain transformation like join, lookup and then sink to azure
sql.
Please reply and give suggestion.
Regards,
Anjali Maithani
You will have to use some intermediate layer for this as I don't see Rest API supported as source in here -
https://docs.microsoft.com/en-us/azure/data-factory/data-flow-source
Probably you can load data using Pipeline Copy activity from API to data lake and then in Data flow Data lake to Azure SQL after transformation
Please don't forget to Accept Answer and Up-vote if the response helped -- Vaibhav
Hello @AnjaliMaithani-7687 ,
Thanks for the ask and using the Microsoft Q&A platform .
Just to add to what @VaibhavChaudhari pointed out , since in your case the sink is SQL , rather then creating a blob you can create a intermediate/staging table in SQL also . Well the advantage with this design is that you will have all the final and intermediate table in SQL , but then you may be loose from CPU power .
Please do let me know how it goes .
Thanks
Himanshu
Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members
7 people are following this question.
