Is there any python version reference for azure data factory unit test?
So far I only found one uisng .NET https://richardswinbank.net/adf/set_up_automated_testing_for_azure_data_factory
Any suggestion will be greatly appreciate.
Is there any python version reference for azure data factory unit test?
So far I only found one uisng .NET https://richardswinbank.net/adf/set_up_automated_testing_for_azure_data_factory
Any suggestion will be greatly appreciate.
Hello @YangChowmun-0538 and welcome to Microsoft Q&A.
After reading the blog entry you linked, I can summarize it as:
Using an API to create a pipeline run, and then check whether the pipeline run results return succeeded or failed.
This can be done in any of the SDK languages. Python is one of them. This is effectively programmatic monitoring on a test environment.
There is a quickstart which includes the necessary python calls.
Based on what you said in your previous message quoted below, it is not unit tests, but belongs to other tests. Please clarify on this.
Using an API to create a pipeline run, and then check whether the pipeline run results return succeeded or failed.
I agree with you, @PingpongSet , this is not unit test. In the context of Azure Data Factory, unit tests are not strictly possible.
Suppose we have a pipeline with 3 activities in series. In order, let us call them A, B, and C. (A -> B -> C)
We cannot run the activities in isolation. There is no functionality for that. This holds especially true when one relies on the output of another. Such a dependency probably breaks the definition of "unit test" anyway.
The Data Factory debug has a feature where we can stop execution after a chosen activity. This would let us run A, A&B, or A&B&C. We cannot run just B or just C or B&C. Pipeline execution starts at the beginning (A).
I suppose we could put each activity in separate pipelines, with mockups to handle inputs. This would be closer to the ideal of unit testing. We can retrieve Activity Run details given a Pipeline Run ID. This provides data similar to debug runs, but in a programatically available format.
In general, when ti comes to Data Factory, a different sort of testing is used.
I see you made a related question post @PingpongSet . Do you still desire attention there?
6 people are following this question.