使用對應資料流程安全地轉換資料Transform data securely by using mapping data flow

適用於: Azure Data Factory Azure Synapse Analytics

如果您不熟悉 Azure Data Factory,請參閱 Azure Data Factory 簡介If you're new to Azure Data Factory, see Introduction to Azure Data Factory.

在本教學課程中,您將使用 Data Factory 使用者介面 (UI) 建立管線,以將資料 從 Azure Data Lake Storage Gen2 來源複製和轉換成 Data Lake Storage Gen2 接收 (兩者都允許 使用 ) 受控虛擬網路中的對應資料流程來存取選取的網路 Data Factory。In this tutorial, you'll use the Data Factory user interface (UI) to create a pipeline that copies and transforms data from an Azure Data Lake Storage Gen2 source to a Data Lake Storage Gen2 sink (both allowing access to only selected networks) by using mapping data flow in Data Factory Managed Virtual Network. 當您使用對應資料流程來轉換資料時,可以在本教學課程中擴充設定模式。You can expand on the configuration pattern in this tutorial when you transform data by using mapping data flow.

在本教學課程中,您會執行下列步驟:In this tutorial, you do the following steps:

  • 建立資料處理站。Create a data factory.
  • 使用資料流程活動建立管線。Create a pipeline with a data flow activity.
  • 使用四個轉換建立對應的資料流程。Build a mapping data flow with four transformations.
  • 對管線執行測試。Test run the pipeline.
  • 監視「資料流程」活動。Monitor a data flow activity.

必要條件Prerequisites

  • Azure 訂用帳戶Azure subscription. 如果您沒有 Azure 訂用帳戶,請在開始前建立免費 Azure 帳戶If you don't have an Azure subscription, create a free Azure account before you begin.
  • Azure 儲存體帳戶Azure storage account. 您可以使用 Data Lake Storage 作為 來源接收 資料存放區。You use Data Lake Storage as source and sink data stores. 如果您沒有儲存體帳戶,請參閱建立 Azure 儲存體帳戶,按照步驟建立此帳戶。If you don't have a storage account, see Create an Azure storage account for steps to create one. 請確定儲存體帳戶只允許從選取的網路存取。Ensure the storage account allows access only from selected networks.

我們將在本教學課程中轉換的檔案是 moviesDB.csv,可在此 GitHub 內容網站找到。The file that we'll transform in this tutorial is moviesDB.csv, which can be found at this GitHub content site. 若要從 GitHub 取出檔案,請將內容複寫到您選擇的文字編輯器,以將它儲存在本機做為 .csv 檔案。To retrieve the file from GitHub, copy the contents to a text editor of your choice to save it locally as a .csv file. 若要將檔案上傳至您的儲存體帳戶,請參閱 使用 Azure 入口網站上傳 blobTo upload the file to your storage account, see Upload blobs with the Azure portal. 這些範例會參考名為 sample 的容器。The examples will reference a container named sample-data.

建立 Data FactoryCreate a data factory

在此步驟中,您會建立資料處理站,並開啟 Data Factory UI,以在 data factory 中建立管線。In this step, you create a data factory and open the Data Factory UI to create a pipeline in the data factory.

  1. 開啟 Microsoft Edge 或 Google Chrome。Open Microsoft Edge or Google Chrome. 目前,只有 Microsoft Edge 和 Google Chrome 網頁瀏覽器支援 Data Factory UI。Currently, only Microsoft Edge and Google Chrome web browsers support the Data Factory UI.

  2. 在左側功能表上,選取 [建立資源] > [分析] > [資料處理站]。On the left menu, select Create a resource > Analytics > Data Factory.

  3. 在 [新增資料處理站] 頁面的 [名稱] 下,輸入 ADFTutorialDataFactoryOn the New data factory page, under Name, enter ADFTutorialDataFactory.

    資料處理站的名稱必須是「全域唯一」的名稱。The name of the data factory must be globally unique. 如果您收到有關名稱值的錯誤訊息,請輸入不同的資料處理站名稱 (例如 yournameADFTutorialDataFactory)。If you receive an error message about the name value, enter a different name for the data factory (for example, yournameADFTutorialDataFactory). 如需 Data Factory 成品的命名規則,請參閱 Data Factory 命名規則For naming rules for Data Factory artifacts, see Data Factory naming rules.

  4. 選取您要在其中建立資料處理站的 Azure 訂用帳戶Select the Azure subscription in which you want to create the data factory.

  5. 針對 [資源群組],採取下列其中一個步驟︰For Resource Group, take one of the following steps:

    • 選取 [使用現有的] ,然後從下拉式清單選取現有的資源群組。Select Use existing, and select an existing resource group from the drop-down list.
    • 選取 [建立新的] ,然後輸入資源群組的名稱。Select Create new, and enter the name of a resource group.

    若要了解資源群組,請參閱使用資源群組管理您的 Azure 資源To learn about resource groups, see Use resource groups to manage your Azure resources.

  6. 在 [版本] 下,選取 [V2]。Under Version, select V2.

  7. 在 [位置] 下,選取資料處理站的位置。Under Location, select a location for the data factory. 只有受到支援的位置會出現在下拉式清單中。Only locations that are supported appear in the drop-down list. 資料存放區 (例如 Azure 儲存體和 Azure SQL Database) 和計算 (例如,資料處理站所使用的 Azure HDInsight) 可位於其他區域。Data stores (for example, Azure Storage and Azure SQL Database) and computes (for example, Azure HDInsight) used by the data factory can be in other regions.

  8. 選取 [建立] 。Select Create.

  9. 建立完成後,您會在通知中心看到通知。After the creation is finished, you see the notice in the Notifications center. 選取 [移至資源],以移至 Data Factory 頁面。Select Go to resource to go to the Data Factory page.

  10. 選取 [編寫與監視],以在個別索引標籤中啟動 Data Factory 使用者介面。Select Author & Monitor to launch the Data Factory UI in a separate tab.

在 Data Factory 受控虛擬網路中建立 Azure IRCreate an Azure IR in Data Factory Managed Virtual Network

在此步驟中,您會建立 Azure IR,並啟用 Data Factory 受控虛擬網路。In this step, you create an Azure IR and enable Data Factory Managed Virtual Network.

  1. 在 Data Factory 入口網站中,移至 [ 管理],然後選取 [ 新增 ] 以建立新的 Azure IR。In the Data Factory portal, go to Manage, and select New to create a new Azure IR.

    顯示建立新 Azure IR 的螢幕擷取畫面。

  2. 在 [ 整合執行時間設定 ] 頁面上,根據所需的功能選擇要建立的整合執行時間。On the Integration runtime setup page, choose what integration runtime to create based on required capabilities. 在本教學課程中,選取 [ Azure]、[自我 裝載],然後按一下 [ 繼續]。In this tutorial, select Azure, Self-Hosted and then click Continue.

  3. 選取 [ azure ],然後按一下 [ 繼續 ] 建立 azure 整合執行時間。Select Azure and then click Continue to create an Azure Integration runtime.

    顯示新 Azure IR 的螢幕擷取畫面。

  4. 在 [虛擬網路設定 (預覽)] 中,選取 [啟用]。Under Virtual network configuration (Preview), select Enable.

    顯示啟用新 Azure IR 的螢幕擷取畫面。

  5. 選取 [建立] 。Select Create.

使用資料流程活動建立管線Create a pipeline with a data flow activity

在此步驟中,您將建立包含「資料流程」活動的管線。In this step, you'll create a pipeline that contains a data flow activity.

  1. 在 [現在就開始吧] 頁面中,選取 [建立管線]。On the Let's get started page, select Create pipeline.

    顯示建立管線的螢幕擷取畫面。

  2. 在管線的 [屬性] 窗格中,針對 [管線名稱] 輸入 TransformMoviesIn the properties pane for the pipeline, enter TransformMovies for the pipeline name.

  3. 在 [ 活動 ] 窗格中,展開 [ 移動和轉換]。In the Activities pane, expand Move and Transform. 將 [ 資料流程 ] 活動從窗格拖曳到管線畫布。Drag the Data Flow activity from the pane to the pipeline canvas.

  4. 在 [ 加入 資料流程] 快顯視窗中,選取 [ 建立新 的資料流程],然後選取 [ 對應資料流程]。In the Adding data flow pop-up, select Create new data flow and then select Mapping Data Flow. 當您完成時,請選取 [確定]Select OK when you're finished.

    顯示對應資料流程的螢幕擷取畫面。

  5. 在 [屬性] 窗格中,將您的資料流程命名為 TransformMoviesName your data flow TransformMovies in the properties pane.

  6. 在管線畫布的頂端列中,滑動 [ 資料流程調試 程式] 滑杆。In the top bar of the pipeline canvas, slide the Data Flow debug slider on. 偵錯工具模式允許針對即時 Spark 叢集進行轉換邏輯的互動式測試。Debug mode allows for interactive testing of transformation logic against a live Spark cluster. 資料流程叢集需要5-7 分鐘的時間來準備,且如果使用者打算進行資料流程開發,建議先開啟 debug。Data Flow clusters take 5-7 minutes to warm up and users are recommended to turn on debug first if they plan to do Data Flow development. 如需詳細資訊,請參閱偵錯模式For more information, see Debug Mode.

    顯示 [資料流程] debug 滑杆的螢幕擷取畫面。

在資料流程畫布中建立轉換邏輯Build transformation logic in the data flow canvas

建立資料流程之後,系統會自動將您傳送到資料流程畫布。After you create your data flow, you'll be automatically sent to the data flow canvas. 在此步驟中,您將建立會在 Data Lake Storage 中使用 moviesDB.csv 檔案的資料流程,並將 comedies 從1910的平均評等匯總至2000。In this step, you'll build a data flow that takes the moviesDB.csv file in Data Lake Storage and aggregates the average rating of comedies from 1910 to 2000. 然後,您會將此檔案寫回 Data Lake Storage。You'll then write this file back to Data Lake Storage.

新增來源轉換Add the source transformation

在此步驟中,您會將 Data Lake Storage Gen2 設定為來源。In this step, you set up Data Lake Storage Gen2 as a source.

  1. 在 [資料流程] 畫布中,選取 [ 新增來源 ] 方塊來新增來源。In the data flow canvas, add a source by selecting the Add Source box.

  2. 命名您的來源 MoviesDBName your source MoviesDB. 選取 [ 新增 ] 以建立新的源資料集。Select New to create a new source dataset.

  3. 選取 Azure Data Lake Storage Gen2,然後選取 [ 繼續]。Select Azure Data Lake Storage Gen2, and then select Continue.

  4. 選取 [ DelimitedText],然後選取 [ 繼續]。Select DelimitedText, and then select Continue.

  5. 將您的資料集命名為 MoviesDBName your dataset MoviesDB. 在 [連結服務] 下拉式清單中,選取 [ 新增]。In the linked service drop-down, select New.

  6. 在 [連結服務建立] 畫面中,為您的 Data Lake Storage Gen2 連結服務 ADLSGen2 命名,然後指定您的驗證方法。In the linked service creation screen, name your Data Lake Storage Gen2 linked service ADLSGen2 and specify your authentication method. 然後輸入您的連接認證。Then enter your connection credentials. 在本教學課程中,我們將使用 帳戶金鑰 來連線到儲存體帳戶。In this tutorial, we're using Account key to connect to our storage account.

  7. 請務必啟用 互動式製作Make sure you enable Interactive authoring. 可能需要一分鐘的時間來啟用。It might take a minute to be enabled.

    顯示互動式製作的螢幕擷取畫面。

  8. 選取 [測試連線]。Select Test connection. 它應該會失敗,因為儲存體帳戶不會啟用存取權,而不需要建立和核准私人端點。It should fail because the storage account doesn't enable access into it without the creation and approval of a private endpoint. 在錯誤訊息中,您應該會看到一個連結,可讓您建立私人端點,以供您建立受控私人端點。In the error message, you should see a link to create a private endpoint that you can follow to create a managed private endpoint. 替代方式是直接移至 [ 管理 ] 索引標籤,並 遵循本節中 的指示來建立受控私人端點。An alternative is to go directly to the Manage tab and follow instructions in this section to create a managed private endpoint.

  9. 讓對話方塊保持開啟,然後移至儲存體帳戶。Keep the dialog box open, and then go to your storage account.

  10. 遵循本節的指示來核准私人連結。Follow instructions in this section to approve the private link.

  11. 返回對話方塊。Go back to the dialog box. 選取 [測試連線],然後選取 [建立] 以部署已連結的服務。Select Test connection again, and select Create to deploy the linked service.

  12. 在 [資料集建立] 畫面的 [檔案 路徑 ] 欄位下,輸入檔案的所在位置。On the dataset creation screen, enter where your file is located under the File path field. 在本教學課程中,檔案 moviesDB.csv 位於容器 範例-資料 中。In this tutorial, the file moviesDB.csv is located in the container sample-data. 因為檔案具有標頭,所以請選取 [ 第一個資料列做為標頭 ] 核取方塊。Because the file has headers, select the First row as header check box. 從連線/存放區 選取,直接從儲存體中的檔案匯入標頭架構。Select From connection/store to import the header schema directly from the file in storage. 當您完成時,請選取 [確定]Select OK when you're finished.

    顯示來源路徑的螢幕擷取畫面。

  13. 如果您的 debug 叢集已啟動,請移至來源轉換的 [ 資料預覽 ] 索引標籤, 然後選取 [ 重新整理] 以取得資料的快照集。If your debug cluster has started, go to the Data Preview tab of the source transformation and select Refresh to get a snapshot of the data. 您可以使用資料預覽來確認是否已正確設定您的轉換。You can use the data preview to verify your transformation is configured correctly.

    顯示 [資料預覽] 索引標籤的螢幕擷取畫面。

建立受控私人端點Create a managed private endpoint

如果您在測試上述連接時未使用超連結,請遵循路徑。If you didn't use the hyperlink when you tested the preceding connection, follow the path. 現在您需要建立受控私人端點,以連線至所建立的已連結服務。Now you need to create a managed private endpoint that you'll connect to the linked service you created.

  1. 移至 管理 索引標籤。Go to the Manage tab.

    注意

    並非所有 Data Factory 執行個體圴可使用 管理 索引標籤。The Manage tab might not be available for all Data Factory instances. 如果您沒有看到該索引標籤,可以選取 [作者] > [連線] > [私人端點] 來存取私人端點。If you don't see it, you can access private endpoints by selecting Author > Connections > Private Endpoint.

  2. 移至 受控私人端點 區段。Go to the Managed private endpoints section.

  3. 受控私人端點 之下選取 [+新增]。Select + New under Managed private endpoints.

    顯示 [受控私人端點新增] 按鈕的螢幕擷取畫面。

  4. 從清單中選取 Azure Data Lake Storage Gen2 圖格,然後選取 [ 繼續]。Select the Azure Data Lake Storage Gen2 tile from the list, and select Continue.

  5. 輸入建立之儲存體帳戶的名稱。Enter the name of the storage account you created.

  6. 選取 [建立]。Select Create.

  7. 在等候幾秒鐘之後,您應該會看到建立的私人連結需要核准。After a few seconds, you should see that the private link created needs an approval.

  8. 選取之前建立的私人端點。Select the private endpoint that you created. 您會看到超連結,引導您在儲存體帳戶層級核准私人端點。You can see a hyperlink that will lead you to approve the private endpoint at the storage account level.

    顯示 [管理私人端點] 窗格的螢幕擷取畫面。

  1. 在儲存體帳戶中,移至 設定 區段下的 私人端點連線In the storage account, go to Private endpoint connections under the Settings section.

  2. 選取您所建立私人端點的核取方塊,然後選取 [ 核准]。Select the check box by the private endpoint you created, and select Approve.

    顯示 [私人端點核准] 按鈕的螢幕擷取畫面。

  3. 新增描述,然後選取 [是]。Add a description, and select yes.

  4. 回到 Data Factory 中 管理 索引標籤的 受控私人端點 區段。Go back to the Managed private endpoints section of the Manage tab in Data Factory.

  5. 大約一分鐘之後,您應該會看到私人端點的核准。After about a minute, you should see the approval appear for your private endpoint.

新增篩選準則轉換Add the filter transformation

  1. 在 [資料流程] 畫布的來源節點旁,選取加號圖示以新增轉換。Next to your source node on the data flow canvas, select the plus icon to add a new transformation. 您將新增的第一個轉換是 篩選準則The first transformation you'll add is a Filter.

    顯示新增篩選準則的螢幕擷取畫面。

  2. 命名您的篩選轉換 FilterYearsName your filter transformation FilterYears. 選取 [ 篩選準則 ] 旁的 [運算式] 方塊,以開啟 [運算式產生器]。Select the expression box next to Filter on to open the expression builder. 您將在這裡指定篩選準則。Here you'll specify your filtering condition.

    顯示 FilterYears 的螢幕擷取畫面。

  3. 資料流程運算式產生器可讓您以互動方式建立要在各種轉換中使用的運算式。The data flow expression builder lets you interactively build expressions to use in various transformations. 運算式可以包含內建函數、輸入架構中的資料行,以及使用者定義的參數。Expressions can include built-in functions, columns from the input schema, and user-defined parameters. 如需有關如何建立運算式的詳細資訊,請參閱 資料流程運算式產生器。For more information on how to build expressions, see Data flow expression builder.

    • 在本教學課程中,您會想要篩選喜劇類型中的電影,以在1910年和2000年之間推出。In this tutorial, you want to filter movies in the comedy genre that came out between the years 1910 and 2000. 因為年份目前為字串,所以您需要使用函數將它轉換成整數 toInteger()Because the year is currently a string, you need to convert it to an integer by using the toInteger() function. 使用大於或等於 (>=) 且小於或等於 (<=) 運算子來比較常值年份值1910和2000。Use the greater than or equal to (>=) and less than or equal to (<=) operators to compare against the literal year values 1910 and 2000. 將這些運算式與 and (&&) 運算子搭配使用。Union these expressions together with the and (&&) operator. 運算式的形式如下:The expression comes out as:

      toInteger(year) >= 1910 && toInteger(year) <= 2000

    • 若要找出哪些電影 comedies,您可以使用函式 rlike() 來尋找資料行內容中的「喜劇」模式。To find which movies are comedies, you can use the rlike() function to find the pattern 'Comedy' in the column genres. Rlike 運算式的聯集與年度比較以取得:Union the rlike expression with the year comparison to get:

      toInteger(year) >= 1910 && toInteger(year) <= 2000 && rlike(genres, 'Comedy')

    • 如果您有使用中的偵錯工具,您可以選取 [重新整理] 來確認您的 邏輯,以 查看與所用輸入相較的運算式輸出。If you have a debug cluster active, you can verify your logic by selecting Refresh to see the expression output compared to the inputs used. 您可以使用資料流程運算式語言來完成這項邏輯,以提供多個正確的答案。There's more than one right answer on how you can accomplish this logic by using the data flow expression language.

      顯示篩選運算式的螢幕擷取畫面。

    • 當您完成運算式之後,請選取 [儲存並完成]Select Save and finish after you're finished with your expression.

  4. 提取 資料預覽 ,以確認篩選器可正常運作。Fetch a Data Preview to verify the filter is working correctly.

    顯示已篩選資料預覽的螢幕擷取畫面。

新增匯總轉換Add the aggregate transformation

  1. 您將新增的下一個轉換是 架構修飾 詞下的 匯總 轉換。The next transformation you'll add is an Aggregate transformation under Schema modifier.

    顯示新增匯總的螢幕擷取畫面。

  2. 命名您的匯總轉換 AggregateComedyRatingName your aggregate transformation AggregateComedyRating. 在 [ 分組方式 ] 索引標籤上,從下拉式方塊中選取 [ ],以依電影的年份將匯總分組。On the Group by tab, select year from the drop-down box to group the aggregations by the year the movie came out.

    顯示匯總群組的螢幕擷取畫面。

  3. 移至 [ 匯總 ] 索引標籤。在左邊的文字方塊中,將匯總資料行命名為 AverageComedyRatingGo to the Aggregates tab. In the left text box, name the aggregate column AverageComedyRating. 選取 [右運算式] 方塊,即可透過運算式產生器來輸入匯總運算式。Select the right expression box to enter the aggregate expression via the expression builder.

    顯示匯總資料行名稱的螢幕擷取畫面。

  4. 若要取得資料行 分級 的平均值,請使用 avg() 彙總函式。To get the average of column Rating, use the avg() aggregate function. 由於 等是字串並 avg() 接受數位輸入,因此我們必須透過函數將值轉換成數位 toInteger()Because Rating is a string and avg() takes in a numerical input, we must convert the value to a number via the toInteger() function. 此運算式看起來像這樣:This expression looks like:

    avg(toInteger(Rating))

  5. 選取 [儲存並 在完成之後完成]。Select Save and finish after you're finished.

    顯示儲存匯總的螢幕擷取畫面。

  6. 移至 [ 資料預覽 ] 索引標籤,以查看轉換輸出。Go to the Data Preview tab to view the transformation output. 請注意,其中只有兩個數據行: yearAverageComedyRatingNotice only two columns are there, year and AverageComedyRating.

新增接收轉換Add the sink transformation

  1. 接下來,您想要在 目的地 下新增 接收 轉換。Next, you want to add a Sink transformation under Destination.

    顯示新增接收的螢幕擷取畫面。

  2. 命名接收 接收器Name your sink Sink. 選取 [ 新增 ] 以建立接收資料集。Select New to create your sink dataset.

    顯示建立接收器的螢幕擷取畫面。

  3. 在 [ 新增資料集 ] 頁面上,選取 Azure Data Lake Storage Gen2 然後選取 [ 繼續]。On the New dataset page, select Azure Data Lake Storage Gen2 and then select Continue.

  4. 在 [ 選取格式 ] 頁面上,選取 [ DelimitedText ],然後選取 [ 繼續]。On the Select format page, select DelimitedText and then select Continue.

  5. 將接收資料集命名為 MoviesSinkName your sink dataset MoviesSink. 針對 [已連結的服務],選擇您為來源轉換所建立的相同 ADLSGen2 連結服務。For linked service, choose the same ADLSGen2 linked service you created for source transformation. 輸入要寫入資料的輸出檔案夾。Enter an output folder to write your data to. 在本教學課程中,我們會寫入容器 範例資料 中的資料夾 輸出In this tutorial, we're writing to the folder output in the container sample-data. 資料夾不需要事先存在,而且可以動態建立。The folder doesn't need to exist beforehand and can be dynamically created. 選取 [ 第一個資料列做為標頭 ] 核取方塊,然後選取 [ 入架構]。Select the First row as header check box, and select None for Import schema. 選取 [確定]。Select OK.

    顯示接收路徑的螢幕擷取畫面。

現在您已完成建立資料流程。Now you've finished building your data flow. 您已經準備好在您的管線中執行它。You're ready to run it in your pipeline.

執行和監視資料流程Run and monitor the data flow

您可以在發行管線之前進行調試。You can debug a pipeline before you publish it. 在這個步驟中,您會觸發資料流程管線的 debug 執行。In this step, you trigger a debug run of the data flow pipeline. 雖然資料預覽不會寫入資料,但 debug 回合會將資料寫入至您的接收目的地。While the data preview doesn't write data, a debug run will write data to your sink destination.

  1. 移至管線畫布。Go to the pipeline canvas. 選取 [ debug ] 以觸發偵錯工具執行。Select Debug to trigger a debug run.

  2. 資料流程活動的管線偵錯工具會使用作用中的 debug 叢集,但仍需要至少一分鐘的時間來初始化。Pipeline debugging of data flow activities uses the active debug cluster but still takes at least a minute to initialize. 您可以透過 [ 輸出 ] 索引標籤來追蹤進度。執行成功後,請選取 [執行詳細資料] 的眼鏡圖示。You can track the progress via the Output tab. After the run is successful, select the eyeglasses icon for run details.

  3. 在 [詳細資料] 頁面上,您可以看到資料列數和每個轉換步驟所花的時間。On the details page, you can see the number of rows and the time spent on each transformation step.

    顯示監視執行的螢幕擷取畫面。

  4. 選取轉換來取得資料行和資料分割的詳細資訊。Select a transformation to get detailed information about the columns and partitioning of the data.

如果您已正確遵循本教學課程,您應該將83個數據列和2個數據行寫入至您的接收資料夾。If you followed this tutorial correctly, you should have written 83 rows and 2 columns into your sink folder. 您可以藉由檢查 blob 儲存體來確認資料是否正確。You can verify the data is correct by checking your blob storage.

總結Summary

在本教學課程中,您已使用 Data Factory UI 來建立管線,以將資料從 Data Lake Storage Gen2 來源複製和轉換成 Data Lake Storage Gen2 接收 (兩者都只允許使用 ) 受控虛擬網路中的對應資料流程來存取選取的網路 Data Factory。In this tutorial, you used the Data Factory UI to create a pipeline that copies and transforms data from a Data Lake Storage Gen2 source to a Data Lake Storage Gen2 sink (both allowing access to only selected networks) by using mapping data flow in Data Factory Managed Virtual Network.