Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,717 questions
0 answers
Copy activity fails with the Error while using Azure Key Vault in my REST Linked Service
I want to fetch data from OdataURL in my Blob storage, so I am using Copy activity to fetch the same using REST in my source and Blob storage in my Sink. I have used REST Linked Service in my source dataset, so while hardcoding the credentials in LS, the…
asked 2024-05-22T08:38:54.8433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(3.2, 34%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHV%3C/text%3E%3C/svg%3E)
Harshita Vishwakarma
40
Reputation points
asked 2024-05-22T08:38:54.8433333+00:00
Harshita Vishwakarma
40
Reputation points
0 answers
Is it possible to hide a pipeline created in Azure Data Factory or Azure Synapse Analytics for a particular user/s?
Scenario : Lets say there are number of department in any organisation. Adf pipeline of one department can only be visible to respective department , not to other department. For ex. if the adf pipeline created by Data ETL related Department then…
asked 2024-05-22T06:43:06.7966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(262.40000000000003, 34%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERP%3C/text%3E%3C/svg%3E)
Rachit Pandya
0
Reputation points
commented 2024-05-22T08:30:46.78+00:00
Rachit Pandya
0
Reputation points
1 answer
How to establish connections of azure data factory connectivity with AWS dynamo DB
How to establish connections of azure data factory connectivity with AWS dynamo DB - provide a Micrsoft Documentation so that we can follow that to establish the connectivity in between the Azure as well as the AWS platform.
asked 2024-05-22T07:10:18.7466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 18%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAB%3C/text%3E%3C/svg%3E)
Aratrika Bose
0
Reputation points
edited an answer 2024-05-22T08:27:50.1+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
10,715
Reputation points Microsoft Vendor
1 answer
Should ADF rerun range cover the whole window?
Hi Team, I'm trying to understand the rerun range in tumbling window triggers. When I want to retry a window, do I need to make the rerun range covers the whole window or it only needs to cover a partial window. Let's say we have three consecutive failed…
asked 2024-05-22T07:17:35.52+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 18%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EZE%3C/text%3E%3C/svg%3E)
Zhengxiao (Edward) Wang
0
Reputation points Microsoft Employee
edited an answer 2024-05-22T08:23:01.32+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
6,630
Reputation points Microsoft Vendor
0 answers
Posting data to a business central table with Azure Data Factory
Hello I am getting this error Job failed due to reason: Error response from server: Some({"error":{"code":"Internal_EntityWithSameKeyExists","message":"The record in table PurchaseLinesInvoicesTempTable…
asked 2024-05-20T13:54:31.2333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(73.60000000000001, 28.000000000000004%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKO%3C/text%3E%3C/svg%3E)
Kayode Ogidan
60
Reputation points
commented 2024-05-22T07:59:03.11+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 48%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EH%3C/text%3E%3C/svg%3E)
Harishga
4,160
Reputation points Microsoft Vendor
1 answer
Slow Data Pipeline Performance - ADF Data Flow to Azure SQL Database
I'm having some performance issues with an Azure Data Factory (ADF) data flow pipeline. The pipeline is designed to move data from a Parquet file and insert/update it into an Azure SQL database table. The data volume is moderate, with batches of 50,000…
asked 2024-05-20T11:51:35.8133333+00:00
Murlidhar Patil
1
Reputation point
commented 2024-05-22T07:55:56.8266667+00:00
Harishga
4,160
Reputation points Microsoft Vendor
2 answers
Need help in understanding times in ADF's tumbling window trigger
Hi Team, I'm confused by many times in tumbling window trigger. We have at least 4 times for tumbling window trigger. @trigger().outputs.windowStartTime @trigger().outputs.windowEndTime @trigger().scheduledTime @trigger().startTime Besides, there…
asked 2024-05-20T10:01:15.2+00:00
Zhengxiao (Edward) Wang
0
Reputation points Microsoft Employee
commented 2024-05-22T06:38:21.54+00:00
Harishga
4,160
Reputation points Microsoft Vendor
1 answer
Azure Synapse Deployment Failure with "Exception has been thrown by the target of an invocation."
0 Failure to deploy azure synapse pipelines with below error: Set-AzSynapsePipeline : Exception has been thrown by the target of an invocation. 2024-05-08T08:29:52.7382260Z [31;1m[31;1m+ … $deploy = Set-AzSynapsePipeline -WorkspaceName $WorkspaceName…
asked 2024-05-22T05:37:02.1733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 55.00000000000001%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
Sree
0
Reputation points
answered 2024-05-22T06:34:17.7133333+00:00
hossein jalilian
4,185
Reputation points
1 answer
dataflow filter expression
hello, can you help me with this filter expression when I am loading data? my goal is: if strategy is one of these two, i.e. stra1, stra2 then make sure the party is one of these, i.e. party=p1 or party=p2 or party = p3 if strategy is not stra1 or stra2…
asked 2024-05-22T05:53:55.4533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(262.40000000000003, 97%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
arkiboys
9,646
Reputation points
answered 2024-05-22T06:29:50.7333333+00:00
hossein jalilian
4,185
Reputation points
2 answers
One of the answers was accepted by the question author.
Data Factory auto create table in Copy activity doesn't seem to work, or isn't very useful
Hi there I'm trying to create copy activities where the source table is replicated into the Sink database, and the table is created according to what is in the Source. I know there is the "Auto create table" option when making the copy…
asked 2021-07-12T08:22:16.947+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(140.8, 53%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECV%3C/text%3E%3C/svg%3E)
Cornel Verster
36
Reputation points
commented 2024-05-22T06:25:52.71+00:00
Smaran Thoomu
10,715
Reputation points Microsoft Vendor
2 answers
Datafactory insertion leads PluginNotRegisteredException,Message=Invalid type 'SalesforceV2' is provided in 'linkedServices
Getting Plug in error while inserting data (Dataset from Cloud Salesforce ) to SQL database (Onprimissive). Unfortunately it leads the…
asked 2024-02-22T14:34:04.67+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 96%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECV%3C/text%3E%3C/svg%3E)
Chandran, Vinitha
40
Reputation points
commented 2024-05-22T05:57:08.0633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 15%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERA%3C/text%3E%3C/svg%3E)
Rohan Adwant
0
Reputation points
2 answers
One of the answers was accepted by the question author.
How to fix backups files extremely slow
We are having an issues in azure cloud, the data backup stuck up in last 5 days still its showing "Data transfer completed and all backed up data in the cloud. Verifying data integrity.." Other servers have been registered to the same vault and…
asked 2024-05-14T06:46:52.11+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 72%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESS%3C/text%3E%3C/svg%3E)
Solairaj Sundarraj
20
Reputation points
accepted 2024-05-22T05:37:30.3766667+00:00
Solairaj Sundarraj
20
Reputation points
1 answer
One of the answers was accepted by the question author.
How can i create a linked service in ADF for Sharepoint online?
I want to extract files from sharepoint to ADL by only using ADF. I followed few steps Step1: Azure Active Directory -> Registered new app -> created new secret key I have the Tenant ID, ClientID(App ID), Secret Key Step2: Sharepoint online ->…
asked 2024-04-27T21:58:38.2233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 73%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESR%3C/text%3E%3C/svg%3E)
Siddartha Reddy Jammula
20
Reputation points
commented 2024-05-22T05:08:28.6733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 64%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENR%3C/text%3E%3C/svg%3E)
Nehruji R
2,891
Reputation points Microsoft Vendor
1 answer
AZURE DATA FACTORY STORED PROCEDURE WITH POSTGRESQL SERVER
Dear Microsoft Support, I have created a stored procedure in a PostgreSQL server and established a linked service for that server in Azure Data Factory (ADF). My goal is to execute this stored procedure within a pipeline in ADF. However, when I attempt…
asked 2024-05-21T05:42:27.17+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(259.20000000000005, 17%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESV%3C/text%3E%3C/svg%3E)
Sai Varma Eswararaju
0
Reputation points
commented 2024-05-22T05:06:12.4833333+00:00
Smaran Thoomu
10,715
Reputation points Microsoft Vendor
0 answers
I am encountering an error when trying to debug a data flow in Azure Data Factory that connects to an Azure SQL Database. My linked service and dataset connections to the Azure SQL Database are working fine, and I can preview data without any issues.
I am encountering an error when trying to debug a data flow in Azure Data Factory that connects to an Azure SQL Database. My linked service and dataset connections to the Azure SQL Database are working fine, and I can preview data without any issues.…
asked 2024-05-21T02:20:06.9066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(92.8, 82%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERB%3C/text%3E%3C/svg%3E)
ram boyapati
5
Reputation points
commented 2024-05-22T04:37:25.15+00:00
phemanth
6,630
Reputation points Microsoft Vendor
1 answer
How to resolve invalid property name error in ADF copy activity?
ErrorCode=AdlsGen2OperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=ADLS Gen2 operation failed for: Operation returned an invalid status code 'BadRequest'. Account: 'rmsdatalakestracc1mqa'. FileSystem:…
asked 2024-05-19T06:23:54.9633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 35%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMY%3C/text%3E%3C/svg%3E)
Mansi Yadav
0
Reputation points
edited an answer 2024-05-22T01:27:25.0833333+00:00
Mansi Yadav
0
Reputation points
1 answer
list of pipeline names
How do I easily get a downloaded list of all pipeline names?
asked 2024-05-17T20:33:42.42+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 34%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EFD%3C/text%3E%3C/svg%3E)
Feuerborn, Diana
0
Reputation points
commented 2024-05-21T21:01:32.3866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
BhargavaGunnam-MSFT
27,326
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Why there is no upsert key for cosmos Db container
Hello, Using the copy activity- Azure SQL to cosmos DB there is no upsert key. Is this expected behavior or this is a product limitation? Appriciate your help!
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(19.2, 26%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
1 answer
Error: 11402 -The value of the property '' is invalid: 'The remote name could not be resolved:
I am setting up a Linked service for ODATA in ADF, for IntegrationRuntime I am selecting "AutoResolveIntegrationRuntime" but connection to ODATA resource is failing: The value of the property '' is invalid: 'The remote name could not be…
asked 2024-05-08T05:48:44.8233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 3%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBA%3C/text%3E%3C/svg%3E)
Bhat, Asif
0
Reputation points
answered 2024-05-21T20:26:36.2933333+00:00
BhargavaGunnam-MSFT
27,326
Reputation points Microsoft Employee
1 answer
Failed to save PrivateLinkServiceMSSQL. Error: Invalid resource request. Resource type: 'ManagedPrivateEndpoint', Resource name: 'PrivateLinkServiceMSSQL' 'Error: Invalid payload'.
Hello Team, I have created a below mentioned points in Azure Resources: standard load balancer :Test_MS_SQL load balancer resources: health probe:MSQL load balancer rule : OnPremisesSQL While creating a private Link service, I am getting error…
asked 2024-05-20T13:21:35.56+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(0, 13%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERK%3C/text%3E%3C/svg%3E)
Rohit Kulkarni
686
Reputation points
edited a comment 2024-05-21T18:53:14.9433333+00:00
BhargavaGunnam-MSFT
27,326
Reputation points Microsoft Employee