Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,807 questions
1 answer
Cash Assist customer%%-Service - care Number//.8207266523//-8207266523
Hi all I can you tell me how to explore new to the world of the world of the world
asked 2024-06-03T10:12:24.92+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(259.20000000000005, 8%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EYP%3C/text%3E%3C/svg%3E)
yifomih542@plughk.com
0
Reputation points
commented 2024-06-03T10:12:50.2933333+00:00
yifomih542@plughk.com
0
Reputation points
4 answers
Script Activity not supporting multi-instruction with new Snowflake connector
I have a pipeline, in Azure Data Factory that has an "Activity Script" that executes 3 instructions against Snowflake. The script was working fine until I have updated the Snowflake connection in order to remove the legacy Snowflake connection…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(208, 40%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ED%3C/text%3E%3C/svg%3E)
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(224.00000000000003, 39%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EL%3C/text%3E%3C/svg%3E)
1 answer
Updating Oracle on-premise table from a Synapse table
Hi, I need to update an Oracle on-premise table from a Synapse table by joining the Oracle table with the Synapse table avoiding to build a staging table on Oracle platform to copy the Synapse data. It is a problem to obtain the authorization from my…
asked 2024-06-01T12:00:58.09+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 41%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
pmscorca
837
Reputation points
commented 2024-06-03T09:26:51.3466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
11,045
Reputation points Microsoft Vendor
1 answer
How can i monitor azure data factory pipelines runtime logs in realtime with no latency?
Hi team, I'm trying to to retrieve Azure Data Factory pipeline execution logs in Realtime without any latency. If tried using log analytics, I see there is a latency in logs to get available. How can I get the Realtime pipeline details including IN…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 3%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
3 answers
How to update and delete a row using Azure Data Factory Change Data Capture
I am exploring Azure Data Factory CDC feature, and I am trying to perform the CDC from one SQL table to another SQL table, also the SQL table in the source has a primary key. Whenever a new row is added to the source table, it gets added to the…
asked 2024-04-28T17:40:36.5966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 25%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJ%3C/text%3E%3C/svg%3E)
jigsm
236
Reputation points
answered 2024-06-03T09:01:35.7833333+00:00
Pinaki Ghatak
2,405
Reputation points Microsoft Employee
3 answers
Lookup activity output into a CSV file in ADLS
I'm using lookup activity to read all tables from a Oracle SQL Developer application for a particular schema. The query is SELECT TABLE_NAME FROM all_tables WHERE owner = 'Schema_Name' Here the scheme name is dynamic taking value from a variable. Now, my…
asked 2024-04-26T16:14:09.8366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(131.20000000000002, 60%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBB%3C/text%3E%3C/svg%3E)
Bhajanthri Bala Sujay
0
Reputation points
answered 2024-06-03T08:56:32.6633333+00:00
Pinaki Ghatak
2,405
Reputation points Microsoft Employee
3 answers
Trying to write an expression for a filter activity in Azure Data Factory
I currently am using 3 filter activities where I am using the following condition @contains(item().name,'example') to filter out a certain group of files from the array obtained by get metadata This works fine but is still bringing back 5 files with the…
asked 2024-04-25T15:31:35.3566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(195.2, 2%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA0%3C/text%3E%3C/svg%3E)
azure-000
1
Reputation point
answered 2024-06-03T08:45:30.17+00:00
Pinaki Ghatak
2,405
Reputation points Microsoft Employee
4 answers
How do I copy data from a public sharepoint URL to Microsoft Blob Storage?
Hi everyone! I am new to using Azure services. My co-author at Stanford has uploaded several files (in csv format) to a public link. This link starts as: office365stanford-my.sharepoint.com/personal It has 40 files, ranging from a few 100 MB to 100 GB. I…
asked 2024-04-08T16:28:15.86+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 12%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENN%3C/text%3E%3C/svg%3E)
Namrata Narain
0
Reputation points
answered 2024-06-03T08:38:37.6966667+00:00
Pinaki Ghatak
2,405
Reputation points Microsoft Employee
0 answers
Azure Data Factory simple copy file and rename
Hello: I have a simple requirement. I have a couple of files in an Azure Blob folder (receiving folder) and I need to copy them, to another fold (Azure Blob folder) and rename them. This posting is really close to what I need to do: …
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(57.599999999999994, 38%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EK%3C/text%3E%3C/svg%3E)
1 answer
Delta to Parquet mapping data flow resulting in one empty partition of 2
Hi, I've been working on something, but I can't get it to work. I seem to have found the issue, but I"m not able to fix it. I've isolated in a single run which I'll explain below: I have a Delta table on ADLSv2 (based on one partition) that I want…
asked 2024-05-31T13:29:25.1766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(89.60000000000001, 8%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EEW%3C/text%3E%3C/svg%3E)
Erp, Wessel van
1
Reputation point
commented 2024-06-03T07:42:58.2333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 48%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EH%3C/text%3E%3C/svg%3E)
Harishga
4,485
Reputation points Microsoft Vendor
2 answers
Error using Azure Data Factory to Copy data (Using Upsert) from Azure Blob to Azure SQL database
Hi all, I keep getting this error when I perform an upsert (under copy activity) with this error code: Failure happened on 'Sink' side. 'Type=System.NullReferenceException,Message=Object reference not set to an instance of an…
asked 2024-05-23T23:34:36.5766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 0%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERL%3C/text%3E%3C/svg%3E)
Rome Lin
5
Reputation points
edited a comment 2024-06-03T07:15:38.6833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 25%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKY%3C/text%3E%3C/svg%3E)
Kevin Young
0
Reputation points
1 answer
ByPass custom logic executions in Azure Data factory copy activity
Hi, I am planning to do data refresh project using Azure data factory and like to disable the custom logics execution. is there any way I can do in ADF copy activity? Like…
asked 2024-05-31T08:32:26.37+00:00
Maniraj K V
20
Reputation points
commented 2024-06-03T06:40:33.92+00:00
Maniraj K V
20
Reputation points
1 answer
Data flow: toBase64 adds break lines after 76th character in specific ADF location
Hi! Recently we've got unexpected outputs from ADF pipelines with no prior changes to the source code. Suddenly, the toBase64 function in data flow activities began to add break lines (\r\n) after the 76th character for output values. During the issue…
asked 2024-04-26T08:01:34.9833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(144, 79%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAD%3C/text%3E%3C/svg%3E)
Andrii Danyl'tsiv
0
Reputation points
commented 2024-06-03T06:32:53.3+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 95%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESV%3C/text%3E%3C/svg%3E)
Sunil Vijayvargiya
0
Reputation points
1 answer
Azure Data Factory pagination using QueryParameters and body field
Hello everyone, I am currently working with a pagination system and facing a challenge with the URL and parameter concatenation. I am trying to append a specific string to the URL retrieved from the first page of an API response. Here's the setup: API…
asked 2024-05-09T22:23:30.69+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(108.80000000000001, 15%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMT%3C/text%3E%3C/svg%3E)
Max Teitelbaum
0
Reputation points
edited a comment 2024-06-03T06:05:27.3466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
6,885
Reputation points Microsoft Vendor
1 answer
How to ignore the records by applying an auditing filed column condition using ADF Data Flows
Hi All I am building a data transamination using mapping data flows ,I have a time stamp field Like TimeStampUpdated in the target table. I want to lockup historical data with incremental data transamination and ignore the records coming in the…
asked 2024-05-30T09:28:46.6066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 6%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVR%3C/text%3E%3C/svg%3E)
venkat rao
45
Reputation points
commented 2024-06-03T05:46:23.9533333+00:00
Smaran Thoomu
11,045
Reputation points Microsoft Vendor
1 answer
All ADF dataflow stages disappear after publish
Hello, I made a source for my Dataflow and added several stages to the existing one. I published and Dataflow showed an error with a new source. I tried to delete it, but could not delete it. I published again and all my stages disappeared. It is not…
asked 2024-05-29T11:22:44.82+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(275.2, 3%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDV%3C/text%3E%3C/svg%3E)
Darjuš Vasiukevič
20
Reputation points
commented 2024-06-03T05:44:52.27+00:00
Darjuš Vasiukevič
20
Reputation points
1 answer
Unable to create a cluster in Databricks getting quota exceeded error while everything is new
Azure Quota Exceeded Exception: Error code: QuotaExceeded, error message: Operation could not be completed as it results in exceeding approved standardDADSv5Family Cores quota. Additional details - Deployment Model: Resource Manager, Location:…
asked 2024-05-30T21:08:45.6433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 7.000000000000001%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESA%3C/text%3E%3C/svg%3E)
Sibbinath, Akhil
0
Reputation points
commented 2024-06-03T05:43:37.29+00:00
Smaran Thoomu
11,045
Reputation points Microsoft Vendor
0 answers
Mapping to polymorphic field in dynamics 365
Hello Team - I am loading data into the Incident table of a Dynamics 365 environment from an Azure SQL DB using the Azure Data Factory. the Incident table has lookups to the Account table on AccountId and Contacts via the ContactId. I have populated…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(80, 10%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EB%3C/text%3E%3C/svg%3E)
1 answer
Failed to get access token from your token endpoint. Error message: No MediaTypeFormatter is available to read an object of type 'OAuth2AccessToken' from content with media type 'text/html
In Azure Data Factory I am trying to connect to a REST API that uses OAuth 2.0 Authentication. When supplying Data Factory with all the necessary credentials and testing the connection I get the error: Failed to get access token from your token…
asked 2024-05-29T16:36:57.1933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 50%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJS%3C/text%3E%3C/svg%3E)
Josh Starita
0
Reputation points
edited a comment 2024-06-03T03:55:46.6866667+00:00
Harishga
4,485
Reputation points Microsoft Vendor
0 answers
Issue with Copy Dataverse data into Azure SQL using Synapse Link Template
I am experiencing issues using ADF I am trying to use the template "Copy Dataverse data into Azure SQL using Synapse Link" to move f&o data from datalake to Azure…
asked 2024-05-30T07:08:10.6766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(275.2, 20%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDO%3C/text%3E%3C/svg%3E)
Dipo Onibile
0
Reputation points
commented 2024-06-03T03:43:19.0666667+00:00
phemanth
6,885
Reputation points Microsoft Vendor