Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,545 questions
1 answer
Linked services issue which connecting to OnPrim DB2
I am trying to connect to Linked services for OnPrim DB2 using a VM as middle layer, I have checked the telnet connectivity everything is connected but when I used the selfhosted and tried to connect then I get the attached error, Can any one help to…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(278.4, 67%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
1 answer
How to parameterize pipeline concurrency (number of parallel runs) in Azure Data Factory
I have two pipelines: parent pipeline to read configuration file (lookup activity), filter it with filter activity and pass the resulting array (e.g. 50 items) to ForEach activity to run child pipeline to perform per-item work in a predefined number of…
asked 2024-04-25T11:49:35.2833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(275.2, 86%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Azurin4848484
0
Reputation points
commented 2024-04-27T05:14:56.8566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
5,735
Reputation points Microsoft Vendor
1 answer
Automating Subscription Creation with Azure Functions
In modern cloud environments, automating routine tasks is key to efficiency and scalability. Azure Functions, a serverless compute service, provides a powerful platform for automating tasks within the Azure ecosystem. One common use case for Azure…
asked 2024-04-26T12:39:54.6633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(198.4, 40%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
Mehul Movadiya
0
Reputation points
answered 2024-04-27T03:59:46.91+00:00
Abhishek Tripathi
0
Reputation points
1 answer
Lookup activity output into a CSV file in ADLS
I'm using lookup activity to read all tables from a Oracle SQL Developer application for a particular schema. The query is SELECT TABLE_NAME FROM all_tables WHERE owner = 'Schema_Name' Here the scheme name is dynamic taking value from a variable. Now, my…
asked 2024-04-26T16:14:09.8366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(131.20000000000002, 60%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBB%3C/text%3E%3C/svg%3E)
Bhajanthri Bala Sujay
0
Reputation points
answered 2024-04-27T03:58:04.4933333+00:00
Abhishek Tripathi
0
Reputation points
0 answers
How to read nested xlsx files in Azure Data Factory
How we can read these types of nested xlsx files in ADF
asked 2024-04-27T03:51:29.27+00:00
Abhishek Tripathi
0
Reputation points
edited the question 2024-04-27T03:53:15.94+00:00
Abhishek Tripathi
0
Reputation points
1 answer
One of the answers was accepted by the question author.
Dynamic ADF Data Transformation Challenge
I'm presently engaged in an ADF data transformation task utilizing the .NET SDK, employing Data Flow for the transformation process. Within a CSV file containing three columns—First Name, Last Name, and Age—I aim to merge First Name and Last Name into a…
asked 2024-04-26T17:23:54.1566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(41.6, 5%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EV2%3C/text%3E%3C/svg%3E)
Vikranth-2626
100
Reputation points
accepted 2024-04-27T02:18:41.89+00:00
Vikranth-2626
100
Reputation points
0 answers
ADF pipeline with upsert throwing error "The data type text cannot be used as an operand to the UNION, INTERSECT or EXCEPT operators because it is not comparable"
ADF pipeline writing data from AZURE sql to on prem SQL Server with upsert operation. TEXT fields are part of target system, Even though TEXT fields are not selected in select query and are not mandatory fields still throwing error. I cannot change…
asked 2024-04-27T01:42:51.18+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(102.4, 57.00000000000001%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESD%3C/text%3E%3C/svg%3E)
Shantanu Dekapurwar
0
Reputation points
asked 2024-04-27T01:42:51.18+00:00
Shantanu Dekapurwar
0
Reputation points
0 answers
Can Azure Data Factory SAP ODP connector read from cluster tables like PCL2 & PCL4 ?
Does Azure Data Factory connector "SAP CDC" uses ODP framework of SAP? Can SAP CDC connector read from SAP Cluster tables from ECC system specifically PCL2 & PCL4 cluster tables?
asked 2024-04-26T22:54:01.79+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(118.4, 86%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESV%3C/text%3E%3C/svg%3E)
Sreedhar Vecireddy [svecireddy]
0
Reputation points
asked 2024-04-26T22:54:01.79+00:00
Sreedhar Vecireddy [svecireddy]
0
Reputation points
0 answers
Data Factory Pipeline, I want to Publish my trigger that I have created through a Http request, is this at all possible.
Hello, I have created a Data Factory Pipeline with help from the below links using a HTTP Rest API call. https://learn.microsoft.com/en-us/rest/api/datafactory/pipelines/create-or-update?view=rest-datafactory-2018-06-01&tabs=HTTP#code-try-0 …
asked 2024-04-24T11:28:52.42+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 78%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECM%3C/text%3E%3C/svg%3E)
Clinton Morgan
0
Reputation points
edited a comment 2024-04-26T20:07:22.4733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
BhargavaGunnam-MSFT
26,136
Reputation points Microsoft Employee
1 answer
Azure Data Factory Old Security->Credential not clear after deploy
Hi, I was testing with the UserAssignedManagedIdentity credential and I created 2 credentials with the same UserAssignedManagedIdentity but different names(UserAssignedManagedIdentity & credentialUAMI). I ran my pipeline to deploy using the dev…
asked 2024-04-25T12:54:35.7966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(112.00000000000001, 73%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELS%3C/text%3E%3C/svg%3E)
Lee, Steven Hong Sing
1
Reputation point
edited a comment 2024-04-26T18:43:03.62+00:00
BhargavaGunnam-MSFT
26,136
Reputation points Microsoft Employee
0 answers
Will the ADF HubSpot Dataset updated to use the HubSpot v3 Owners API?
Is there any plan to update the ADF HubSpot Dataset to use the HubSpot v3 Owners API before the v2 Owners API gets sunset on August 30, 2024? Currently, the ADF HubSpot Dataset relies on the v2 Owners API, but HubSpot has announced the end of life for…
Azure
Azure
A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.
945 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,545 questions
asked 2024-04-26T18:39:11.2566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(272, 7.000000000000001%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBJ%3C/text%3E%3C/svg%3E)
Brice Johnson
0
Reputation points
asked 2024-04-26T18:39:11.2566667+00:00
Brice Johnson
0
Reputation points
0 answers
How to store set variable json array output into csv file in data factory
I used the Lookup activity under for each loop and used Store Procedure which gave me some record as an output if the condition is fails So that output I have stored into the Set Variable and then those values I need to store into csv file whose file…
asked 2024-04-26T18:01:30.62+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 61%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMB%3C/text%3E%3C/svg%3E)
Mohit Bokade
0
Reputation points
edited the question 2024-04-26T18:10:50.99+00:00
Mohit Bokade
0
Reputation points
1 answer
How to translate database content with Azure Translator by ADF or Synapse notebook?
There is an Azure Database table. Some of columns need to be translated from one language to another into additional columns. Such as from English to Spanish, or Portuguese to English, etc. I am exploring how I can use ADF or Synapse notebook to…
asked 2024-04-22T15:17:46.4133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 49%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ED%3C/text%3E%3C/svg%3E)
Dataholic
80
Reputation points
commented 2024-04-26T16:31:02.7966667+00:00
BhargavaGunnam-MSFT
26,136
Reputation points Microsoft Employee
0 answers
How to maintain same folder structure as source while sink the processed file
I have a requirement to process the JSON to parquet on daily basis. I have folder A,B,C needs to sink the file to another container with same structure as A,B,C for example if I'm processing a file from folder A it should sink to output container folder…
asked 2024-04-26T08:36:30.47+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(112.00000000000001, 11%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVS%3C/text%3E%3C/svg%3E)
Venkatesh Srinivasan
0
Reputation points
edited a comment 2024-04-26T16:21:27.6266667+00:00
Venkatesh Srinivasan
0
Reputation points
1 answer
Error "The DeveloperEmail field is required." when setting up Graph Data Connect
Hello, I'm trying to set up Graph Data Connection using this tutorial: https://github.com/microsoftgraph/dataconnect-solutions/tree/main/solutions/ona/PreRequisites Unfortunately, I cannot finalize the process, because when I click on "Create",…
Microsoft Graph
Microsoft Graph
A Microsoft programmability model that exposes REST APIs and client libraries to access data on Microsoft 365 services.
10,592 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,545 questions
Office Development
Office Development
Office: A suite of Microsoft productivity software that supports common business tasks, including word processing, email, presentations, and data management and analysis.Development: The process of researching, productizing, and refining new or existing technologies.
3,489 questions
asked 2024-04-26T08:56:28.39+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 96%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJV%3C/text%3E%3C/svg%3E)
Jeroen Vrijkorte
26
Reputation points
commented 2024-04-26T15:15:15.19+00:00
Jeroen Vrijkorte
26
Reputation points
1 answer
One of the answers was accepted by the question author.
How to split rows and columns by using ADF?
How can I retrieve data from the following table by using ADF lookup activity and want to get the result as below? Thanks
asked 2024-04-15T15:57:51.3866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(214.4, 32%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EZY%3C/text%3E%3C/svg%3E)
Zhu, Yueli YZ [NC]
235
Reputation points
commented 2024-04-26T14:50:54.6133333+00:00
Zhu, Yueli YZ [NC]
235
Reputation points
1 answer
Azure Data Factory - Copy Data action from SQLServer to Oracle gives an error like "Character, decimal, and binary parameters cannot have a precision of zero"
I am using Copy data in order to move files from Sql server to Oracle, I have cases where in Sql server side , columns have data type of nvarchar and can have null value or empty string '' for the cases where I have empty string - it can't be inserted…
asked 2024-03-08T12:45:10.03+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(48, 37%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Stergiou Theofilos
0
Reputation points
commented 2024-04-26T13:55:38.84+00:00
Stergiou Theofilos
0
Reputation points
1 answer
ADF Lookup activity not returning correct datetime value
A pipeline uses a lookup activity and runs a script against Snowflake, returns first row of data and we are using it in a update statement. Running the lookup to get MAX(load_date) from a snowflake table. In Snowflake it is "2024-04-25 17:26:01.548…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 54%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EH%3C/text%3E%3C/svg%3E)
3 answers
One of the answers was accepted by the question author.
Converting UTC to EST with daylight saving
I am trying to convert UTC to EST with following formula but its not taking daylight saving time to consideration. toString(fromUTC(currentUTC(),'EST'),'yyyyMMddHHmmss') it shows time one hour behind EST. Is there a workaround?
asked 2024-04-02T14:40:10.5866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(144, 38%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJJ%3C/text%3E%3C/svg%3E)
Jiby Jacob
40
Reputation points
accepted 2024-04-26T12:26:44.5233333+00:00
Jiby Jacob
40
Reputation points
0 answers
In ADF, I am not able to merge data from multiple streams with different columns with different data types
Hello, I am not able to merge data from multiple streams in ADF DATAFLOWS. Stream 1) flow let - only 1 column. Stream 2) flow let - only 1 column. Stream 3) source columns (54 columns) Stream 4) regular…
asked 2024-04-24T16:32:40.4033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(73.60000000000001, 9%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENA%3C/text%3E%3C/svg%3E)
neeraj aitha
0
Reputation points
commented 2024-04-26T10:55:46.11+00:00
neeraj aitha
0
Reputation points