9,269 questions with Azure Data Factory tags

Sort by: Updated
0 answers

Azure Data Factory Fuzzy logic implementation for multiple tables in the SQL DB

I am trying to build a solution using Azure Data Factory to implement the Fuzzy Logic for two data source which is stored in Synapse SQL DB. for i.e. Table 1 and Table 2 which has identical structures. I am trying to use ADF to build a pipelines using…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,269 questions
asked 2024-03-19T03:46:26.49+00:00
MG 40 Reputation points
1 answer One of the answers was accepted by the question author.

AdlsGen2OperationFailed - Error

Hi, I could migrate a single table from the MS SQL Server Database OnPremises to Azure Data Lake Storage in the data factory and the pipeline is running successfully. However, when I try to migrate multiple tables from MS SQL Server Database OnPremises…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,269 questions
asked 2024-03-18T17:25:18.2033333+00:00
Sivakumar Rajabather 20 Reputation points
commented 2024-03-19T03:44:23.28+00:00
Sivakumar Rajabather 20 Reputation points
1 answer One of the answers was accepted by the question author.

Unable to preview the file in Azure Data Factory - Failure

Hi, I have created a pipeline in Azure Data Factory to migrate a table from MS SQL Server and loaded the table in Azure Data Factory. But unable to preview the table and getting the below error. Please look into this and help me to get this fixed.…

SQL Server
SQL Server
A family of Microsoft relational database management and analysis systems for e-commerce, line-of-business, and data warehousing solutions.
12,405 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,269 questions
asked 2024-03-16T15:57:23.7833333+00:00
Sivakumar Rajabather 20 Reputation points
commented 2024-03-19T03:40:50.2333333+00:00
Sivakumar Rajabather 20 Reputation points
1 answer

Grand finserv customer service Toll free Number - ((((7387901504)))) (((((( 8766430231 *** Call +Naw

Calcnv and hh h to hai d hd

Microsoft Graph
Microsoft Graph
A Microsoft programmability model that exposes REST APIs and client libraries to access data on Microsoft 365 services.
10,233 questions
SQL Server
SQL Server
A family of Microsoft relational database management and analysis systems for e-commerce, line-of-business, and data warehousing solutions.
12,405 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,269 questions
Active Directory
Active Directory
A set of directory-based technologies included in Windows Server.
5,666 questions
Not Monitored
Not Monitored
Tag not monitored by Microsoft.
35,365 questions
asked 2024-03-19T03:30:36.3566667+00:00
Vxbdnd Bdbd 0 Reputation points
answered 2024-03-19T03:30:45.5766667+00:00
Vxbdnd Bdbd 0 Reputation points
1 answer

Grand finserv customer service Toll free Number - ((((7387901504)))) (((((( 8766430231 *** Call +Naw

Dvvx CNBC awaaz ty for the winter y

SQL Server
SQL Server
A family of Microsoft relational database management and analysis systems for e-commerce, line-of-business, and data warehousing solutions.
12,405 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,269 questions
Microsoft Entra ID
Microsoft Entra ID
A Microsoft Entra identity service that provides identity management and access control capabilities. Replaces Azure Active Directory.
18,954 questions
asked 2024-03-19T03:24:16.1866667+00:00
Ragdhd ncnc 0 Reputation points
answered 2024-03-19T03:24:35.87+00:00
Ragdhd ncnc 0 Reputation points
1 answer

Ocean Credit cusTomer care service Tollfree Number)+91)9861431692(call)9861431692/ call

Get the best way I get to know about

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,269 questions
Exchange Server Management
Exchange Server Management
Exchange Server: A family of Microsoft client/server messaging and collaboration software.Management: The act or process of organizing, handling, directing or controlling something.
7,265 questions
asked 2024-03-19T03:03:16.15+00:00
Ajb Wush 0 Reputation points
answered 2024-03-19T03:03:29.7733333+00:00
Ajb Wush 0 Reputation points
1 answer One of the answers was accepted by the question author.

folder not showing in ADF dataset

I have created folders for my datasets, but when I search for a folder, nothing is found. The folders only show when I search for text that is contained in the name of the dataset. Do I need to flip a switch that will expose the folder names.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,269 questions
asked 2024-03-18T04:10:16.3866667+00:00
MissingLinq 20 Reputation points
accepted 2024-03-19T02:31:18.95+00:00
MissingLinq 20 Reputation points
0 answers

ADF data flow json sometimes array and sometimes object

I think it relates to this issue [http://michaelcummings.net/mathoms/using-a-custom-jsonconverter-to-fix-bad-json-results/] Our procedure was like below: Create a Copy Activity in ADF pipeline: calling workday API, and sink response in Azure Data Lake…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,269 questions
asked 2024-03-13T00:55:18.3933333+00:00
Zhenqi Liu 0 Reputation points
commented 2024-03-19T01:46:20.4566667+00:00
Zhenqi Liu 0 Reputation points
0 answers

Filtering data for last 24 months in Mapping Data Flows

How can I filter data using Mapping Data Flows for the last 24 months, starting from the max date and going back? The date column is in the mm/dd/yyyy format.

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,305 questions
Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
2,581 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,350 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,212 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,269 questions
asked 2024-03-19T01:29:58.9966667+00:00
ADITYA SINGH 0 Reputation points
0 answers

How to execute java command on Azure Virtual Machine from Azure Data Factory

I have executable fat jar deployed on Azure Virtual Machine (LINUX). I want to run this jar using java command from Azure Data Factory. Basically, I want to create a pipeline to call several java commands, one at a time in pipeline. I tried to create…

Azure Virtual Machines
Azure Virtual Machines
An Azure service that is used to provision Windows and Linux virtual machines.
6,931 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,269 questions
asked 2024-03-19T01:29:08.2466667+00:00
satya 0 Reputation points
1 answer

How to move data from one table to another table within same workspace of LOG ANALYTICS using Azure Data Factory ?

How to move data from one table to another table within same workspace of LOG ANALYTICS using Azure Data Factory 2.from one table in log analytics workspace 1 to another table in log analytics 2 workspace using Azure Data Factory ?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,269 questions
asked 2024-03-14T00:37:19.5866667+00:00
Rishabh Srivastava 40 Reputation points Microsoft Employee
commented 2024-03-18T23:14:18.87+00:00
Rishabh Srivastava 40 Reputation points Microsoft Employee
0 answers

Copy Activity allows HTTP Post to get Excel but when debugging it fails

So I am trying to import an excel file from a website that uses a HTTP Post to pull the file. I have configured the linked service, dataset and then the Copy Activity. In the copy activity I am able to specifiy it use a Post method. When I click…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,269 questions
asked 2024-03-18T22:51:22.9666667+00:00
Steve Dunker 20 Reputation points
1 answer

I get the following error while Sinking data in Azure: ErrorCode: 'EndpointUnsupportedAccountFeatures'. Message: 'This endpoint does not support BlobStorageEvents or SoftDelete. Please disable these account features if you would like to use this endpoint.

When I try to select the file path under Linked service while sinking the data (ADLS Gen2), I get this error: "ErrorCode: 'EndpointUnsupportedAccountFeatures'. Message: 'This endpoint does not support BlobStorageEvents or SoftDelete. Please disable…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,305 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,269 questions
asked 2024-03-18T18:50:49.3766667+00:00
Priyal 0 Reputation points
answered 2024-03-18T21:46:22.67+00:00
KarishmaTiwari-MSFT 17,752 Reputation points Microsoft Employee
0 answers

Moving 10TB of SharePoint data to Azure blob storage (Contains files with 5GB+ in size)

Greetings, I hope this message finds you well. I'm currently working on a solution for transferring over 10TB of data from a "Preservation Hold Library" to "Azure Blob Storage" for archiving SharePoint online deleted files. Upon…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,269 questions
SharePoint
SharePoint
A group of Microsoft Products and technologies used for sharing and managing content, knowledge, and applications.
9,338 questions
Windows Server PowerShell
Windows Server PowerShell
Windows Server: A family of Microsoft server operating systems that support enterprise-level management, data storage, applications, and communications.PowerShell: A family of Microsoft task automation and configuration management frameworks consisting of a command-line shell and associated scripting language.
5,307 questions
asked 2024-03-12T07:06:19.6933333+00:00
Aman Varama 0 Reputation points
commented 2024-03-18T21:21:11.27+00:00
BhargavaGunnam-MSFT 24,726 Reputation points Microsoft Employee
0 answers

How to define pipeline return value in SetVariableActivity in .NET project?

Hi, I'm implementing SetVariableActivity in my .NET project. I wanted to define the return value pipeline but it is defined as DataFactoryElement<BinaryData> Value although it is an array of values. How can I define it in my project? I…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,269 questions
asked 2024-03-18T10:35:28.33+00:00
OGINTZ Marina 85 Reputation points
commented 2024-03-18T20:41:13.8566667+00:00
BhargavaGunnam-MSFT 24,726 Reputation points Microsoft Employee
0 answers

In ADF Copy Activity- Synapse Sink giving SQL DW Error : The Microsoft Distributed Transaction Coordinator (MS DTC) has cancelled the distributed transaction

When we are Copying data from ADLS to Azure Synapse Tables , we are getting following error since Friday . We are not able to write on Azure synapse from Azure pipeline Copy Activity. Below is the error.  Note : Earlier it was working fine , we are…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,212 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,269 questions
asked 2024-03-18T07:52:06.1533333+00:00
Kapoor, Sipra (S) 0 Reputation points
commented 2024-03-18T20:03:29.56+00:00
BhargavaGunnam-MSFT 24,726 Reputation points Microsoft Employee
0 answers

How to post multi-GB data file to REST API using Azure Data Factory?

Hello, I am currently working on a data ingestion pipeline which needs to process some larger files (up to 5GB) and post them to a REST API. The files are stored in Azure Blob Storage. The data cannot be sent in chunks and so needs to be sent whole,…

Azure
Azure
A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.
873 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,269 questions
asked 2024-03-18T12:42:07.5466667+00:00
JegorsM 0 Reputation points
edited the question 2024-03-18T19:36:39.8166667+00:00
KarishmaTiwari-MSFT 17,752 Reputation points Microsoft Employee
1 answer One of the answers was accepted by the question author.

Row count of parquet files

Hello, How can we get the row count of a parquet file? I want do a conditional copy activity based on the row count of the source data file. I tried with the lookup condition and used a if condition, based on output of lookup activity the if condition…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,269 questions
asked 2022-02-23T08:40:00.77+00:00
M. Adil 26 Reputation points
accepted 2024-03-18T18:13:19.1633333+00:00
M. Adil 26 Reputation points
3 answers

Can I connect directly to CosmosDB from ADF using the key vault?

I am trying to set up an azure key vault key to connect from Azure Data Factory directly to CosmosDB. I am following this link but it seems to be for a web app service. https://learn.microsoft.com/en-us/azure/cosmos-db/store-credentials-key-vault This…

Azure Key Vault
Azure Key Vault
An Azure service that is used to manage and protect cryptographic keys and other secrets used by cloud apps and services.
1,068 questions
Azure Cosmos DB
Azure Cosmos DB
An Azure NoSQL database service for app development.
1,384 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,269 questions
asked 2024-03-06T17:59:38.0433333+00:00
Andrew Hopkinson 6 Reputation points
commented 2024-03-18T16:34:04.8066667+00:00
Andrew Hopkinson 6 Reputation points
0 answers

ADF Dataflow - dervied column, Projection, Schema

Hi, I have observed that ADF dataflow derived column or any other transformations require projection to be available in previous transformations. For eg, I have a derived column after source. If I have to add a derived column 'xyz' as 'column1+5',…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,269 questions
asked 2024-03-18T15:54:42.3366667+00:00
Santhi Dhanuskodi 125 Reputation points
commented 2024-03-18T16:28:16.9766667+00:00
Suba Balaji 11,021 Reputation points