Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,405 questions
1 answer
In ADF, I am not able to merge data from multiple streams with different columns with different data types
Hello, I am not able to merge data from multiple streams in ADF DATAFLOWS. Stream 1) flow let - only 1 column. Stream 2) flow let - only 1 column. Stream 3) source columns (54 columns) Stream 4) regular…
asked 2024-04-24T16:32:40.4033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(73.60000000000001, 9%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENA%3C/text%3E%3C/svg%3E)
neeraj aitha
0
Reputation points
edited a comment 2024-05-08T05:31:05.46+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
5,990
Reputation points Microsoft Vendor
0 answers
Error from ADF pipeline loading data into Dynamics 365 table
I have an Azure pipeline that is copying data from a table in an azure sql db to a table in Dynamics 365 CE. The D365 has MFA attached so I am using an azure service principal in the linked service connection for the pipeline. The Service principal is…
asked 2024-05-08T05:28:41.8833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(80, 10%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EWB%3C/text%3E%3C/svg%3E)
Wayne Bartkowski
656
Reputation points
asked 2024-05-08T05:28:41.8833333+00:00
Wayne Bartkowski
656
Reputation points
0 answers
Get error invalid with correct url in copy activity
Hi, nice to meet you. Currently, I am working with SAP odata API, I am using url https://xxxxxxxxxxxxxxxx.cloud.sap/sap/byd/odata/analytics/ds/Fingxxx.svc/Fingxx?$select=xxx,xxx,xx,,xxx,xxx&$format=json&$filter=(C_Setofbks eq 'xxxxx') and…
asked 2024-05-08T04:39:51.0966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(169.60000000000002, 2%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPH%3C/text%3E%3C/svg%3E)
Pham Huynh Tan Dat
0
Reputation points
asked 2024-05-08T04:39:51.0966667+00:00
Pham Huynh Tan Dat
0
Reputation points
0 answers
ADF Data Flow - CDC - Schema Drift not working
Hi, I have setup ADF pipeline for data sync of tables between Azure SQL Database to SQL Managed Instance databases. I am using Data Flow and CDC to track changes and sink activity to replicate it to target. Since there are multiple tables involved, I am…
asked 2024-05-08T04:10:23.8866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 83%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Ashwani SRIVASTAVA
40
Reputation points
edited the question 2024-05-08T04:34:34.1966667+00:00
Ashwani SRIVASTAVA
40
Reputation points
0 answers
ADF - CDC - enable and disable CDC tables during deployment
Hi, I have setup ADF pipeline for data sync of tables between Azure SQL Database to SQL Managed Instance databases. I am using Data Flow and CDC to track changes and sink activity to replicate it to target. However, when we deploy any table changes (add…
asked 2024-05-08T04:29:23.9233333+00:00
Ashwani SRIVASTAVA
40
Reputation points
asked 2024-05-08T04:29:23.9233333+00:00
Ashwani SRIVASTAVA
40
Reputation points
1 answer
How to add Azure SQL Database as linked service to Azure Data Factory
I keep running into errors adding Azure SQL as a linked service to Data Factory. I'm new to Azure so please bear with me. Following all the online guides, I've done all of the following: On the SQL Server side: Under Microsoft Entra ID, enabled…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(316.8, 44%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
0 answers
How can we connect Blackboard DDA (a SaaS using PostgreSQL to store data ) via Azure Synapse analytics using SSL certificate for authentication in SHIR for Private Vnet enabled networks.
Blackboard DDA is a SaaS application which uses PostgreSQL as DB to store data. The DB uses a SSL certificate key to authenticate and we have private Vnet enabled. How to create a linked service for this DB and where do we store certificate in SHIR. Are…
asked 2024-05-03T05:41:52.49+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(9.6, 47%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDJ%3C/text%3E%3C/svg%3E)
Dravya Jain
0
Reputation points
commented 2024-05-08T04:10:27.7266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 48%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EH%3C/text%3E%3C/svg%3E)
Harishga
3,580
Reputation points Microsoft Vendor
2 answers
How to copy the data from Amazon S3 to Azure SQL database incrementally.
Hello team, I wanted to wanted to copy data from Amazon S3(JSON format) to Azure SQL database. I wanted to learn the mechanism for incrementally updating the data from Amazon S3 to Azure SQL. Also, the environment has only SHIR deployed, so data flows…
asked 2024-05-01T20:06:01.4533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(131.20000000000002, 0%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESC%3C/text%3E%3C/svg%3E)
Shweta Chavan
40
Reputation points
commented 2024-05-08T00:41:30.6+00:00
Shweta Chavan
40
Reputation points
0 answers
Azure Data factory - jwt token using Key Vault certificate
There doesn't appear to be a lot of help on the internet for this so hopefully someone has managed to achieve this! We need to create a JWT token to pass on for authentication to Salesforce. A certificate has been created and placed within our Key Vault.…
asked 2024-05-02T23:09:06.92+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(96, 30%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERA%3C/text%3E%3C/svg%3E)
Ryan Abbey
0
Reputation points
commented 2024-05-07T23:04:18.92+00:00
Ryan Abbey
0
Reputation points
0 answers
How to conect to VPN using azure data factory linked service
I have a client who don't want to install the SHIR in their SQL Server, so the client give me acess to a VPN connection and this informations: IP SQL Port Username Password Is there anything I can do to connect in their server using the linked…
asked 2024-05-02T18:09:00.5166667+00:00
Antônio Farias
20
Reputation points
commented 2024-05-07T22:40:17.1133333+00:00
Antônio Farias
20
Reputation points
1 answer
ADF Rest API Copy Tool - How to configure pagination using partial URL from response.
Trying to configure ADF pipeline to read from a REST API and write to SQL db using pagination. I'm able to read data from the API and write to SQL, but only the first 200 records (due to default pagination from vendor) Here is a sample response showing…
asked 2024-04-30T19:03:34.55+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 77%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERW%3C/text%3E%3C/svg%3E)
Ryan Wiedman
0
Reputation points
commented 2024-05-07T21:13:10.4433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
BhargavaGunnam-MSFT
26,316
Reputation points Microsoft Employee
0 answers
ADF dataflows are creating temporary folders / files and fails while trying to rename the file at SFTP
Hi, We're facing the issue while using SFTP sink via a Dataflow. We wish to generate output files using Dataflow at the SFTP location but dataflow is creating temp folders / files and in the end, it tries to rename the file but fails due to reason: Spark…
asked 2024-04-23T14:11:30.7633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 93%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAV%3C/text%3E%3C/svg%3E)
Aditya Verma
0
Reputation points
commented 2024-05-07T21:12:15.6133333+00:00
BhargavaGunnam-MSFT
26,316
Reputation points Microsoft Employee
0 answers
I have a control table in ADF pipeline with total of 36tables in it. I want to encrypt and decrypt storage account target file data columns. How can i get this done? Please Note: Encryption will not be done in SQL Server
I am about to begin an SFTP process and need to encrypt some tables columns in a control table in Azure Data Factory. How best can this be done.
asked 2024-05-07T21:00:19.3266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(259.20000000000005, 22%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGM%3C/text%3E%3C/svg%3E)
Godswill Mbata
0
Reputation points
asked 2024-05-07T21:00:19.3266667+00:00
Godswill Mbata
0
Reputation points
0 answers
COPY ACTIVITY FROM SOURCE LIKED REST TO DESTINATION LOINED REST SERVICE
Want to copy from source table to destination table in the same log analytics workspace. Using Azure Data Factory. My configuration: Source Linked REST Service to read data from Log Analytics workspace w1 table t1 Destination Linked REST service to…
asked 2024-05-02T23:31:22+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(144, 39%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERS%3C/text%3E%3C/svg%3E)
Rishabh Srivastava
60
Reputation points Microsoft Employee
commented 2024-05-07T20:51:18.4333333+00:00
Rishabh Srivastava
60
Reputation points Microsoft Employee
0 answers
Custom libraries (wheel) for ADF Databricks Python activity run on serverless compute
I want to be able to execute Python scripts (via Databricks Python) from Azure Data Factory using serverless compute. Serverless compute does not support cluster level (compute scoped) libraries. In databricks workflows, it is being done as…
asked 2024-05-01T12:30:52.2366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 31%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKP%3C/text%3E%3C/svg%3E)
Krzysztof Przysowa
0
Reputation points
commented 2024-05-07T17:06:59.4733333+00:00
Krzysztof Przysowa
0
Reputation points
1 answer
One of the answers was accepted by the question author.
Automatic updates on source systems
Hi friends, I have a requirement that the business has 12 different upstream sources SAP, CRM, Oracle HCM, SqlServer, PostgreSQL, and multiple other open-source databases and custom applications, etc. and every system has its master record, and just…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 15%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
1 answer
ADF Lookup activity not returning correct datetime value
A pipeline uses a lookup activity and runs a script against Snowflake, returns first row of data and we are using it in a update statement. Running the lookup to get MAX(load_date) from a snowflake table. In Snowflake it is "2024-04-25 17:26:01.548…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 54%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EH%3C/text%3E%3C/svg%3E)
0 answers
adf data flow acitivity stuck in queued status when the pipelines run
when I trigger the pipelines, it got stuck in the data flow activity as queued status
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 93%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EB%3C/text%3E%3C/svg%3E)
0 answers
ADF dataflow activity fails with large record sets
I have dataflow activity that retrieves data from an Azure MySQL database and invokes an address API using external call transformation to update the database with standardized addresses. However, when dealing with a large number of records…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(236.8, 41%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAA%3C/text%3E%3C/svg%3E)
0 answers
How to post data with REST API dataset as output to Business central
Hello, I am using the copy activity to copy data from a csv containing invoices to a temporary table in Business Central. I am getting this error on the sink side, I set up app registrations and for the Business central. I am using a web activity to get…
asked 2024-05-07T15:21:14.9333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(73.60000000000001, 28.000000000000004%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKO%3C/text%3E%3C/svg%3E)
Kayode Ogidan
0
Reputation points
asked 2024-05-07T15:21:14.9333333+00:00
Kayode Ogidan
0
Reputation points