Long queue time for copy activity in synapse pipeline
This Synapse Analytics pipeline runs a simple SQL script that joins a table with its change table and outputs the results (less than 100 rows) as a parquet file in Azure Data Lake Storage Gen2. A Copy activity with the SQL script as a source and Storage…
rename table in synapse
Hi Team, rename table is not working in synapse RENAME OBJECT [dim].[test1] TO [dim].[test]; Parse error at line: 1, column: 49: Incorrect syntax near '.'.
Pass value from an array selectively to For Each
Hi, I have a parent pipeline that captures values in an array - ArrayFullLoadData. Currently ArrayFullLoadData have three values [A,B,C] and the same is passed to a ForEach with @pipeline.parameters.ArrayFullLoadData. Inside ForEach, values A, B,C are…
Synapse Scala | Read Cosmos DB Container data using managed identity
Hi Team, I tried reading Cosmos data using Managed Identity inside a Synapse Scala notebook, but it's not working as expected. Already provided access for synapse workspace by using below command New-AzCosmosDBSqlRoleAssignment -ResourceGroupName…
Error trying to validate storage account name while creating a new synapse workspace
I am unable to create a new data lake storage (gen2) account name in the basic tab for create Azure Synapse Workspace. The error I get is - " There was an error trying to validate storage account name. please try again ". The error message is…
Linked services and Key Vault
Hello, Just a question regarding link services, basically, our orgnisation has linked services with one datasbases from different business area, now we need to ingest data from different source, ideally from sharepoint with all the csv files for other…
Can you assist with a Synapse issue involving memory allocation in SQL DW Copy Command in Data Factory, without solely resorting to increasing DWU?
I'm encountering an issue while using the Copy Activity in Data Factory with the Polybase copy option. The error message I receive is as follows: “…..SQL DW Copy Command operation failed with error 'Unable to allocate 257707 KB for columnstore…
Backup/Restore and collation update in Azure Synapse
We have one requirement that to change the collation of Azure SQL serverless database in Azure Synapse Analytics to default collation. So I have 2 questions, 1. How to take backup and restore Azure SQL serverless database in synapse. 2. How to update…
Access Azure SQL DBs using Azure Synapse studio
I would like to understand how Azure Synapse Analytics can be used within the organization to facilitate data analysis and development. How will developers access Azure Datasets in a convenient way Is there a case for using Azure Synapse? I have a use…
ADF Copy Activity : Data load from ADLS (parquet files) to Azure Synapse Table Failing after running for 24 hours
ADF pipeline copy failed after running for 24 hours. Timeout is set to 7days. Copy Activity is trying to copy data from ADLS (Parquet files) to Azure Synapse Table on Production Environment . Could you please suggest what could be the possible issue for…
Synapse workspace deployment node-js supported versions
Hi, I would like to know the list of node.js versions supported for Synapse workspace deployment (Synapse Workspace Deployment@2 task). I can't seem to find any documentation describing the same.
Synapse Pipeline Error: Bad Request
Hello, We occasionally encounter the following error during the execution of several of our pipelines in Azure Synapse Analytics: { "errorCode": "BadRequest", "message": "The request failed with status code…
How to parse nested json array of document in ADF data flow
Hi all I am trying to fitch the values from a nested josh array of document , I have used aggregate to convert into objects but not able to fitch the values of all child nodes like as below itOffer.item itOffer.item.SplOfr itOffer.item.buy …
Configuring sql user permission to INSERT only via stored procedure in Synapse dedicated pool
We currently have a client with External users set up via AAD and each group has access to a certain schema with grant permission to select, insert, delete and update on their dedicated SQL pool. Our client would like to revoke explicit insert, delete…
Azure Synapse Analytics Workspace And its Dedicated SQL Pool Migration From one subscription to another subscription during Cross Subscription Migration
I just Want to know that how can I migrate synapse workspace and its dedicated SQL pool from one subscription to another subscription during cross subscription migration
How can we connect Blackboard DDA (a SaaS using PostgreSQL to store data ) via Azure Synapse analytics using SSL certificate for authentication in SHIR for Private Vnet enabled networks.
Blackboard DDA is a SaaS application which uses PostgreSQL as DB to store data. The DB uses a SSL certificate key to authenticate and we have private Vnet enabled. How to create a linked service for this DB and where do we store certificate in SHIR. Are…
auto table join
Hi Team, once we pull the 2 table to synapse... is there any way it will automatically joined with primary and foreign keys like we see in power BI
InvalidHttpRequestToLivy: Your Spark job requested 12 vcores. However, the workspace has a 0 core limit.
it gives me this error and when i tried to increase core size limit i could not find that option so how to solve this error?
Send large amount of data to external systems
Hi friends, we have a requirement to send a large amount of data in gigabytes from Azure to third-party external systems. The data needs to be sent to both azure and API public endpoint methods. How to send this securely without latency? What are the…
Synapse workspace deployment node-js error ERR_INVALID_ARG_TYPE
Hi I am working on a CI/CD DevOps pipeline for Synapse Analytics. The Synapse workspace is integrated in my git repo, and I want to use task 'Synapse Workspace Deployment' with operation 'validate' to generate the ARM templates. The task is configured as…