4,533 questions with Azure Synapse Analytics tags

Sort by: Updated
1 answer

synapse table partition on single letter i.e Y or N?

In synapse, table reached over 5 billions rows, contain various history of version records, and every record row has status flag of Y or N. Y is active record and N is oldest . Almost 80% rows are history record and only 20% row are active. Majority of…

Azure SQL Database
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,533 questions
asked 2020-12-22T11:34:29.187+00:00
a1990 11 Reputation points
commented 2020-12-29T18:58:33.817+00:00
MartinJaffer-MSFT 26,041 Reputation points
1 answer

How to query a 3rd party Azure Data Lake (2nd Gen) via Shared Access Siganture (SAS URL) in Azure Synapse?

I have been granted read only access to a Vendor's azure data lake and given a SAS connect string. How can I query this data via Azure Synapse and store the results in my own data lake? fyi- The vendor's data is stored in JSON file format in this…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,533 questions
Azure Data Lake Analytics
asked 2020-12-13T19:43:57.3+00:00
JasonW-5564 161 Reputation points
commented 2020-12-29T11:20:21.937+00:00
HarithaMaddi-MSFT 10,136 Reputation points
1 answer

Azure data lake 2 parallel load to multiple files and use delta lake

Hi Guys, We have multiple shipment files coming in from multiple clients as below Shipment_2020111801 Shipment_2020111802 Shipment_2020111803 Shipment_2020111804 Shipment_2020111805 Shipment_2020111806 Shipment_2020111807 Each file is…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,533 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,856 questions
asked 2020-11-19T01:54:02.727+00:00
Dondapati, Navin 281 Reputation points
commented 2020-12-29T02:05:16.65+00:00
Nasreen Akter 10,766 Reputation points
0 answers

Azure synapse : How to check transaction log size usage ?

I have few questions for Azure synapse transaction :. How to check the current transaction log size used by Azure synapse db ? How to check the transaction log size used by each query ? How to truncate the transaction log ? or shrink ?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,533 questions
asked 2020-12-05T04:08:11.2+00:00
sakuraime 2,321 Reputation points
commented 2020-12-28T18:54:32.437+00:00
Saurabh Sharma 23,771 Reputation points Microsoft Employee
2 answers

Azure data warehouse vs snowflake

Hello, We would like to move to the cloud, ideally azure and use power bi... We have a lot of ssis packages at present and possibily need to convert to cloud... I have to give a presentation about microsoft azure (synapse, ADF, etc.) vs snowflake …

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,533 questions
asked 2020-12-16T09:00:46.293+00:00
arkiboys 9,686 Reputation points
commented 2020-12-25T20:00:56.177+00:00
arkiboys 9,686 Reputation points
1 answer One of the answers was accepted by the question author.

SQL Pools missing in the data flows source settings

Hi, I have created a couple dedicated sql pools within the synapse workspace with a view to load data from one poll to the other. However, the sql pools doesn't show in the source type (inline). Can you help please? I wonder if I need to add an extension…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,533 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,856 questions
asked 2020-12-24T10:55:07.62+00:00
Synapse 21 Reputation points
commented 2020-12-24T15:48:33.363+00:00
Synapse 21 Reputation points
1 answer One of the answers was accepted by the question author.

External Database Scoped Credential via SAS

Is it possible to create a data base scoped credential in synapse for Azure Blob Storage in SYnapse? Tried this scenario : WITH IDENTITY = 'SHARED ACCESS SIGNATURE', SECRET = '<your SAS secret without the preleading ?>'; CLOSE MASTER KEY; -- only…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,533 questions
asked 2020-12-24T06:45:45.343+00:00
Sahil Mehta 61 Reputation points
accepted 2020-12-24T06:46:40.207+00:00
Sahil Mehta 61 Reputation points
1 answer

Querying flat files (parquet, csv) with different columns

Hi, I'm trying to Query a set of files that are stored in ADL Gen2. Those files have different columns between them like: File 1: Id, Name, LastName File 2: Id, Name, LastName, Phone How could I create an external table without conflicts? …

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,391 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,533 questions
asked 2020-12-11T12:48:57.28+00:00
Jhonatan Reyes 61 Reputation points
commented 2020-12-24T01:56:27.907+00:00
HimanshuSinha-msft 19,386 Reputation points Microsoft Employee
2 answers

Visualization/plots/graphs executed successfully on PySpark in Azure Synapse but not showing up! Please help!!

Visualization/plots/graphs executed successfully on PySpark in Azure Synapse but not showing up! Please help!!

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,533 questions
asked 2020-09-28T10:12:00.817+00:00
angie578 1 Reputation point
commented 2020-12-23T07:36:24.637+00:00
Wipada Chanthaweethip 1 Reputation point
1 answer

Can you connect to an Azure Synapse Analytics Spark cluster through SSMS?

Hi All, I'm new to Azure Synapse Analytics and would like to use the Spark Cluster feature. My team are SSMS SQL users and would like to query the data on a Spark Cluster through SSMS. I know the Serverless SQL endpoint link can be used was…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,533 questions
asked 2020-12-22T15:19:52.11+00:00
Courtney Haedke 151 Reputation points
answered 2020-12-22T15:45:48.66+00:00
Vaibhav Chaudhari 38,666 Reputation points
1 answer

How to query 3rd party Azure DataLake Gen2 and only store the results

First, what I am trying to do is I want to query and aggregate raw JSON files stored in a 3rd party's Azure Data Lake (Gen2) and store those aggregates in my own data lake or relation db. I do not want to physically copy all of those raw JSON files…

Azure SQL Database
Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,391 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,533 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,008 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,856 questions
asked 2020-12-15T16:14:02.687+00:00
JasonW-5564 161 Reputation points
commented 2020-12-22T12:18:09.123+00:00
HarithaMaddi-MSFT 10,136 Reputation points
4 answers One of the answers was accepted by the question author.

temp table getting dropped in synapse in the same session.

Hi All, I am trying to create temp tables for some analysis on the synapse analytics. While I am able to operate the data from temp table in the same execution but I am unable to access temp table individually in the same session. Just wondering if…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,533 questions
SQL Server
SQL Server
A family of Microsoft relational database management and analysis systems for e-commerce, line-of-business, and data warehousing solutions.
13,028 questions
Transact-SQL
Transact-SQL
A Microsoft extension to the ANSI SQL language that includes procedural programming, local variables, and various support functions.
4,574 questions
asked 2020-12-19T19:28:59.903+00:00
VS29 246 Reputation points
commented 2020-12-21T22:03:04.63+00:00
Erland Sommarskog 102.9K Reputation points
1 answer

Azure Synapse resource group concept

Please correct me if I am wrong Suppose I have two users in smallrc and two users in xlargerc If both users in smallrc run a query at the same time, user B will need to wait user A to complete before executing , because Slots used by smallrc is…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,533 questions
asked 2020-12-01T09:39:39.787+00:00
sakuraime 2,321 Reputation points
commented 2020-12-21T05:52:55.763+00:00
PRADEEPCHEEKATLA-MSFT 81,646 Reputation points Microsoft Employee
1 answer One of the answers was accepted by the question author.

Azure synapse : is it possible to check each query DWU usage ?

Are there any DMV in Azure synapse that can check each query DWU usage ?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,533 questions
asked 2020-12-03T06:07:01.057+00:00
sakuraime 2,321 Reputation points
accepted 2020-12-18T03:48:59.287+00:00
sakuraime 2,321 Reputation points
1 answer One of the answers was accepted by the question author.

Best options to pull data from SaaS API to Azure Synapse

What are the best options for ingesting data from SaaS Application to Synapse through API with Authentication? We have a couple of SaaS Applications such as NetSuite, EyeShare & SAP SaaS Applications, and planning to ingest the data to Synapse but…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,533 questions
asked 2020-12-15T07:29:32.403+00:00
Subhashini Karur Gunasekaran 41 Reputation points
commented 2020-12-18T03:43:19.707+00:00
Subhashini Karur Gunasekaran 41 Reputation points
1 answer

Are composite primary keys supported in Azure Synapse external tables?

Hi. Are composite primary keys in Azure Synapse external tables supported? If yes, can you kindly share the syntax for creating the composite PK? Thank you. Hans Kiamzon

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,533 questions
asked 2020-12-11T07:15:02.073+00:00
HansJKiamzon 56 Reputation points
commented 2020-12-17T22:54:04.483+00:00
HimanshuSinha-msft 19,386 Reputation points Microsoft Employee
1 answer

SQL SERVER Source table create point in table of 70 gb table

Hi Guys, We want to create snapshot of an table everyday in azure synapse on sQldw, what would be best approach for daily load of 70GB table from on premise sql,oracle or sap hana, we want load the data in mill seconds max in minutes. Hint -Similar…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,533 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,856 questions
asked 2020-12-08T02:30:26.787+00:00
Dondapati, Navin 281 Reputation points
commented 2020-12-17T22:03:19.62+00:00
HimanshuSinha-msft 19,386 Reputation points Microsoft Employee
0 answers

Synapse with git integration - publish issues with spark job

I'm having issues with git integration in Synapse. They concern spark job definition in pipelines. When creating new job, the execution policy is saved, commited to git. It is present on collaborative branch after completing pull request. …

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,533 questions
asked 2020-12-15T08:01:53.003+00:00
Janusz Ciesielski 6 Reputation points
commented 2020-12-17T12:44:07.353+00:00
HarithaMaddi-MSFT 10,136 Reputation points
2 answers One of the answers was accepted by the question author.

No results when querying using Openrowset from ADLS Gen2.

No results when querying using Openrowset from ADLS Gen2. I used to able to query earlier and for the past two weeks it is showing me No results. Any change in permissions? I did follow all the permissions as in below URL. …

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,533 questions
asked 2020-12-14T20:47:33.823+00:00
Jeeva 161 Reputation points
accepted 2020-12-16T13:29:58.333+00:00
Jeeva 161 Reputation points
1 answer

Azure synapse : partition a table that has no partition

Is it possible to partition a table (which current not partitioned) , without recreate the table ?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,533 questions
asked 2020-12-03T03:02:21.147+00:00
sakuraime 2,321 Reputation points
commented 2020-12-15T07:35:02.057+00:00
HarithaMaddi-MSFT 10,136 Reputation points