question

SteveCardella-9597 avatar image
1 Vote"
SteveCardella-9597 asked SamaraSoucy-MSFT commented

Powershell Set-AzSynapseLinkedService returns error Object reference not set to an instance of an object

I'm trying to migrate a known-working Data Factory solution to an Azure Synapse Workspace. I'm getting the "object reference not set to an instance of an object" error on using Powershell to deploy one of my linked services.

I've done a similar deployment before with Data Factory with no issues. Here, I'm getting issues deploying one of my linked services and it's preventing me from auto-deploying the rest. The self-hosted runtime deployed fine. 11 of 12 linked services deployed fine.

However, our databricks connection will not deploy (with the same code in a loop) via the powershell command:

 Set-AzSynapseLinkedService -WorkspaceObject $ws -DefinitionFile $ls -Name $ls.BaseName

I've deployed the Databricks linked service with the same file and the equivalent DataFactoryV2 Powershell command. I tried creating it in the Workspace portal by pasting in the JSON from the file, and that worked fine. When it saved, it added the autoresolveintegrationruntime reference to the json when I saved it in the workspace. This seems to be an error with the Powershell command, not an absolute barrier to deployment. I saved off the json with the runtime info to a file. Then I deleted the linked service from the portal and tried to deploy with the revised json file. It was still not working.

Here's the databricks JSON. (filename = ls_azure_databricks.json)

 {
     "name": "ls_azure_databricks",
     "properties": {
         "annotations": [],
         "type": "AzureDatabricks",
         "typeProperties": {
             "domain": {
                 "type": "AzureKeyVaultSecret",
                 "store": {
                     "referenceName": "ls_keyvault",
                     "type": "LinkedServiceReference"
                 },
                 "secretName": "kv-databricks-url"
             },
             "authentication": "MSI",
             "workspaceResourceId": {
                 "type": "AzureKeyVaultSecret",
                 "store": {
                     "referenceName": "ls_keyvault",
                     "type": "LinkedServiceReference"
                 },
                 "secretName": "kv-databricks-workspace-resourceid"
             },
             "existingClusterId": {
                 "type": "AzureKeyVaultSecret",
                 "store": {
                     "referenceName": "ls_keyvault",
                     "type": "LinkedServiceReference"
                 },
                 "secretName": "kv-databricks-clusterid"
             }
         }
     }
 }


Any ideas?

windows-server-powershellazure-synapse-analyticsazure-databricksazure-key-vault
· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I tried to use the Azure CLI just to see what it would say:
az synapse linked-service create --file @./ls_azure_databricks.json --name ls_azure_databricks --workspace-name mysynapseworkspace

That succeeded as well, so it appears to be specific to PowerShell.

I had similar issues where certain files would not work in our dataset definitions in PowerShell but worked perfectly fine in the Az CLI. So, there's something strange in how PowerShell is handling the JSON compared to the Az CLI.

0 Votes 0 ·

Hello Steve,

I'm taking a look at this to see if I can discover why this is happening and if there is a workaround within Powershell. As you have found, using the CLI does act as a possible workaround this issue. If you happen to remember anything about the failed dataset definitions that may help to track down the problem.

0 Votes 0 ·

I can supply the file itself if that's helpful. I checked for weird non-printable characters but couldn't find any. I also ensured that the line endings were LF and that the quote usage was consistently double quotes. It's encoded UTF-8, and that is consistent with the others as well.

0 Votes 0 ·

If you could that would be incredibly helpful. If you aren't comfortable posting here I can setup a private conversation, or you can send me an email to my attention to AzCommunity@microsoft.com. if you send it via email, just be sure to link this thread so it gets routed correctly.

0 Votes 0 ·

0 Answers