Copy activity fails with the Error while using Azure Key Vault in my REST Linked Service

Question

I want to fetch data from OdataURL in my Blob storage, so I am using Copy activity to fetch the same using REST in my source and Blob storage in my Sink. I have used REST Linked Service in my source dataset, so while hardcoding the credentials in LS, the pipeline succeeds but while using Azure key vault in the Linked service the pipeline is failing with below error:

1. When using Basic Authentication type in my LS.
2. When using Service principal Authentication type in my LS. Failure happened on 'Source' side. ErrorCode=JsonInvalidDataFormat,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Error occurred when deserializing source JSON file ''. Check if the data is in valid JSON object format.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Newtonsoft.Json.JsonReaderException,Message=Unexpected character encountered while parsing value: <. Path '', line 0, position 0.,Source=Newtonsoft.Json,'

We tried using Additional headers also in the copy activity where we added content type-> application/json but this also did not work.

Can you please check and let us know the issue.

Answer 1

@Harshita Vishwakarma ,

Thankyou for using Microsoft Q&A platform and thanks for posting your query here.

As per my understanding , you are facing an issue while using Azure Key Vault in their REST Linked Service to fetch data from OdataURL to Blob storage while using REST API as source. The pipeline is failing with an error when using Azure Key Vault in the Linked Service.

• Kindly Check Key Vault Permissions is set corrrectly. Make sure that the managed identity or service principal used by your Azure Data Factory has the necessary permissions to access the Key Vault. Assign the Get and List permissions to the Key Vault access policy for the Data Factory managed identity or service principal.
• Ensure that the secrets stored in Azure Key Vault are correct. For Basic Authentication, you need a username and password; for Service Principal Authentication, you need a client ID, client secret, and tenant ID.
• Verify that the Linked Service is correctly referencing the Key Vault secrets.
• Ensure the REST API URL is correct and accessible. Use a tool like Postman to verify that the credentials and headers are correct, and the API responds as expected.
• Kindly make sure headers are properly set , in your description it seems you have used Content Type instead of Content-Type , please try this : "additionalHeaders": {"Content-Type": "application/json"}

Please share the linked service configuration screenshot if after following the above steps, you are still facing the issue. Also , kindly share the sample response after testing the API with other tool as suggested.

Hope it helps. Kindly accept the answer if it helped. Thankyou