question

MakabataJMokgolobotho-4931 avatar image
0 Votes"
MakabataJMokgolobotho-4931 asked MakabataJMokgolobotho-4931 answered

Azure Data Factory- Data Source sink path Issues

I building a pipeline that pulls data from an Azure SQL table into Azure Blob storage in a parquet file. The data get pulled correctly but when the file has to be saved an error occurs related to the file path.

Error message:
Failure happened on 'Sink' side. ErrorCode=UserErrorFailedFileOperation,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Upload file failed at path **<container name>/<folder>/<subfolder>/<subfolder>/<subfolder>/<subfolder>/<subfolder>/*<filename>* .,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.WindowsAzure.Storage.StorageException,Message=The remote server returned an error: (400) Bad Request.,Source=Microsoft.WindowsAzure.Storage,''Type=System.Net.WebException,Message=The remote server returned an error: (400) Bad Request.,Source=Microsoft.WindowsAzure.Storage,'

The error is caused by the "\" which should not be there and I do not have it on the file path. And it only happens on the filename
data_source
How do I resolve this?


azure-data-factory
image.png (30.3 KiB)
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @MakabataJMokgolobotho-4931,

As suggested by @VaibhavChaudhari, could you please confirm if you are seeing the issue even if you hardcode the dataset values?

Please let us know how it goes.


1 Vote 1 ·

Does it work when you provide full path as hard coded?

0 Votes 0 ·

1 Answer

MakabataJMokgolobotho-4931 avatar image
0 Votes"
MakabataJMokgolobotho-4931 answered

In an attempt to answer your question with a bit more detail, I went on a journey and tested in numeral ways until I got to what I was actully implementing, and for some reason, it works just fine now. Thank you, Vaibhav.

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @MakabataJMokgolobotho-4931, Glad to know your issue was resolved and thanks for confirming. Do let us know if you see any issues or feel free to open a new thread.

Thank you

0 Votes 0 ·