question

CornelVerster-9491 avatar image
0 Votes"
CornelVerster-9491 asked KranthiPakala-MSFT commented

Azure Data Factory - Reading JSON Array and Writing to Individual CSV files

Hi all!

I have many JSON files that lands in my data lake that all have the same format like this:

 [
   {
     "AzureContainer": "someContainer",
     "AzureFolderName": "someFolder",
     "AzureFilename": "myFilename",
     "Timestamp: "123123213",
     "ItemName": "myItem",
     "Value": 567
   },
   {
     "AzureContainer": "someContainer",
     ...
   },
   ...
 ] 

What I want to do, is to use the information in these items to append to CSV files in my Data Lake in the format:
[Timestamp],[ItemName],[Value]
[Timestamp],[ItemName],[Value]
....

What I've done thus far is create a lookup that reads all of the files within my landing blob. I can then extract each of these individual JSON objects. Now, I need to use the information in the JSON object to write to the specified location in my Azure storage account (using AzureContainer, AzureFolderName and AzureFilename), and I need to create a CSV entry using Timestamp, ItemName and Value.

How can I do this? I'm trying to find something that allows me to write to a CSV without having to assign a Source datasource? Thus I want to format my CSV content, and use the Azure parameters to append the CSV file in the specified location.

azure-data-factoryazure-data-lake-storage
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

KranthiPakala-MSFT avatar image
0 Votes"
KranthiPakala-MSFT answered KranthiPakala-MSFT commented

Hi @CornelVerster-9491,

Thanks for using Microsoft Q&A forum and posting your query.
The same question was answered by one of our engineer in Stackoverflow forum.

Here is the stackoverflow link with detailed step by step explanation of the implementation to achieve this requirement: Azure Data Factory - Reading JSON Array and Writing to Individual CSV files

Sharing it here as it would be beneficial for others reading this thread.
Hope this helps. Please feel free to let us know if you have further query


  • Please accept an answer and up-vote if it helps. Original posters help the community find answers faster by identifying the correct answer. Here is how.

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification.



· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @CornelVerster-9491,

Just checking in to see if the above suggestion was helpful. If it answers your query, please do click “Accept Answer” and/or Up-Vote, as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

0 Votes 0 ·

Hi @CornelVerster-9491,

We still have not heard back from you. Just wanted to check if the above suggestion was helpful? If it answers your query, please do click “Accept Answer” and/or Up-Vote, as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

0 Votes 0 ·