Hi all!
I have many JSON files that lands in my data lake that all have the same format like this:
[
{
"AzureContainer": "someContainer",
"AzureFolderName": "someFolder",
"AzureFilename": "myFilename",
"Timestamp: "123123213",
"ItemName": "myItem",
"Value": 567
},
{
"AzureContainer": "someContainer",
...
},
...
]
What I want to do, is to use the information in these items to append to CSV files in my Data Lake in the format:
[Timestamp],[ItemName],[Value]
[Timestamp],[ItemName],[Value]
....
What I've done thus far is create a lookup that reads all of the files within my landing blob. I can then extract each of these individual JSON objects. Now, I need to use the information in the JSON object to write to the specified location in my Azure storage account (using AzureContainer, AzureFolderName and AzureFilename), and I need to create a CSV entry using Timestamp, ItemName and Value.
How can I do this? I'm trying to find something that allows me to write to a CSV without having to assign a Source datasource? Thus I want to format my CSV content, and use the Azure parameters to append the CSV file in the specified location.