question

balaa-uk avatar image
0 Votes"
balaa-uk asked HimanshuSinha-MSFT commented

How do I write each row of SQL Table to Datalake as json file?

I have a table with 100K+ rows. I want to write each row as a JSON file into Datalake. I am not quite sure how to approach this.

Thought of Lookup activity but it has various limitations (Rows, Size, Run duration etc).

Would appreciate any pointers on how to approach this?

azure-data-factory
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @balaa-uk ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .
Thanks
Himanshu

0 Votes 0 ·

Hello @balaa-uk ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .
Thanks
Himanshu

0 Votes 0 ·

1 Answer

HimanshuSinha-MSFT avatar image
1 Vote"
HimanshuSinha-MSFT answered HimanshuSinha-MSFT edited

Hello @balaa-uk ,
Thanks for the ask and using the Microsoft Q&A platform .

You can achieve this by using data flow . I did created a source ( in this case some emplyee data )

122493-image.png

Use the Surrogate key

122516-image.png

Set the value accordingly

122447-image.png

On the sink side set the partition option to key column .

122525-image.png


When you run the data flow , you will have three folders which will have files for each record .


122504-image.png


Please do let me know how it goes .
Thanks
Himanshu
Please do consider clicking on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members



image.png (64.5 KiB)
image.png (82.0 KiB)
image.png (24.0 KiB)
image.png (55.5 KiB)
image.png (124.6 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.