RaghuRayala-7358 avatar image
0 Votes"
RaghuRayala-7358 asked MartinJaffer-MSFT commented

Pipeline data flow activity removes the complete column from json when the column value is Null

I am using Data flow activity from azure Data factory pipeline to insert some data in to the cosmos db, and stuck with an issue with having null values for a column in the Json, as the output is currently ignoring properties in the Json which has null values. Can we not store the null value in to json column, as I don't want to replace that with either empty or another value, as this will have more implications to further down the application.

Here is my preview of Data set


complete data flow


What do I need to do in order to retain the columns in the JSON which has null values

image.png (10.1 KiB)
image.png (53.9 KiB)
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @RaghuRayala-7358 and welcome to Microsoft Q&A.

I think I have reproduced your issue.

Given source data like:

after pushing to Cosmos (SQL) I see like

I suspect this lies somewhere in the writing mechanism. If that be the case, then I don't have a ready work-around.
Fortunately, null is what you should get when asking for a property that doesn't exist.

To better understand, can you tell me whether the property is missing from all records, or only the ones where it should be null?

0 Votes 0 ·

0 Answers