A community member has associated this post with a similar question:
Incremental Data load with Data Factory for Dynamic columns

Only moderators can edit this content.

Data Factory to Table Storage Load

Imran Mondal 246

I used Data Factory to Load CSV data From Blob Storage to Table storage, it worked but I am facing the below issues now.

every time the Number of columns is changing in the Source CSV files, which are stored in Blob Storage.

Now, I am trying to do incremental load using the last modified option.

when I run it next time, the error I am getting is that one particular column that is there in my target column (Table storage) is not available in the source file. this is because every time number of columns is changing in the source file.

My question is, how can I load the column dynamically, If the column is not there it should put as a null.

MartinJaffer-MSFT 26,021 Reputation points

2021-04-02T02:00:52.523+00:00

Hello @Imran Mondal and welcome to Microsoft Q&A.

This issue looks similar to, or overlapping with your other issue https://learn.microsoft.com/en-us/answers/questions/340217/index.html
Do you think these should be merged, like a duplicate?
Imran Mondal 246 Reputation points

2021-04-02T02:03:11.46+00:00

@MartinJaffer-MSFT , yes we can merge this with other issue.
MartinJaffer-MSFT 26,021 Reputation points

2021-04-02T16:06:32.327+00:00

I have redirected this thread.