question

FlintHarrell-0170 avatar image
0 Votes"
FlintHarrell-0170 asked ShaikMaheer-MSFT commented

Testing load logic efficacy

So I have flat files that are being pulled from Blob storage and being transformed into an analytical model in Azure SQL database. Is there a way to test the data loaded matches what was in the flat files natively? I was thinking I could use Databricks manually for this type of thing, but ideally I would like it to occur as part of the pipeline so I can implement failover.

azure-data-factory
· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @FlintHarrell-0170 ,

Welcome to Microsoft Q&A Platform. Thank you for posting your query here.

Could you please confirm which activity you are using for data load in Azure SQL database? Is it copy activity or data flow activity?

Copy activity or Data flow activity output json will have some useful information related to your data movement. is that's not helpful in your case ?

0 Votes 0 ·

Hi @FlintHarrell-0170 ,

Just checking did you get chance to share details on above commented asks? Thank you.

0 Votes 0 ·

Hi @FlintHarrell-0170 ,

Did you get chance to share details on above commented asks? Thank you.

0 Votes 0 ·

0 Answers