question

CTrudgeon-1880 avatar image
0 Votes"
CTrudgeon-1880 asked KranthiPakala-MSFT commented

Data Factory copy activity not copying all records

Hello. I've created a copy data activity that is supposed to bring records from Salesforce into Oracle. We have the exact same job set up in Informatica, so I know how many records should be copied (we are trying to migrate from Informatica to DF). The number of records that should be copied into Oracle is over 5000. However, the Data Factory job is only bringing 133 records into Oracle. Any tips as to what I'm doing wrong? I've been working on this intermittently for weeks now and it's getting very frustrating.

azure-data-factory
· 5
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @CTrudgeon-1880,

Thanks for reaching out and sorry for your experience. This is strange. As per my understanding your copy activity is running successfully but only copying fewer records than expected.

Could you please check your copy activity run to verify if the rowsRead count is same as the rowsWritten. you can verify by looking at the activity runs from Monitor hub.

93599-image.png

If rowsRead is greater than rowsWritten then please check if you have enabled skipIncompatible rows settings in you copy activity.

If the rowsRead count is equal to the rowsWritten count then please check if the query used on the source settings is correct and reading all records that you are expecting, unless there is something wrong with the query, you should be able read all the expected records. If that's the case, then I would suggest filing a support ticket for deeper analysis and immediate assistance.



0 Votes 0 ·
image.png (26.0 KiB)

Hi @CTrudgeon-1880,
Following up to see if you have got a chance to verify the steps mentioned in my previous comment? If that didn't help, please file a support ticket for further troubleshooting, and if you don't have a support plan, please do let us know.

Thank you

0 Votes 0 ·

Hello @KranthiPakala-MSFT,

I checked my copy activity run and it claims the same amount of rows were written as read. However, again the Oracle table only has 133 records. Very confusing.93985-copy-activity.png94038-oracle-record-count.png


0 Votes 0 ·
Show more comments

1 Answer

CTrudgeon-1880 avatar image
1 Vote"
CTrudgeon-1880 answered KranthiPakala-MSFT commented

I resolved the problem. One of the fields in Oracle was not wide enough. The DF copy activity stopped when it hit that record. Strangely, that did not trigger an error message. It simply said that the job was a success. I widened the Oracle column, reran the activity and all of the records flowed in properly. It was not detected when using Informatica because the value was automatically trimmed.

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @ctrudgeon-1880,

Thanks for the update and glad to know you were able to resolve the issue.

0 Votes 0 ·