Data Copy From teradata to snowflake

Aditya Raj 286

I am trying to use copy activity for copying data from teradata to snowflake. In some of columns, data has / and \ .So, data is being skipped but I don't want to skip those rows. Is there any way to handle those in copy activity ? Apart from this issue, there is another issue that empty data is being populated as null in snowflake. I am not sure how to handle this as well. Can somebody please help?

HimanshuSinha-msft 19,381 Reputation points Microsoft Employee

2021-09-07T19:39:21.533+00:00

Hello @Aditya Raj ,
Thanks for the ask and using the Microsoft Q&A platform .
I am not having a teradata instance so I am not able to repro this . I did tried something similar with SQL as the source and it does work as expected .

Can you please try the below query ?

select * from Yourtable
where columenname is null

I know many IDE does offer the feature where the IDR replaces the NULL as blank when you view the data .

On the other ask , Can you please check how the columns are mapped in Source and sink , are they having the same daata type ? Also if you can share some dummy data that can be very helpful .

Please do let me know how it goes .
Thanks
Himanshu
Please do consider clicking on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members
Aditya Raj 286 Reputation points

2021-09-08T11:34:13.58+00:00

For both issue, I tried to load same data using parquet as in between for storing data in azure blob. It didn't populate empty as null. In this case,I copied data from teradata to blob in parquet and then using that parquet file,I loaded data into snowflake. It is working correctly. But when I am using copy activity of adf,it is populating null in place of empty. Is this default behaviour of adf or is there any way to pass any property to change it. Is there any property that I can use to tell adf that while writing data into staging ,don't create text or csv file instead use parquet files.
Aditya Raj 286 Reputation points

2021-09-08T11:35:29.77+00:00

For mapping data_type of sink and source, I am using translator. I have done mapping as mentioned in mapping sheet as per adf
HimanshuSinha-msft 19,381 Reputation points Microsoft Employee

2021-09-23T16:59:48.447+00:00

Hello @Aditya Raj ,
Thanks for the patience , unfortunately we are not getting the kind of response from the team here . if you have a support plan you may file a support ticket, else could you please send an email to azcommunity@microsoft.com with the below details, so that we can create a one-time-free support ticket for you to work closely on this matter.
Subscription ID:
Subject : Attn Himanshu
Please let me know once you have done the same.

Thanks
Himanshu