question

LouDriehuis avatar image
0 Votes"
LouDriehuis asked LouDriehuis answered

Azure DataFactory .NET SDK CopyTranslator

Hi Guys,

I want to make a pipeline in Azure Data Factory with the use of the .NET SDK. I followed the given example and the basics work really fine and nice.
but I am really struggling with the Mapping of fields. I 'Binged' and 'Googled' the night away, but I found it really hard to find any examples of how to use the CopyTranslator or TabularTranslator.

For example when working with The CopyTranslator, I the properties 'Translator' or the CopyActivity to a a new Instance of CopyTranslator. Then I try to set the AdditionalProperties (IDictionary<String, Object>), but what needs to go into the value part of that dictionary???

Can somebody help me with this, or do you know some locations were I can find examples?
Help is very much appriciated.

Many thanks,
Lou

azure-data-factory
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

SaurabhSharma-msft avatar image
0 Votes"
SaurabhSharma-msft answered SaurabhSharma-msft commented

Hi @loudriehuis,
Thanks for using Microsoft Q&A !!
Please find below the .NET Code for your reference which you could use to add a Tabular Translator to a Copy Data activity.

 PipelineResource newSQLPipeline = new PipelineResource
             {
                 Parameters = new Dictionary<string, ParameterSpecification>
                 {
                     { "inputPath", new ParameterSpecification { Type = ParameterType.String } },
                     { "outputPath", new ParameterSpecification { Type = ParameterType.String } }
                 },
                 Activities = new List<Activity>
                 {
                     new CopyActivity
                     {
                         Name = "CopyData",
                         Inputs = new List<DatasetReference>
                         {
                             new DatasetReference()
                             {
                                 ReferenceName = sqlSourceDatasetName
                                    
                             }
                         },
                         Outputs = new List<DatasetReference>
                         {
                             new DatasetReference
                             {
                                 ReferenceName = sqlSinkDatasetName                                
                             }
                         },
                         Source = new SqlSource { },
                         Sink = new SqlSink { },
                         Translator =  new TabularTranslator()
                         {
                             Mappings = new List<Dictionary<string,Dictionary<string,string>>>()
                             {
                                 new Dictionary<string, Dictionary<string,string>>()
                                 {
                                     { "source",new Dictionary<string, string>(){
                 {"name","Id" }}},
                                     { "sink",new Dictionary<string, string>(){
                 {"name","CustomerId" }}}
                                 },
                                 new Dictionary<string, Dictionary<string,string>>()
                                 {
                                     { "source",new Dictionary<string, string>(){
                 {"name","Name" }}},
                                     { "sink",new Dictionary<string, string>(){
                 {"name","LastName" }}}
                                 },
                                 new Dictionary<string, Dictionary<string,string>>()
                                 {
                                     { "source",new Dictionary<string, string>(){
                 {"name", "LastModifiedDate" } }},
                                     { "sink",new Dictionary<string, string>(){
                 {"name", "ModifiedDate" } }}
                                 }
                             },
    
                         }
                     }
                 }
             };

Results:
137113-image.png

Data Factory Mapping section -
137059-image.png

Please let me know if you have any questions.

Thanks
Saurabh


Please do not forget to "Accept the answer" wherever the information provided helps you to help others in the community.


image.png (197.9 KiB)
image.png (24.0 KiB)
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @loudriehuis,

I have not heard back from you. Did my answer solve your issue? If so, please mark as accepted answer. If not, please let me know how I may better assist.

Thanks
Saurabh

0 Votes 0 ·
LouDriehuis avatar image
0 Votes"
LouDriehuis answered

Hi @SaurabhSharma-msft,

Many thanks for you answer. It will certainly help.

Ciao
Lou

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.