question

ShahinMortazave-1426 avatar image
0 Votes"
ShahinMortazave-1426 asked HimanshuSinha-MSFT commented

copy tables

HI,

I have a demo environment for Azure Data Factory, I have setup pipeline to copy some tabled from my on-perm SQL server to an Azure SQL server. When trigger the pipeline on-perm tables get copied to the Azure DB.
Now I have to copy more tables from the same on-perm DB to the same SQL SQL Azure DB.
I did create a new data set in existing pipelines, now I see the original dataset with 4 tables and destination dataset and the newly added dataset for the new table, after triggered the pipeline I can see that only the tables from the first dataset get copied to the Azure and the newly added table is not get copied.

Any suggestion on how to add extra tables to existing pipeline?

Thanks

azure-data-factory
· 2
5 |1600 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Visible to all users

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

HimanshuSinha-MSFT avatar image HimanshuSinha-MSFT ·

Hello @ShahinMortazave-1426 ,
We haven’t heard from you on the last response from @NandanHegde-7720 and was just checking back to see if you have a resolution yet .In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .
Thanks
Himanshu

0 Votes 0 ·
HimanshuSinha-MSFT avatar image HimanshuSinha-MSFT HimanshuSinha-MSFT ·

Hello @ShahinMortazave-1426 ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .
Thanks
Himanshu

0 Votes 0 ·

2 Answers

NandanHegde-7720 avatar image
0 Votes"
NandanHegde-7720 answered

Hey @ShahinMortazave-1426
Can you please share the image and json code of your pipeline
Based on whatever data you have provided, it seems that you have created new datasets but you have not created new copy activity in the pipeline to map the data sets

5 |1600 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Visible to all users

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

ShahinMortazave-1426 avatar image
0 Votes"
ShahinMortazave-1426 answered NandanHegde-7720 commented

@NandanHegde-7720 Thank you for your reply,
I agree with you, I have just create a new dataset and have no idea how to put this dataset in copy activity, should I create the dataset by editing the copy activity?
These are some screenshots from my demo ADF:

76386-workingdataset.jpg


76345-newdataset.jpg


76298-pipeline.jpg



workingdataset.jpg (59.6 KiB)
newdataset.jpg (53.5 KiB)
pipeline.jpg (84.3 KiB)
· 1
5 |1600 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Visible to all users

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

NandanHegde-7720 avatar image NandanHegde-7720 ·

Hey ,
Sorry to have missed out on this.
There are multiple ways to create a dataset : either you can create dataset while setting up the copy activity or you get an option to create a new dataset as below:

77975-dataset.png


Also you can have multiple copy activities within a single pipeline.
So once you have created the datasets, then create a copy activity for each table and map the already created datasets in the copy activities.

0 Votes 0 ·
dataset.png (13.2 KiB)