question

CTrudgeon-1880 avatar image
0 Votes"
CTrudgeon-1880 asked CTrudgeon-1880 commented

Setting Up Pipeline Filters in Data Factory?

I'm new to Data Factory, so please forgive my ignorance. I've successfully created a Copy Data activity pipeline and now I'm looking to filter down the data. The activity copies accounts from Salesforce to an Oracle table. Not all of our accounts in Salesforce contain customer numbers. I'm looking to only copy accounts that have a customer number.

In Informatica, it was a matter of a adding a simple expression like: Customer_Number__c <> ''

I'm sure I'll eventually get used to it, but right now I'm finding it confusing and cumbersome to add a filter in Data Factory. Can somebody tell what I need to put into the filter settings (Items & Condition) to properly filter my activity?

Activity: TEST_SF_ACCOUNTS
Source Dataset: Account
Source Field: Customer_Number_c
Sink Dataset: DFACTORY_SF_ACCOUNTS
Sink Field: CUSTOMER_NUMBER
Condition: Customer_Number
_c field needs to contain a value

Any help would be appreciated!




azure-data-factory
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

HimanshuSinha-MSFT avatar image
0 Votes"
HimanshuSinha-MSFT answered CTrudgeon-1880 commented

Hello @CTrudgeon-1880 ,
Thanks for the ask and using the Microsoft Q&A platform & a very warm welcome to Azure data factory .


I think your ask is on how to copy the data for valid customers . Well after reading the ask I think at this time you are copying the whole table/Object to Oracle. Are you aware that you can write a query on the SF side in SOQL . Please read about the same here .

95368-image.png


I think if you use query you can acheive what you are looking for . Unfortunately at this time we dont have a SF accounts and so I am not ablwe to crodd check this .

Please do let me know how it goes .
Thanks
Himanshu
Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members



image.png (34.2 KiB)
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thank you @HimanshuSinha-MSFT. That worked wonderfully.

0 Votes 0 ·