question

yosafatsaragih-1755 avatar image
0 Votes"
yosafatsaragih-1755 asked Monalv-msft edited

SSIS integration to run simple clustering task

Dear all SSIS fellow,

Right now I work on SSIS pipeline to make automatic feature engineering which also have simple AI process to make the class. In the pipeline, we need to make a cluster based on simple feature (8 column pulled from database directly) to declare it as the class, and join it with our SSIS data cleaning process. The source and destination database are PostgreSQL and not in Azure environment. Since I working with Python mainly to make the AI part, I'm very new to C# environment of SSIS. Is there any possibilities to use Cython module to work the clustering or another possible option to run simple clustering process on SSIS?

sql-server-integration-services
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

YitzhakKhabinsky-0887 avatar image
0 Votes"
YitzhakKhabinsky-0887 answered YitzhakKhabinsky-0887 commented

@yosafatsaragih-1755,

You can safely ignore Mona's answer. It is pertaining to the SQL Server and SSIS multi-server failover clustering for high availability.

You can definitely use SSIS Script Task with any c# libraries for your needs.

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

is there any example for that implementation? Since i'm unfamiliar with C# environment?

0 Votes 0 ·

Here is a good starting point: getting-started-with-the-ssis-script-task

There are many other good tutorials.


0 Votes 0 ·
Monalv-msft avatar image
0 Votes"
Monalv-msft answered Monalv-msft edited

Hi @yosafatsaragih-1755 ,

Clustering Integration Services is not recommended because the Integration Services service is not a clustered or cluster-aware service, and does not support failover from one cluster node to another. Therefore, in a clustered environment, Integration Services should be installed and started as a stand-alone service on each node in the cluster.

Please refer to Integration Services (SSIS) in a Cluster and Clustering SSIS.

Best regards,
Mona


If the answer is helpful, please click "Accept Answer" and upvote it.

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.



5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

yosafatsaragih-1755 avatar image
0 Votes"
yosafatsaragih-1755 answered

How about if i put the process in the script task, I have read that C# has several ML library such as Accord.NET or ML-NET? I mean the data clustering process are single task which not need additional server (in this case, i run my SSIS in my local environment which connected to external server).

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.