question

PranviKanwar-2531 avatar image
0 Votes"
PranviKanwar-2531 asked KranthiPakala-MSFT commented

Streamsets Pipeline Query

Can streamsets pipeline be called from adf?

azure-data-factory
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

KranthiPakala-MSFT avatar image
0 Votes"
KranthiPakala-MSFT answered KranthiPakala-MSFT commented

Hi @pranvikanwar-2531,

Thanks for using Microsoft Q&A forum and posting your query.

Unfortunately there is no out of box ADF connector for Streamlets. But you can try exploring Custom Activity and Azure Function activity in ADF. To move data to/from a data store that Data Factory does not support, or to transform/process data in a way that isn't supported by Data Factory, you can create a Custom activity with your own data movement or transformation logic and use the activity in a pipeline.

I'm not familiar with Steamsets but incase if there is a REST API to call the Streamsets pipeline, you can try exploring REST connector in ADF and use it in a Web activity.

And if you have any feedback for a feature request for Streamsets, I would recommend you to please share it in ADF uservoice forum : https://feedback.azure.com/forums/270578-data-factory

All the feedback shared in this forum are actively monitored and reviewed by ADF product team.

Hope this info helps.



Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.



· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @pranvikanwar-2531,

Just checking in to see if the above suggestion was helpful. If it answers your query, please do click “Accept Answer” and/or Up-Vote, as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

1 Vote 1 ·