question

PorscheMe-6235 avatar image
0 Votes"
PorscheMe-6235 asked PRADEEPCHEEKATLA-MSFT commented

Pyignite Client.connect failed: Connection refused

  • vNet address: 10.106.0.0/15

  • databricks public subnet address prefix: 10.106.3.0/24

  • databricks private subnet address prefix: 10.106.4.0/24

  • Databricks Runtime 7.4 (Apache Spark 3.0.1) deployed with delegated subnets

  • Kubernetes subnet address prefix: 10.106.8.0/22

  • Kubernetes version 1.19.7

  • Apache Ignite 2.9.1 deployed in the kubernetes

    $ kubectl get svc -n ayushman
    NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
    cohortstore LoadBalancer 10.0.163.252 10.106.8.255 8080:31751/TCP,10800:30608/TCP,10900:31092/TCP 29h


We are getting 'Connection refused' error when trying to connect to Apache Ignite cluster from Databricks Notebooks. Below is our simple code snippet.

 from pyignite import Client
    
 client = Client(timeout=40.0)
 client.connect('10.106.8.255', 10800)


We are able to connect to Apache Ignite using ./sqlline.sh from a Linux machine with in the vNet (outside the K8S subnet)

Any help to resolve this issues?






azure-databricksdotnet-ml-big-data
· 3
5 |1600 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Visible to all users

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

PRADEEPCHEEKATLA-MSFT avatar image PRADEEPCHEEKATLA-MSFT ·

Hello @PorscheMe-6235,

Thanks for the ask and also for using the Microsoft Q&A forum.

For a deeper investigation and immediate assistance on this issue, if you have a support plan you may file a support ticket. In case if you don't have a support plan, I will enable a one-time free support request for your subscription. Please do let us know.

0 Votes 0 ·
PorscheMe-6235 avatar image PorscheMe-6235 PRADEEPCHEEKATLA-MSFT ·

Thanks @PRADEEPCHEEKATLA-MSFT for the reply.

We do not yet have a support plan.

0 Votes 0 ·
PRADEEPCHEEKATLA-MSFT avatar image PRADEEPCHEEKATLA-MSFT ·

Hello @PorscheMe-6235,

Just checking are you able to resolve the issue with the help on support?

Once the issue is sorted out with the support, please do share the resolution, which might be beneficial to other community members reading this thread.

0 Votes 0 ·

1 Answer

PorscheMe-6235 avatar image
1 Vote"
PorscheMe-6235 answered PRADEEPCHEEKATLA-MSFT commented

The issue was resolved with Microsoft support help.
Issues was on Databricks side, the Databricks High Concurrency cluster setting - spark.databricks.pyspark.enableProcessIsolation was set to true.
When spark.databricks.pyspark.enableProcessIsolation was set to true, Databricks will block all outbound connection apart from port 443.
Solution was to add spark.databricks.pyspark.iptable.outbound.whitelisted.ports 10800 configuration on cluster advanced setting, since Apache Ignite thin clients connects on port 10800.

· 1
5 |1600 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Visible to all users

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

PRADEEPCHEEKATLA-MSFT avatar image PRADEEPCHEEKATLA-MSFT ·

Hello @PorscheMe-6235,

Glad to know that your issue has resolved.
Now you can accept it as answer and thanks for sharing the solution, which might be beneficial to other community members reading this thread.

0 Votes 0 ·