Pyignite Client.connect failed: Connection refused

Question

question

PorscheMe-6235 asked Feb 18, '21 PRADEEPCHEEKATLA-MSFT commented Mar 25, '21

Pyignite Client.connect failed: Connection refused

vNet address: 10.106.0.0/15
databricks public subnet address prefix: 10.106.3.0/24
databricks private subnet address prefix: 10.106.4.0/24
Databricks Runtime 7.4 (Apache Spark 3.0.1) deployed with delegated subnets
Kubernetes subnet address prefix: 10.106.8.0/22
Kubernetes version 1.19.7
Apache Ignite 2.9.1 deployed in the kubernetes

$ kubectl get svc -n ayushman
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
cohortstore LoadBalancer 10.0.163.252 10.106.8.255 8080:31751/TCP,10800:30608/TCP,10900:31092/TCP 29h

We are getting 'Connection refused' error when trying to connect to Apache Ignite cluster from Databricks Notebooks. Below is our simple code snippet.

 from pyignite import Client
    
 client = Client(timeout=40.0)
 client.connect('10.106.8.255', 10800)

We are able to connect to Apache Ignite using ./sqlline.sh from a Linux machine with in the vNet (outside the K8S subnet)

Any help to resolve this issues?

azure-databricks dotnet-apache-spark

· 3

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

PRADEEPCHEEKATLA-MSFT · Feb 19 at 05:49 AM

Hello @PorscheMe-6235,

Thanks for the ask and also for using the Microsoft Q&A forum.

For a deeper investigation and immediate assistance on this issue, if you have a support plan you may file a support ticket. In case if you don't have a support plan, I will enable a one-time free support request for your subscription. Please do let us know.

0 ·

PorscheMe-6235 PRADEEPCHEEKATLA-MSFT · Feb 19 at 02:45 PM

Thanks @PRADEEPCHEEKATLA-MSFT for the reply.

We do not yet have a support plan.

0 ·

PRADEEPCHEEKATLA-MSFT · Mar 11 at 10:31 AM

Hello @PorscheMe-6235,

Just checking are you able to resolve the issue with the help on support?

Once the issue is sorted out with the support, please do share the resolution, which might be beneficial to other community members reading this thread.

0 ·

Answer 1 · 2021-03-11T17:07:06.957Z

PorscheMe-6235 answered Mar 11, '21 PRADEEPCHEEKATLA-MSFT commented Mar 25, '21

The issue was resolved with Microsoft support help.
Issues was on Databricks side, the Databricks High Concurrency cluster setting - spark.databricks.pyspark.enableProcessIsolation was set to true.
When spark.databricks.pyspark.enableProcessIsolation was set to true, Databricks will block all outbound connection apart from port 443.
Solution was to add spark.databricks.pyspark.iptable.outbound.whitelisted.ports 10800 configuration on cluster advanced setting, since Apache Ignite thin clients connects on port 10800.

· 1

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

PRADEEPCHEEKATLA-MSFT · Mar 25 at 10:50 AM

Hello @PorscheMe-6235,

Glad to know that your issue has resolved.
Now you can accept it as answer and thanks for sharing the solution, which might be beneficial to other community members reading this thread.

0 ·

question