I'm trying to configure Azure event hub and databricks(Python) notebook and getting the following error, any help is appreciated! Issue is while configuring variable - 'eventHubConfiguration'

Question

question

vishalchitrala-8227 asked May 5, '22 PRADEEPCHEEKATLA-MSFT commented May 11, '22

I'm trying to configure Azure event hub and databricks(Python) notebook and getting the following error, any help is appreciated! Issue is while configuring variable - 'eventHubConfiguration'

 #event hub name 
 eventhubname = "rideeventhub"
    
 #Event Hub Connection String 
 eventHubConnectionstring = namespaceConnectionString + ";EntityPath=" + eventhubname
    
    
 **eventHubConfiguration** = {
   'eventhubs.connectionString' : sc._jvm.org.apache.spark.eventhubs.EventHubsUtils.encrypt(eventHubConnectionstring)
 }

Error message :
ERROR:root:Exception while sending command.
Traceback (most recent call last):
File "/databricks/spark/python/lib/py4j-0.10.9.1-src.zip/py4j/java_gateway.py", line 1207, in send_command
raise Py4JNetworkError("Answer from Java side is empty")
py4j.protocol.Py4JNetworkError: Answer from Java side is empty

azure-databricks azure-event-hubs azure-stream-analytics

111.png (107.2 KiB)

· 1

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

PRADEEPCHEEKATLA-MSFT · May 10 at 07:50 AM

Hello @vishalchitrala-8227,

Following up to see if the below suggestion was helpful. And, if you have any further query do let us know.

Please don't forget to click on or upvote button whenever the information provided helps you.

0 ·

Answer 1 · 2022-05-09T07:04:29.283Z

PRADEEPCHEEKATLA-MSFT answered May 9, '22 PRADEEPCHEEKATLA-MSFT commented May 11, '22

Hello @vishalchitrala-8227,

Thanks for the question and using MS Q&A platform.

All configuration relating to Event Hubs happens in your Event Hubs configuration dictionary. The configuration dictionary must contain an Event Hubs connection string:

 connectionString = "YOUR.CONNECTION.STRING"
    
 ehConf = {}
    
 # For versions before 2.3.15, set the connection string without encryption
 # ehConf['eventhubs.connectionString'] = connectionString
    
 # For 2.3.15 version and above, the configuration dictionary requires that connection string be encrypted.
 ehConf['eventhubs.connectionString'] = sc._jvm.org.apache.spark.eventhubs.EventHubsUtils.encrypt(connectionString)

For more details, refer to Structured Streaming + Event Hubs Integration Guide for PySpark.

Hope this will help. Please let us know if any further queries.

Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
Want a reminder to come back and check responses? Here is how to subscribe to a notification
If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

· 1

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

PRADEEPCHEEKATLA-MSFT · May 11 at 09:45 AM

Hello @vishalchitrala-8227,

Just checking in to see if the above answer helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

0 ·

question