question

snehilsingh-4120 avatar image
0 Votes"
snehilsingh-4120 asked ·

error: not found: value EventHubsConf

Hi Everyone,
I was trying to do some real-time stream analytics using databricks and azure event hub but I keep running into this error even when I have configured everything well.
Here is the code snippet :

import java.util.
import scala.collection.JavaConverters.

import com.microsoft.azure.eventhubs.
import org.apache.spark.eventhubs.EventHubsConf.

import java.util.concurrent.
import org.apache.spark.sql.functions.

import org.apache.spark.sql.types._



//Read the data from the csv files
var tripdata = spark.read.option("header", true).csv(wasbsPath).limit(10000).rdd.collect.toList

//Set up Connection to Azure Event Hubs
val connStr = new ConnectionStringBuilder()
.setNamespaceName(namespaceName)
.setEventHubName(eventHubName)
.setSasKeyName(sasKeyName)
.setSasKey(sasKey)

val pool = Executors.newScheduledThreadPool(1)
val eventHubClient = EventHubClient.create(connStr.toString(), pool)

def sendEvent(message: String) = {
val messageData = EventData.create(message.getBytes("UTF-8"))
eventHubClient.get().send(messageData)
};

val customEventhubParameters = EventHubsConf(connStr.toString()).setMaxEventsPerTrigger(1) //set the maximun event at a time



// We will create a function to send records to Event Hubs. This is done to simulate a realtime stream of events. Whenever we use our incoming stream, we can call sendingEvents() function to send fresh events so that our analysis is performed on a realtime stream.

def sendingData(count : Int) : Unit ={
// Send stream to Event Hubs
for( a <- 0 to (tripdata.length - 1)){
sendEvent(tripdata(a).toString)
}
}



command-1988148540822794:29: error: not found: value EventHubsConf
val customEventhubParameters = EventHubsConf(connStr.toString()).setMaxEventsPerTrigger(1) //set the maximun event at a time

azure-databricksazure-event-hubsazure-stream-analytics
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

HimanshuSinha-MSFT avatar image
1 Vote"
HimanshuSinha-MSFT answered ·

Hello @snehilsingh-4120 ,
Thanks for the ask and using the Microsoft Q&A platform .

Are you not missing the consumer group here ?

EventHubsConf(connStr.toString())
.setConsumerGroup)"your consumer group")
.setMaxEventsPerTrigger(1)

Thanks
Himanshu
Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members


·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

snehilsingh-4120 avatar image
0 Votes"
snehilsingh-4120 answered ·

Hi @HimanshuSinha-MSFT
I tried after creating new consumer group and added that to my command but it still shows the same error

·
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.