Rest API Data stream to event hub

Question

Hi Guys,

We are using below rest api get method of accuweather to fetch location info

curl --location --request GET 'http://dataservice.accuweather.com/locations/v1/adminareas/countryCode?apikey=VU86G'

PostMan output:
http://dataservice.accuweather.com/locations/v1/adminareas/countryCode?apikey=VU86G

{
"ID": "AMA",
"LocalizedName": "Amazonas",
"EnglishName": "Amazonas",
"Level": 1,
"LocalizedType": "Department",
"EnglishType": "Department",
"CountryID": "countryCode"
},
{
"ID": "ANT",
"LocalizedName": "Antioquia",
"EnglishName": "Antioquia",
"Level": 1,
"LocalizedType": "Department",
"EnglishType": "Department",
"CountryID": "countryCode"
}

We want to use the url as producer in eventhub and pass through stream analytics and load to blob storage,

How do i make above data as producer to eventhub and make sure it load data every 4 hours?

Regards,
Navin

Answer 1

Hello and welcome back to Microsoft Q&A @Anonymous .

Data gets pushed to Event hubs. Event hubs does not pull the data for you.
This means you will need to write a producer, and run the producer on a schedule.
Alternatively you could write a long-running producer that waits between batches.

There a number of languages available. For this example, I will build off the python quickstart.

import asyncio  
from azure.eventhub.aio import EventHubProducerClient  
from azure.eventhub import EventData  
  
from time import sleep  
#import urllib  
  
async def run():  
    # Create a producer client to send messages to the event hub.  
    # Specify a connection string to your event hubs namespace and  
    # the event hub name.  
    producer = EventHubProducerClient.from_connection_string(conn_str="EVENT HUBS NAMESPACE - CONNECTION STRING", eventhub_name="EVENT HUB NAME")  
    async with producer:  
        # Create a batch.  
        event_data_batch = await producer.create_batch()  
  
        #get the data  
        #use pycurl http://pycurl.io/  
        #or use liburl  
        # or use requests https://requests.readthedocs.io/en/master/  
        data = request_data(url)  

        # Add events to the batch.  
        for record in data:  
            event_data_batch.add(EventData(record))  
  
        # Send the batch of events to the event hub.  
        await producer.send_batch(event_data_batch)  
  
        # make the process wait for 4 hours between iterations  
        sleep(4*60*60)  
  
loop = asyncio.get_event_loop()  
loop.run_until_complete(run())  

I would also like to note the data you shared above appears to be dimension data defining the regions of world, rather than event data describing the weather. This type of data changes very slowly / infrequently. It would be better used in a reference table in Stream Analytics.

Answer 2

Hello @Anonymous ,

Unfortunately, REST API is not supported as the producer/consumer side in Azure Event Hubs.

For more detail, you have refer the similar threads which address a similar issue:

https://stackoverflow.com/questions/53433372/is-it-possible-to-receive-events-from-azure-event-hubs-using-rest-api

https://stackoverflow.com/questions/32332714/how-to-send-and-consume-messages-in-event-hub-via-rest-api

Hope this helps. Do let us know if you any further queries.

------------

• Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.
• Want a reminder to come back and check responses? Here is how to subscribe to a notification.