I have a database table that will provide a large number of ids (500k). For each id I will need to make a REST api call (to Yammer), extract a value from the response JSON and then update the source table against the id with the extracted value.
id1 : value1
id2 : value2
id3 : value3
What kind of activities and configurations will I need to use to make it simple, fast and efficient? I am fairly new to Azure and learning about lookup and foreach activities. I see that lookup activity has a 5000 row limit.
How can I chunk the data read from the database?
How can I update the row in the table for each id? Is stored procedure activity the only way? Is it efficient to call a stored procedure for thousands of times?