anyone had issues yesterday with Teams Call Subscription getting old hits and callRecords

Drako 141

Hi,
We subscribed to teams subscription using: https://graph.microsoft.com/beta/subscriptions/XXXXX
This works fine. We actually do a patch to this subscription so it does not expire. Just change the expiration date. All good here. We have been doing this for some time without issues.
The odd thing is that yesterday we started getting calls to our subscribed method from teams calls that were repeated and some I even think are not ours. I think. For example we have a meeting that has 166 people and all are external users.... we have never had a meeting of this magnitude. I was not able to find in the team admin web page where I can find a team ID call to validate.
For every call we receive from the subscription we call the "https://graph.microsoft.com/beta/communications/callRecords/IDcall", this works fine and returns the details of the call. Curiously yesterday we started also getting the response "404", call not found. Even more odd as we are receiving the call from the subscription...so how can it not exist? Then we saw were where getting multiple call from the same meeting...repeated....repeated....

Did something happen yesterday? Did anyone see this behavior?

Hunaid Hanfee-MSFT 976 Reputation points

2021-09-08T16:37:35.72+00:00

Hello,
APIs under the /beta version in Microsoft Graph are subject to change. Use of these APIs in production applications is not supported. I would suggest you to use v1.0 instead of using beta APIs and let us know if you are facing the same with v1.0.

Here is the reference of v1.0 subscription - link and callRecords - link

I am looking internally for the issue you are facing with beta but you should use v1.0 as it is more reliable.
Drako 141 Reputation points

2021-09-08T16:44:36.54+00:00

I'll change to V1 but I think I had made the change before with no results...still will chnage it back to 1 and wait 24 hours to see the results....
Drako 141 Reputation points

2021-09-08T17:02:28.807+00:00

I just ran the Patch for the subscription to update expiration date and got this result

{
"@odata.context": "https://graph.microsoft.com/v1.0/$metadata#subscriptions/$entity",
"id": "e79379e7-eeca-425c-8019-a97693f4db72",
"resource": "communications/callRecords",
"applicationId": "xxxxxxx",
"changeType": "created",
"clientState": "secretClientValue",
"notificationUrl": "xxxxxxx",
"notificationQueryOptions": null,
"lifecycleNotificationUrl": null,
"expirationDateTime": "2021-09-10T16:47:59.0547852Z",
"creatorId": "xxxxxxx",
"includeResourceData": null,
"latestSupportedTlsVersion": "v1_2",
"encryptionCertificate": null,
"encryptionCertificateId": null,
"notificationUrlAppId": null
}

the only thing I see is that it always says created in the change type. I called the update (patch) like this:
https://graph.microsoft.com/v1.0/subscriptions/[mysubscriptionID]

Actually what we do is call this patch everyday at night so we always keep the expiration date active.

Maybe this helps....to reproduce so many calls.... actually we have the call being received (url in subscription) in power automate flow and it is receiving so many calls that we are getting the warning that the Flow will be turned off.

}
Drako 141 Reputation points

2021-09-10T00:07:13.413+00:00

Hi, after the change to non beta version of API I still have the same odd behviour...it is odd... for example I get the call to our subscribed url for call XXX at 8am with all the call info, we save this in SQL with the Team Call unique ID and the participant ID. This helps us keep only one record without duplicates. Then later...during the day we get the call again...same team id and same participant id.... and this happens all day long... what we are expecting is that when the team meeting finishes, we just get called once to our subscribed url with all the team call information. But this is not the case...

Are you guys seeing the same?
Drako 141 Reputation points

2021-09-15T00:48:02.02+00:00

Hi @Hunaid Hanfee-MSFT ,

Has anyone reported anything similar.... Our Power Automate Flow says it will be turned off because of so many calls. We don't know what else to do.
Any insights are welcomed.

Kind Regards
Hunaid Hanfee-MSFT 976 Reputation points

2021-09-15T19:50:56.38+00:00

We didn't receive anything similar.

I am still looking internally for possible reason causing the issue and way to solve this issue. I will update once get know about anything.
Hunaid Hanfee-MSFT 976 Reputation points

2021-09-15T22:10:04.707+00:00

Is call Id coming to you is different always?
Drako 141 Reputation points

2021-09-15T22:47:04.637+00:00

Hi @Hunaid Hanfee-MSFT
What do you mean, if the Call ID is different? If so then this is my answer: we are getting the same Caller ID with same Participant. I know this because in our flow when we get the CALL we check in our SQL if the CALL ID and Participant ID (key) exist in the SQL. If it does not exist we add the record, if it does exists then we skip it. We added this validation because of so many calls we are getting to the subscribed url.
Drako 141 Reputation points

2021-09-16T16:52:22.923+00:00

Hi @Hunaid Hanfee-MSFT --> apart from this forum where can I put a ticket for this item...I tried in Azure and these guys said it was out of scope... where can I put a ticket... this issue is affecting my power Automate flow affecting company data and $$.
Hunaid Hanfee-MSFT 976 Reputation points

2021-09-16T18:32:59.023+00:00

I have setup app with the call subscription to understand more about the issue. I am trying to repro the issue at my end, so I just want to confirm how I will check if I am getting same call again. Now it is clear.

I have created two subscription one I am check with multiple update(Patch) and the other with just subscription.

I would like you to check one more stuff whenever you got the same call notification, could you please get that call with Id and check for the version in response JSON?

Also if possible can you try deleting the existing subscription and creating a new and try if you are facing issue in that subscription too?

About other forum- This is the right forum to ask question about Teams development, We are looking into your issue and trying repro and resolve it. I will keep you updated. Sorry about the wait but we need to find the exact source of problem and solve it.

Thanks for understanding.
Drako 141 Reputation points

2021-09-16T19:38:33.12+00:00

Hi @Hunaid Hanfee-MSFT ,

Thanks for response and support:
1) Could you please get that call with Id and check for the version in response JSON?
Answer: "version": 2,
2) I will Delete the subscription and create another one.

Drako 141

Hi @Hunaid Hanfee-MSFT ,

I Deleted the subscription and to make sure it was deleted I executed the getlist of subscriptions making sure it was not present. Confirmed deleted. I proceeded to create another Subscription and also updated the process that does the PATCH to keep the expiration date active for the new subscription ID.   

Please let me know if I should do something else. Also see my previous response regarding the version

Kind Regards

Drako 141 Reputation points

2021-09-16T21:53:02.67+00:00

Hi @Hunaid Hanfee-MSFT ,

After a few hours I'm still getting calls records that we have already received. Not sure if this is just some sort of delay or what. I do see that the JSON version of the call now says 3.
Drako 141 Reputation points

2021-09-17T14:49:34.797+00:00
Hi @Hunaid Hanfee-MSFT ,

Some update:

Yesterday we got repeated calls and curiously they were of the subscription we had deleted.

Today I reviewed and we still are getting repeated calls. For example, today we got a call at 6:06am, this was perfect. But then at 8:30:44 AM we got the same call again. Not sure how many times we get the same call called. This call IS from the new subscription.

Could this be that when we patch the subscription it starts catching the calls again so everything starts repeating?
Drako 141 Reputation points

2021-09-17T18:06:29.12+00:00

More info: I checked the version of the CALL information returned and now I have seen some in version1 and others in version 2. I have not found in version 3 but it is tedious to check all the calls

Not really sure what this means.

I
Hunaid Hanfee-MSFT 976 Reputation points

2021-09-17T19:44:31.2+00:00

Thanks for providing this information. In the documentationfor version it states that

Monotonically increasing version of the call record. Higher version call records with the same id includes additional data compared to the lower version.
Drako 141 Reputation points

2021-09-17T20:56:53.07+00:00

Any update on the amoutn of calls and old calls...for example Just 15 minutes ago we recieved the id for a call that is not ours or is so old it doesn't find it. we got this:

{
"error": {
"code": "NotFound",
"message": "Could not find the requested call record. Records about calls or online meetings that started more than 30 days ago are not available.",
"innerError": {
"date": "2021-09-17T20:42:07",
"request-id": "386d6095-4aaf-40f5-89a4-29d2e54450fb",
"client-request-id": "386d6095-4aaf-40f5-89a4-29d2e54450fb"
}
}
}
Drako 141 Reputation points

2021-09-20T18:10:04.45+00:00

Hi @Hunaid Hanfee-MSFT ,

Doesn't the subscription call the URL when the call is competed. I had read this, it is not real time but at the end of the call we get the call. So why would their be any change in the version. I should get the latest information as the call finished and that's it....

Is their a way only to get the call to our subscribed url only ONCE when the call finished and get the last version?
Prasad-MSFT 5,621 Reputation points Microsoft Vendor

2023-05-30T08:54:14.34+00:00

We tried to repro this issue recently by subscribing to /communications/callRecords resource and can confirm that now we are getting unique call records instead of multiple records for the same call. Please do verify

2 answers

Hunaid Hanfee-MSFT 976 Reputation points

2021-09-20T20:53:24.797+00:00

Hello @Drako ,
I tried myself and I was able to repro the issue. We have raised a bug and concerned team is looking into it. Bug is internal doesn't have public accessible URL, I will let you know the updates.

Thanks,
Hunaid Hanfee

If the response is helpful, please click "Accept Answer" and upvote it. You can share your feedback via Microsoft Teams Developer Feedback link. Click here to escalate.
Please sign in to rate this answer.
Drako 141 Reputation points

2021-10-06T18:07:28.493+00:00

Hi @Hunaid Hanfee-MSFT ,

I have a problem with this issue. My Power automate flow will be turned off by MS because it is receiving so many calls from the subscription. Is their a way you can contact the team of Power automate or some email we can share between all of us so they can see the case and consdier an exception until this is solved?

Prasad-MSFT 5,621 Reputation points Microsoft Vendor

2023-05-30T08:53:51.2933333+00:00

We tried to repro this issue recently by subscribing to /communications/callRecords resource and can confirm that now we are getting unique call records instead of multiple records for the same call. Please do verify
Sign in to comment
Deleted

This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

Comments have been turned off. Learn more