Azure Notification Hubs - difficult to decipher metrics

Question

Hi,

My team develops and maintains a solution for a customer with technologies such as Azure Functions/Durable Functions, Cosmos DB, Azure Service Bus, .NET based APIs and Notification Hubs. We use templates for sending push notifications and there are several templates per device (iOS/Android). We use both APNS and GCM/FCM. Pricing tier is basic.

I am trying to understand what the different measurement types represent when I use Monitor | Metrics. I've looked at https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/metrics-supported#microsoftnotificationhubsnamespacesnotificationhubs but not getting much wiser.

My basic task is to ensure that all messages sent to Notification Hubs also reach the devices. I do not see any errors appearing in the graph. It says 0 in, for example, "All incoming Failed Requests" and "APNS/GCM Errors". I also see that "All Outgoing Notifications" usually have the same value as "Successful notifications".

Questions:

1. What does "All incoming Requests" mean?
2. And how does it differ from "Incoming Messages"?
3. How should I compare the above with "All Outgoing Notifications"?
4. If you see a difference between "All Outgoing Notifications" and "Successful notifications" (in both directions), is there any way to find out the notifications that are failing or may be ongoing?
5. If you get an error in, for example, "APNS Errors", is there any way to find out what went wrong?

Below is an example from our Azure environment that shows:

• "All Incoming Requests" differ from "Incoming Messages".
• "Incoming Messages" are different from "All Outgoing Notifications".
• "Incoming Messages" + "Installations Management Operations" are not exactly the same as "All Incoming Requests".
  

Best regards,

Crippe

Answer 1

@Christer Alm ,

Following-up to answer your 5 questions, after some background of the Notification Hub concepts (in the comments section):

As mentioned in the comments -Notification hub is a single push resource holds cross-platforms (Android, iOS, and Windows versions) push information of one app. - 'All incoming Requests' – Is the total incoming requests for a notification hub (all platforms).

1. What does "All incoming Requests" mean?
A. This is the number of API calls for the given hub (send, installation, etc).

2. And how does it differ from "Incoming Messages"?
A. This is the number of send API calls for the hub ( is all send API calls)

3. How should I compare the above with "All Outgoing Notifications"?
A. Outgoing notifications is the number of pushes we sent to the PNS.
This will not match “Incoming Messages” as this is the number of target devices, not the number of send API calls.

So, in general it will not match Incoming Messages unless each send request only targets a single device

4. If you see a difference between "All Outgoing Notifications" and "Successful notifications" (in both directions), is there any way to find out the notifications that are failing or may be ongoing?
A. For namespaces which are Standard SKU, we provide telemetry/feedback via the following APIs:
https://learn.microsoft.com/en-us/rest/api/notificationhubs/get-notification-message-telemetry
https://learn.microsoft.com/en-us/rest/api/notificationhubs/get-pns-feedback

5. If you get an error in, for example, "APNS Errors", is there any way to find out what went wrong?
A. See answer to #4 above

Summarizing my points from the comments section:

Successful notifications mean simply that push notifications have been delivered to the external PNS (for example, APNs for iOS and macOS or FCM for Android devices). It is the responsibility of the PNS to deliver the notifications to target devices.

Push notifications depend on third-party Platform Notification Systems such (APNs) and (FCM), there is no SLA guarantee for the delivery of these messages. After Notification Hubs sends the batches to Platform Notification Systems (SLA guaranteed), it is the responsibility of the Platform Notification Systems to deliver the pushes (no SLA guaranteed).

In regard to delivery of the message, Notification delivery can fail at any of the four stages in the push notification process (client, application back end, Notification Hubs, and the platform's push notification service).

Diagnose dropped notifications in Azure Notification Hubs -This article identifies why notifications might get dropped or not be received by devices. It also explains how to determine the root cause.

Hope this helps! Thanks for your feedback.

Answer 2

Hi @ajkuma

Thank you for very good comments/answers. It is clear that you want to help the community. The plan to divide comments based on how much you know at any given time is especially appreciated!

Now to some follow-up questions. I think it's the vocabulary that I have a hard time grasping.

1 and 2
Please see the new point 6 first.
a) What is the difference between "number of API calls" and "number of send API calls"?
b) Is the difference between point 1 and 2 that the installation management is included in 1?

2 and 3
If I understand correctly, the data in 2 and 3 is the same, ie the same notification text. I understand that a notification can be sent to several devices. But, shouldn't the vocabulary be the same? Shouldn't it be called "All Incoming Notifications" instead of "Incoming Messages"? (Then you can avoid the distortion between notification and message.)

6
The word send, what does it mean? Is that what I send to Notification Hubs? Ie the calls I make via SDKs, applications like PostMan etc? Or can you consider "send" both from outside and inside NH? Is it possible to explain a little more what send means, especially based on incoming and outgoing traffic.

Br,
Crippe