question

FelixMller-2336 avatar image
0 Votes"
FelixMller-2336 asked JuliaGoloshubina-4475 commented

Application Insights - Java Agent Custom Metrics by Cloud Role

We are running some containerized Java applications on AKS. We added the Application Insights Java Agent to the Pod and provide the role via APPLICATIONINSIGHTS_ROLE_NAME.

Custom metrics are visible, and the different applications show up with their role name on the application map.

However, we are unable to filter custom metrics, including common JVM metrics, by role or apply splitting by role:

79594-screenshot-20210319-094059.png

When I look at the logs, I see cloud_RoleName and cloud_RoleInstance populated. The Application Insights standard system metrics like "Server requests" can be filtered / grouped by role instance or role, so this seems to be working.

Is filtering and grouping by Cluster Role / Cluster Role Instance generally supposed to work for the metrics collected by the Java agent? If so, how can I check what we are doing wrong?


azure-monitor
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

JuliaGoloshubina-4475 avatar image
0 Votes"
JuliaGoloshubina-4475 answered JuliaGoloshubina-4475 commented

@FelixMller-2336
If you select Log Based metrics under the Metric Namespace, you should have the same metrics available, and filtering/splitting applied. Let me know if it works for you!

· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @JuliaGoloshubina-4475 ,

wow you are correct! I can split and filter by own metrics by the usual dimensions when I select the "Log-Based Metrics" namespace.

I am just a little bit confused because I thought that the goal of sending custom metrics to Application Insights was to circumvent the log data store. Do I need to worry?

Have a great day!

0 Votes 0 ·

@JuliaGoloshubina-4475 I am attaching some screenshots to illustrate the phenomenon. Perhaps I misunderstood the documentation java-in-process-agent

81717-doku-custommetrics.png

What I am noticing is that the application metric ingester_totals shows up twice:

  1. "Standard / Log-Based Metrics" with filtering and grouping options. Interestingly, the Metric ID reads "customMetrics/ingester_errors" 81718-menu-logbased-namespace.png

2."Custom / azure.applicationinsights" without these capabilities. The Metric ID reads "ingester_errors" 81708-menu-appi-namespace.png

Is this how the collection of custom metrics via Java agent is supposed to work for one metric?


0 Votes 0 ·

@FelixMller-2336

I apologize for the delay! Took time the described behavior.

My findings:
1. Seeing the same metrics in both log-based and custom metrics namespaces is by design. It is so because all Application Insights telemetry always goes to Log Analytics, making log-based metrics, but to enable alerting, we also push a copy of each metric (with or without dimensions, as you decide - please follow the instructions to enable the collection of dimensions on custom metrics, this will also let you split and filter from the custom metrics namespace see the doc here
2. Regarding the pricing and extra costs - the log-based metrics are not going to generate any extra cost. The custom-based metrics are priced extra - see the doc here, scroll down to "Metrics" section.

1 Vote 1 ·