Persist all releated to exceptions metrics, while sampling everything else
We are trying to collect all metrics related to exceptions (request, dependency, trace) without sampling) without sampling, to provide a better debug experience.
According to https://learn.microsoft.com/en-us/azure/azure-monitor/app/sampling-classic-api
It seems like it is enough to customise Exception event sampling rate.
However))
What we have tried:
Try 1
- Fixed-rate sampling for exceptions set to 99.9999 (internal SDK stuff which ignore 100)
- Disabled sampling for other metrics (relying on ingesting)
- Ingesting (AI level sampling) set to 2%.
Results
Every Exception Request Trace and Dependency are visible in 'Live Metrics'
Every Exception is persisted and is visible in the Transaction search
Only 2% of related metrics are visible in the Transaction search which affects End-To-End transactions (you are able to see the expected full request history only in this 2% cases)
And official monitor request against the log shows that.
union requests,dependencies,pageViews,browserTimings,exceptions,traces
| where timestamp > ago(1d)
| summarize RetainedPercentage = 100/avg(itemCount) by bin(timestamp, 1h), itemType
Try 2
- Default adaptive sampling
- Custome ITelemetryInitializer
public class ExceptionTelemetrySamplingFilter : ITelemetryInitializer
{
public void Initialize(ITelemetry telemetry)
{
if (telemetry is ExceptionTelemetry)
{
((ISupportSampling)telemetry).SamplingPercentage = 99.99999;
}
}
}
- Ingesting (AI level sampling) set to 2%.
The results are the same as for try 1.
Try 3
- Fixed-rate sampling for exceptions set to 99.9999 (internal SDK stuff which ignores 100)
- Fixed-rate sampling for others set to 2 (prevent ingesting)
The result is the same as for previous, except that now in 'Live Metrics' we see only Exceptions and others with 2% chance.
Any suggestions so it can be achieved?