Spark has a configurable metrics system based on the Dropwizard Metrics Library. This allows users to report Spark metrics to a variety of sinks including HTTP, JMX, and CSV files. I know that Spark on HDInsight can be connected to a Log Analytics Workspace and Application Insights. But it looks there is only a subset of the metrics available that Spark provides. How can I configure Spark on HDInsight to customize the metrics that are sent to Log Analytics and Application Insights. What sink of dropwizard is HDInsight actually using? How is it connected?