(DEPRECATED) Monitor an Azure Container Service cluster with ELK

  • Article

Warning

The Azure Container Service (ACS) is being deprecated. No new features or functionality are being added to ACS. All of the APIs, portal experience, CLI commands and documentation are marked as deprecated.

For more information, see the Azure Container Service deprecation announcement on Azure.com.

We recommend that you deploy one of the following Azure Marketplace solutions:

If you want to use Kubernetes, see Azure Kubernetes Service.

In this article, we demonstrate how to deploy the ELK (Elasticsearch, Logstash, Kibana) stack on a DC/OS cluster in Azure Container Service.

Prerequisites

Deploy and connect a DC/OS cluster configured by Azure Container Service. Explore the DC/OS dashboard and Marathon services here. Also install the Marathon Load Balancer.

ELK (Elasticsearch, Logstash, Kibana)

ELK stack is a combination of Elasticsearch, Logstash, and Kibana that provides an end to end stack that can be used to monitor and analyze logs in your cluster.

Configure the ELK stack on a DC/OS cluster

Access your DC/OS UI via https://localhost:80/ Once in the DC/OS UI navigate to Universe. Search and install Elasticsearch, Logstash, and Kibana from the DC/OS Universe and in that specific order. You can learn more about configuration if you go to the Advanced Installation link.

ELK1 ELK2 ELK3

Once the ELK containers and are up and running, you need to enable Kibana to be accessed through Marathon-LB. Navigate to Services > kibana, and click Edit as shown below.

ELK4

Toggle to JSON mode and scroll down to the labels section. You need to add a "HAPROXY_GROUP": "external" entry here as shown below. Once you click Deploy changes, your container restarts.

ELK5

If you want to verify that Kibana is registered as a service in the HAPROXY dashboard, you need to open port 9090 on the agent cluster as HAPROXY runs on port 9090. By default, we open ports 80, 8080, and 443 in the DC/OS agent cluster. Instructions to open a port and provide public assess are provided here.

To access the HAPROXY dashboard, open the Marathon-LB admin interface at: http://$PUBLIC_NODE_IP_ADDRESS:9090/haproxy?stats. Once you navigate to the URL, you should see the HAPROXY dashboard as shown below and you should see a service entry for Kibana.

ELK6

To access the Kibana dashboard, which is deployed on port 5601, you need to open port 5601. Follow instructions here. Then open the Kibana dashboard at: https://localhost:5601.

Next steps