question

SandeepYarashi-3149 avatar image
0 Votes"
SandeepYarashi-3149 asked srbose-msft edited

Cannot access prometheus and grafana from browser

I have aks,appgw , sample webapp ,public ip deployed in azure. In aks I installed prometheus and grafana using helm.
I can curl -L http:localhost.com:9090 (prometheus port -9090) in aks but can't access localhost:9090 in browser as my laptop and aks is in different network. I want to access prometheus and grafana with public IP of app gateway.
For eg:public ip is 10.0.2.2 I want to access prometheus by 10.0.2.2/prometheus and
access grafana by 10.0.2.2/ grafana. How to achieve this.

azure-kubernetes-serviceazure-application-gateway
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

srbose-msft avatar image
1 Vote"
srbose-msft answered srbose-msft edited

@SandeepYarashi-3149 , Thank you for your question.

This sounds like a perfect use case of an ingress controller (in your case it would be best to use an Application Gateway ingress Controller) for path based routing of prometheus and grafana services running on the AKS cluster.

The ingress controller must be deployed before installing grafana and prometheus.

For Grafana Helm values must have:

 grafana:
   ingress:
     enabled: true
     annotations:
       kubernetes.io/ingress.class: "nginx"
       nginx.ingress.kubernetes.io/rewrite-target: /
       nginx.ingress.kubernetes.io/backend-protocol: "HTTP"
     paths:
     - /grafana

For prometheus Helm values must have:

 prometheus:
   enabled: true
   service:
     nodePort: 9090
     type: NodePort
   ingress:
     enabled: true
     annotations:
       kubernetes.io/ingress.class: nginx
       nginx.ingress.kubernetes.io/rewrite-target: /
       nginx.ingress.kubernetes.io/backend-protocol: "HTTP"
     paths:
     - /prometheus

Note:

I can curl -L http:localhost.com:9090 (prometheus port -9090) in aks but can't access localhost:9090 in browser as my laptop and aks is in different network.

The kubectl proxy:

- runs on a user's desktop or in a pod
- proxies from a localhost address to the Kubernetes apiserver
- client to proxy uses HTTP
- proxy to apiserver uses HTTPS
- locates apiserver
- adds authentication headers

Without a public IP address this can be used to access services running on the cluster from your workstation machine which is in a different network.


Hope this helps.

Please "Accept as Answer" if it helped, so that it can help others in the community looking for help on similar topics.


· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Can you explain this in a step by step process to make it work. I am unable to get it working.

0 Votes 0 ·

@SandeepYarashi-3149 , the Helm values I mentioned earlier would suffice for a basic ingress controller. Thank you for choosing Application Gateway Ingress Controller. I missed adding the following information in my earlier answer. For an Ingress resource to be observed by AGIC, it must be annotated with kubernetes.io/ingress.class: azure/application-gateway. Only then AGIC will work with the Ingress resource in question. [Reference]

  • Deploy AGIC on an AKS cluster via AKS add-on (either Greenfield or Brownfield)
    [[contd in next comment]]

0 Votes 0 ·

@SandeepYarashi-3149 ,

Create a grafana-values.yaml with:

 grafana:
    ingress:
      enabled: true
      annotations:
        kubernetes.io/ingress.class: azure/application-gateway
        kubernetes.io/ingress.class: "nginx"
        nginx.ingress.kubernetes.io/rewrite-target: /
        appgw.ingress.kubernetes.io/backend-protocol: http
      paths:
      - /grafana

Create a prometheus-values.yaml with:

 prometheus:
    enabled: true
    service:
      nodePort: 9090
      type: NodePort
    ingress:
      enabled: true
      annotations:
        kubernetes.io/ingress.class: azure/application-gateway
        kubernetes.io/ingress.class: nginx
        nginx.ingress.kubernetes.io/rewrite-target: /
        appgw.ingress.kubernetes.io/backend-protocol: http
      paths:
      - /prometheus

Install grafana and prometheus helm charts respectively as follows:

  • helm install -n <namespace> install -f grafana-values.yaml <grafana-installation-name> <grafana-helm-repository>

  • helm install -n <namespace> install -f prometheus-values.yaml <prometheus-installation-name> <prometehus-helm-repository>

Hope this helps. Please do let us know if you are facing any challenges.

0 Votes 0 ·