Grafana

Grafana

Using kube-prometheus-stack

Values

Grafana Helm Chart Base Values

Values on the Grafana helm chart can be overridden from the parent kube-prometheus-stack chart

Datasources

Useful Datasources

GitHub Datasource

https://github.com/grafana/github-datasource

This datasource can help provide insight into GitHub within Grafana! This is super handy for pull requests and releases ![[Pasted image 20220228121552.png]]

Adding New Datasources

The chart can be tweaked so that the Grafana instance is preloaded with the desired datasources CloudWatch Example

Example

      additionalDataSources:
        - name: Loki
          type: loki
          uid: loki
          access: proxy
          orgId: 1
          url: http://loki-distributed-gateway.loki-distributed
          basicAuth: false
          isDefault: false
          version: 1
          editable: false
          jsonData:
            derivedFields:
            - datasourceUid: my-tempo
              matcherRegex: 'trace_id\":\"(\w+)\"'
              url: '$${__value.raw}'
              name: TraceID
        - name: Tempo
          type: tempo
          uid: my-tempo
          access: proxy
          url: http://tempo-distributed-query-frontend.tempo:3100
          isDefault: false
          version: 1
          editable: false
          basicAuth: false

Dashboards

Transforming

To remove certain columns, use Transform and select Filter by name to remove columns that aren't desired

Variables

Query

CloudWatch

To query for specific resource ARNs run the following

resource_arns(default, rds:db, {"environment": ["$environment"]})

A regex can be performed as well to tidy up the ARN and extract just the resource itself

/.*db:([^$]*)/

A list of some available resources

• rds:db

• elasticache:cluster

Prometheus

To get the label values for all deployments, run the following

label_values(kube_deployment_labels{namespace="portal"}, deployment)