Data Source Configuration#
Grafana connects to backend data stores through data sources. For a complete Kubernetes observability stack, you need three: Prometheus for metrics, Loki for logs, and Tempo for traces.
Provision data sources declaratively so they survive Grafana restarts and are version-controlled:
# grafana/provisioning/datasources/observability.yml
apiVersion: 1
datasources:
- name: Prometheus
type: prometheus
access: proxy
url: http://prometheus-operated:9090
isDefault: true
jsonData:
timeInterval: "15s"
exemplarTraceIdDestinations:
- name: traceID
datasourceUid: tempo
- name: Loki
type: loki
access: proxy
url: http://loki-gateway:3100
jsonData:
derivedFields:
- name: TraceID
matcherRegex: '"traceID":"(\w+)"'
url: "$${__value.raw}"
datasourceUid: tempo
- name: Tempo
type: tempo
access: proxy
url: http://tempo:3100
jsonData:
tracesToMetrics:
datasourceUid: prometheus
tags: [{key: "service.name", value: "job"}]
serviceMap:
datasourceUid: prometheus
nodeGraph:
enabled: trueThe cross-linking configuration lets you click from a metric data point to the trace that generated it, and extract trace IDs from log lines to link to Tempo.