VictoriaMetrics

VictoriaMetrics is the monitoring stack that collects, stores, and queries metrics for your Kubernetes environment. The SRE team manages the VictoriaMetrics components (including vmagent); you only need to tell it which of your own workloads to scrape.

1. Scraping your own metrics

This part explains how to expose your application's metrics and have them collected automatically.

Prerequisites

Your application must:

• Expose metrics in Prometheus exposition format over HTTP (for example on /metrics).
• Have a named container port for the metrics endpoint. VMPodScrape references the port by name, not by number.

Example pod/deployment snippet exposing metrics on a named port:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  namespace: my-namespace
spec:
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
        - name: my-app
          image: my-registry/my-app:1.0.0
          ports:
            - name: metrics      # <-- named port, referenced by the VMPodScrape
              containerPort: 8080

Exposing metrics through a Pod

Create the VMPodScrape resource in your own namespace. It selects the pods to scrape with selector.matchLabels and tells vmagent which port and path to use.

apiVersion: operator.victoriametrics.com/v1beta1
kind: VMPodScrape
metadata:
  name: my-app
  namespace: my-namespace
spec:
  selector:
    matchLabels:
      app: my-app          # must match the labels on your pods
  podMetricsEndpoints:
    - port: metrics        # the named container port from above
      path: /metrics       # defaults to /metrics if omitted
      interval: 30s        # optional, defaults to the global scrape interval

By default a VMPodScrape only selects pods in its own namespace. This is the recommended setup: keep the VMPodScrape in the same namespace as the pods it targets.

Exposing metrics through a Service

If you prefer to scrape via a Kubernetes Service instead of directly from pods, use a VMServiceScrape instead. It works the same way but selects Service objects and references the service port:

apiVersion: operator.victoriametrics.com/v1beta1
kind: VMServiceScrape
metadata:
  name: my-app
  namespace: my-namespace
spec:
  selector:
    matchLabels:
      app: my-app
  endpoints:
    - port: metrics        # name of the Service port
      path: /metrics

Verify your metrics are collected

1. Check that the operator accepted your resource:

kubectl -n my-namespace get vmpodscrape

1. Check the vmagent targets page (/targets), or confirm that your metrics are queryable in Grafana / VictoriaMetrics. Targets normally appear within a minute after the VMPodScrape is applied.

If a target does not appear, the most common causes are:

• selector.matchLabels does not match the actual pod labels.
• The port name in the VMPodScrape does not match the named container port.
• The application is not actually serving metrics on the configured path/port.