Data Sources

Data Sources are the foundation of monitoring in SRExpert. They connect your monitoring backends (like Prometheus) to the platform, enabling metrics visualization, alerting, and observability features.

Prerequisites: Before configuring a Data Source, you need to have Prometheus installed in your cluster. You can easily install it using the Helm Charts feature in SRExpert - just search for kube-prometheus-stack and click Install.

Overview

The Data Sources page displays:

• Total Sources: Number of configured data sources
• Connected: Data sources with successful connections
• Disconnected: Data sources with connection issues
• Health: Overall health percentage of your data sources

Adding a Data Source

Step 1: Click Add Data Source

Click the “Add Data Source” button in the top right corner, or “Add Your First Data Source” if you haven’t configured any yet.

Step 2: Configure the Connection

Fill in the required fields:

Understanding the URL Format

The URL format for Kubernetes services follows this pattern:

http://<service-name>.<namespace>:<port>

How to find each component:

1. Service Name: Run kubectl get svc -n <namespace> to list services
2. Namespace: The Kubernetes namespace where Prometheus is deployed (commonly monitoring)
3. Port: The service port (default for Prometheus is 9090)

Common examples:

# Prometheus Operator / kube-prometheus-stack
http://kube-prometheus-stack-prometheus.monitoring:9090
 
# Standalone Prometheus
http://prometheus-server.monitoring:9090
 
# Custom installation
http://my-prometheus.observability:9090

Finding the correct URL via kubectl:

# List services in the monitoring namespace
kubectl get svc -n monitoring
 
# Example output:
# NAME                                  TYPE        CLUSTER-IP      PORT(S)
# kube-prometheus-stack-prometheus      ClusterIP   10.96.100.50    9090/TCP
 
# Your URL would be:
# http://kube-prometheus-stack-prometheus.monitoring:9090

Step 3: Test the Connection

Before saving, always click “Test Connection” to verify:

• ✅ Connection successful! - Your configuration is correct
• ❌ Connection failed - Check your URL, network policies, and service availability

Common connection issues:

Step 4: Save and Verify

After a successful connection test, click “Save” to add the data source.

Your data source will now appear in the list with its connection status.

Managing Data Sources

Editing a Data Source

1. Click on the data source name in the list
2. Modify the configuration
3. Test the connection
4. Save changes

Deleting a Data Source

1. Click the three dots menu (⋮) on the data source row
2. Select Delete
3. Confirm the deletion

Warning: Deleting a data source will remove all associated dashboards and alerts that depend on it.

Supported Data Sources

Currently, SRExpert supports:

Coming soon:

• Grafana
• InfluxDB
• Datadog
• And more…

Best Practices

1. Use descriptive names: Name your data sources clearly (e.g., prod-cluster-prometheus, staging-metrics)

2. Test before saving: Always verify the connection before saving

3. Monitor health: Regularly check the Data Sources page for connection issues

4. Use internal DNS: When possible, use Kubernetes internal DNS (service.namespace:port) instead of external URLs

Troubleshooting

Cannot connect to Prometheus

1. Verify Prometheus is running:

   kubectl get pods -n monitoring -l app=prometheus

2. Check the service exists:

   kubectl get svc -n monitoring | grep prometheus

3. Test connectivity from another pod:

   kubectl run curl-test --image=curlimages/curl -it --rm -- \
     curl http://prometheus.monitoring:9090/-/healthy

Connection drops intermittently

• Check if Prometheus pods are being restarted
• Verify resource limits on Prometheus
• Check for network policy changes

Next Steps

• Monitoring Overview - View your cluster metrics
• Alerts & Notifications - Set up alerting based on metrics
• Custom Dashboards - Create custom metric dashboards