「Grafana / Loki / Promtail」- 常见问题处理

节点日志无法被抓取

问题描述:
某个节点的容器日志无法被正常抓取;

环境信息:
Kubernetes v1.22.15, Promtail 5.0.0, Loki 2.5.0, Grafana 8.5.3

原因分析:
Bug: Promtail log is reporting error for non-existent symlink · Issue #2513 · grafana/loki
promtail logs no such file or directory · Issue #429 · grafana/loki

解决方案:
重启该节点的 Promtail 服务;

… HTTP status 429 Too Many Requests …

how can I increase the maximum active stream limit? · Issue #3335 · grafana/loki · GitHub

问题描述:

level=warn ts=2022-11-10T05:10:49.533015596Z caller=client.go:349 component=client
 host=loki-loki-distributed-gateway.loki.svc.cluster.local msg="error sending batch, will retry" 
 status=429 error="server returned HTTP status 429 Too Many Requests (429): Maximum 
 active stream limit exceeded, reduce the number of active streams (reduce labels or 
 reduce label values), or contact your Loki administrator to see if the limit can be increased"

解决方案:
修改 limits_config.max_streams_per_user: 0 配置;