当前位置:网站首页>「开源摘星计划」Loki实现Harbor日志的高效管理
「开源摘星计划」Loki实现Harbor日志的高效管理
2022-07-07 21:46:00 【51CTO】
本文已参与「 开源摘星计划」,欢迎正在阅读的你加入。
前言
在Harbor的
使用Loki实现Harbor日志的管理,在Grafana页面查询Harbor的日志。
环境:
已通过Helm在Kubernetes集群中安装Harbor,Harbor服务部署在Harbor命名空间中。
Helm部署Harbor的文档请阅读:
Helm部署Harbor,实现高可用的镜像仓库(超详细分享)~后附踩坑记录
https://blog.51cto.com/lidabai/5195706
[[email protected] harbor] # kubectl -n harbor get pods
N A M E R E A D Y S T A T U S R E S T A R T S A G E
h a r b o r - c h a r t m u s e u m - 6 8 f 6 8 d 8 8 f f - n q 9 7 s 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - c o r e - 7 5 6 5 b f b 9 c c - w 5 k 2 9 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - d a t a b a s e - 0 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - j o b s e r v i c e - 6 7 4 7 b 4 8 6 d f - v m 9 z s 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - n g i n x - 9 5 6 f f f 9 8 8 - r 8 d p x 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - n o t a r y - s e r v e r - 7 7 6 f 8 5 f 9 c 6 - 4 n l s n 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - n o t a r y - s i g n e r - 7 f 8 9 5 d 5 9 d 6 - k 5 9 5 7 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - p o r t a l - 5 7 6 6 b 7 8 4 c 7 - s n d m z 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - r e d i s - 0 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - r e g i s t r y - 6 b 8 8 c f b 4 6 5 - f c x v n 2 / 2 R u n n i n g 0 2 m 5 2 s
h a r b o r - t r i v y - 0 1 / 1 R u n n i n g 0 2 m 5 2 s
[[email protected] harbor] # kubectl -n harbor get svc
N A M E T Y P E C L U S T E R - I P E X T E R N A L - I P P O R T ( S ) A G E
h a r b o r N o d e P o r t 1 0 . 9 8 . 7 1 . 1 3 7 < n o n e > 8 0: 3 0 0 0 2 / T C P , 4 4 4 3: 3 0 0 0 4 / T C P 3 m 5 s
h a r b o r - c h a r t m u s e u m C l u s t e r I P 1 0 . 1 0 2 . 1 9 4 . 1 4 8 < n o n e > 8 0 / T C P 3 m 5 s
h a r b o r - c o r e C l u s t e r I P 1 0 . 1 0 0 . 8 7 . 1 7 4 < n o n e > 8 0 / T C P 3 m 5 s
h a r b o r - d a t a b a s e C l u s t e r I P 1 0 . 1 0 0 . 7 9 . 7 2 < n o n e > 5 4 3 2 / T C P 3 m 5 s
h a r b o r - j o b s e r v i c e C l u s t e r I P 1 0 . 1 1 1 . 3 3 . 2 3 0 < n o n e > 8 0 / T C P 3 m 5 s
h a r b o r - n o t a r y - s e r v e r C l u s t e r I P 1 0 . 9 7 . 1 4 4 . 2 2 2 < n o n e > 4 4 4 3 / T C P 3 m 5 s
h a r b o r - n o t a r y - s i g n e r C l u s t e r I P 1 0 . 1 1 0 . 1 3 1 . 6 2 < n o n e > 7 8 9 9 / T C P 3 m 5 s
h a r b o r - p o r t a l C l u s t e r I P 1 0 . 1 0 5 . 2 1 3 . 1 4 5 < n o n e > 8 0 / T C P 3 m 5 s
h a r b o r - r e d i s C l u s t e r I P 1 0 . 1 1 0 . 1 8 . 2 1 0 < n o n e > 6 3 7 9 / T C P 3 m 5 s
h a r b o r - r e g i s t r y C l u s t e r I P 1 0 . 1 1 0 . 1 2 9 . 1 6 < n o n e > 5 0 0 0 / T C P , 8 0 8 0 / T C P 3 m 5 s
h a r b o r - t r i v y C l u s t e r I P 1 0 . 1 0 9 . 0 . 1 5 5 < n o n e > 8 0 8 0 / T C P 3 m 5 s
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26.
Loki简介
Loki是 Grafana Labs 团队最新的开源项目,是一个水平可扩展,高可用性,多租户的日志聚合系统。它的设计非常经济高效且易于操作,因为它不会为日志内容编制索引,而是为每个日志流编制一组标签。
与其他日志系统不同,Loki 是围绕仅索引有关日志的元数据的想法构建的:标签(就像 Prometheus 标签一样)。然后,日志数据本身会被压缩并以块的形式存储在对象存储(例如 S3 或 GCS)中,甚至本地存储在文件系统中。小索引和高度压缩的块简化了操作并显着降低了 Loki 的成本。
使用Helm部署Loki日志平台
环境准备
1)创建命名空间
2)安装Helm
如果你已经安装好helm了,可以忽略本步骤。
[[email protected] ~] # wget https://get.helm.sh/helm-v3.7.2-linux-amd64.tar.gz
[[email protected] ~] # tar zxvf helm-v3.7.2-linux-amd64.tar.gz
[[email protected] ~] # cp linux-amd64/helm /usr/local/bin/
[[email protected] ~] # helm versio
v e r s i o n . B u i l d I n f o { V e r s i o n: " v 3 . 7 . 2 " , G i t C o m m i t: " 6 6 3 a 8 9 6 f 4 a 8 1 5 0 5 3 4 4 5 e e c 4 1 5 3 6 7 7 d d c 2 4 a 0 a 3 6 1 " ,
G i t T r e e S t a t e: " c l e a n " , G o V e r s i o n: " g o 1 . 1 6 . 1 0 " }
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
3)添加Chart仓库
[[email protected] ~] # helm repo add grafana https://grafana.github.io/helm-charts
[[email protected] ~] # helm repo list
N A M E U R L
g r a f a n a h t t p s: / / g r a f a n a . g i t h u b . i o / h e l m - c h a r t s
[[email protected] ~] # helm repo update
- 1.
- 2.
- 3.
- 4.
- 5.
4)搜索chart
[[email protected] ~] # helm search repo grafana
N A M E C H A R T V E R S I O N A P P V E R S I O N D E S C R I P T I O N
g r a f a n a / g r a f a n a 6 . 3 2 . 1 9 . 0 . 2 T h e l e a d i n g t o o l f o r q u e r y i n g a n d v i s u a l i z i n g t . . .
g r a f a n a / g r a f a n a - a g e n t - o p e r a t o r 0 . 2 . 2 0 . 2 5 . 1 A H e l m c h a r t f o r G r a f a n a A g e n t O p e r a t o r
g r a f a n a / e n t e r p r i s e - l o g s 2 . 2 . 2 v 1 . 4 . 1 G r a f a n a E n t e r p r i s e L o g s
g r a f a n a / e n t e r p r i s e - l o g s - s i m p l e 1 . 2 . 0 v 1 . 4 . 0 D E P R E C A T E D G r a f a n a E n t e r p r i s e L o g s ( S i m p l e S c a l . . .
g r a f a n a / e n t e r p r i s e - m e t r i c s 1 . 9 . 0 v 1 . 7 . 0 D E P R E C A T E D G r a f a n a E n t e r p r i s e M e t r i c s
g r a f a n a / f l u e n t - b i t 2 . 3 . 1 v 2 . 1 . 0 U s e s f l u e n t - b i t L o k i g o p l u g i n f o r g a t h e r i n g l o . . .
g r a f a n a / l o k i 2 . 1 2 . 2 v 2 . 5 . 0 L o k i: l i k e P r o m e t h e u s , b u t f o r l o g s .
g r a f a n a / l o k i - c a n a r y 0 . 8 . 1 2 . 5 . 0 H e l m c h a r t f o r G r a f a n a L o k i C a n a r y
g r a f a n a / l o k i - d i s t r i b u t e d 0 . 5 0 . 0 2 . 5 . 0 H e l m c h a r t f o r G r a f a n a L o k i i n m i c r o s e r v i c e s m o d e
g r a f a n a / l o k i - s i m p l e - s c a l a b l e 1 . 6 . 1 2 . 5 . 0 H e l m c h a r t f o r G r a f a n a L o k i i n s i m p l e , s c a l a b l e . . .
g r a f a n a / l o k i - s t a c k 2 . 6 . 5 v 2 . 4 . 2 L o k i: l i k e P r o m e t h e u s , b u t f o r l o g s .
g r a f a n a / m i m i r - d i s t r i b u t e d 2 . 1 . 0 2 . 1 . 0 G r a f a n a M i m i r
g r a f a n a / m i m i r - o p e n s h i f t - e x p e r i m e n t a l 2 . 1 . 0 2 . 0 . 0 G r a f a n a M i m i r o n O p e n S h i f t E x p e r i m e n t
g r a f a n a / o n c a l l 1 . 0 . 2 v 1 . 0 . 3 D e v e l o p e r - f r i e n d l y i n c i d e n t r e s p o n s e w i t h b r i l l . . .
g r a f a n a / p r o m t a i l 6 . 0 . 2 2 . 5 . 0 P r o m t a i l i s a n a g e n t w h i c h s h i p s t h e c o n t e n t s o . . .
g r a f a n a / r o l l o u t - o p e r a t o r 0 . 1 . 2 v 0 . 1 . 1 G r a f a n a r o l l o u t - o p e r a t o r
g r a f a n a / t e m p o 0 . 1 5 . 4 1 . 4 . 1 G r a f a n a T e m p o S i n g l e B i n a r y M o d e
g r a f a n a / t e m p o - d i s t r i b u t e d 0 . 2 0 . 3 1 . 4 . 1 G r a f a n a T e m p o i n M i c r o S e r v i c e m o d e
g r a f a n a / t e m p o - v u l t u r e 0 . 2 . 0 1 . 3 . 0 G r a f a n a T e m p o V u l t u r e - A t o o l t o m o n i t o r T e m p o . . .
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
grafana/loki-stack: 将多个进程封装到一个Pod内,未做数据持久化,适用于测试环境,单体模式。
grafana/loki-canary: 金丝雀更新模式;
grafana/loki-distributed: 微服务模式,适合生产较大规模场景;
grafana/loki-simple-scalable: 读写分离模式,简单可扩展;
下载并安装Loki Chart
- 下载解压chart
[[email protected] ~] # helm pull grafana/loki-stack --untar
[[email protected] ~] # cd loki-stack/
[[email protected] loki-stack] # ls
c h a r t s C h a r t . y a m l R E A D M E . m d r e q u i r e m e n t s . l o c k r e q u i r e m e n t s . y a m l t e m p l a t e s v a l u e s . y a m l
- 1.
- 2.
- 3.
- 4.
--untar:下载并解压Chart包;
- 修改values.yaml配置
[[email protected] loki-stack] # vim values.yaml
g r a f a n a:
e n a b l e d: t r u e # 启 用 G r a f a n a 组 件
s i d e c a r:
d a t a s o u r c e s:
e n a b l e d: t r u e
m a x L i n e s: 1 0 0 0
i m a g e:
t a g: 8 . 3 . 5
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 安装
[[email protected] loki-stack] # helm install loki-stack . -n loki
N A M E: l o k i - s t a c k
L A S T D E P L O Y E D: T h u J u l 7 1 3: 3 1: 0 2 2 0 2 2
N A M E S P A C E: l o k i
S T A T U S: d e p l o y e d
R E V I S I O N: 1
N O T E S:
T h e L o k i s t a c k h a s b e e n d e p l o y e d t o y o u r c l u s t e r .
L o k i c a n n o w b e a d d e d a s a d a t a s o u r c e i n G r a f a n a .
S e e h t t p: / / d o c s . g r a f a n a . o r g / f e a t u r e s / d a t a s o u r c e s / l o k i / f o r m o r e d e t a i l .
[[email protected] loki-stack] # helm -n loki ls
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
查看验证服务
[[email protected] loki-stack] # kubectl -n loki get pods
N A M E R E A D Y S T A T U S R E S T A R T S A G E
l o k i - s t a c k - 0 1 / 1 R u n n i n g 0 7 4 s
l o k i - s t a c k - g r a f a n a - 6 9 9 6 6 b c f 9 4 - l p 4 z 5 2 / 2 R u n n i n g 0 7 4 s
l o k i - s t a c k - p r o m t a i l - c k p s h 1 / 1 R u n n i n g 0 7 4 s
l o k i - s t a c k - p r o m t a i l - g 7 k f t 1 / 1 R u n n i n g 0 7 4 s
l o k i - s t a c k - p r o m t a i l - g f 8 6 b 1 / 1 R u n n i n g 0 7 4 s
l o k i - s t a c k - p r o m t a i l - j n z n c 1 / 1 R u n n i n g 0 7 4 s
n f s - p r o v i s i o n e r - 5 6 f d 4 7 f c 9 c - l k t r v 1 / 1 R u n n i n g 3 2 3 h
[[email protected] loki-stack] # kubectl -n loki get svc
N A M E T Y P E C L U S T E R - I P E X T E R N A L - I P P O R T ( S ) A G E
l o k i - s t a c k C l u s t e r I P 1 0 . 1 0 2 . 8 1 . 2 5 0 < n o n e > 3 1 0 0 / T C P 7 6 s
l o k i - s t a c k - g r a f a n a C l u s t e r I P 1 0 . 1 0 3 . 6 2 . 3 1 < n o n e > 8 0 / T C P 7 6 s
l o k i - s t a c k - h e a d l e s s C l u s t e r I P N o n e < n o n e > 3 1 0 0 / T C P 7 6 s
l o k i - s t a c k - m e m b e r l i s t C l u s t e r I P N o n e < n o n e > 7 9 4 6 / T C P 7 6 s
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
Pod都部署成功!
- 修改Grafana服务访问方式为NodePort
[[email protected] loki-stack] # kubectl -n loki edit svc loki-stack-grafana
s p e c:
t y p e: N o d e P o r t
- 1.
- 2.
- 3.
Grafana配置
登录Grafana
[[email protected] loki-stack] # kubectl -n loki get svc loki-stack-grafana
N A M E T Y P E C L U S T E R - I P E X T E R N A L - I P P O R T ( S ) A G E
l o k i - s t a c k - g r a f a n a N o d e P o r t 1 0 . 1 0 3 . 6 2 . 3 1 < n o n e > 8 0: 3 2 5 5 5 / T C P 4 m 9 s
- 1.
- 2.
- 3.
查看Grafana密码
浏览器登录Grafana
- 登录地址: k8s节点IP:NodePort
- 用户名:admin
- 密码: 刚才查看到的secret解析串
Grafana查看Loki日志
然后在输入: {namespace="harbor"} 即可查看Harbor命名空间的所有日志。
边栏推荐
- 谈谈制造企业如何制定敏捷的数字化转型策略
- Px4 autonomous flight
- 【Azure微服务 Service Fabric 】如何转移Service Fabric集群中的种子节点(Seed Node)
- Main functions of OS, Sys and random Standard Libraries
- Remove the default background color of chrome input input box
- Matplotlib快速入门
- What is the difference between the three values of null Nan undefined in JS
- UWA问答精选
- Ternary expressions, generative expressions, anonymous functions
- PHP method of obtaining image information
猜你喜欢
Matplotlib quick start
IP network active evaluation system -- x-vision
使用 BlocConsumer 同时构建响应式组件和监听状态
The PHP source code of the new website + remove authorization / support burning goose instead of pumping
. Net automapper use
客户案例|华律网,通过观测云大幅缩短故障定位时间
微服务远程Debug,Nocalhost + Rainbond微服务开发第二弹
【Azure微服务 Service Fabric 】因证书过期导致Service Fabric集群挂掉(升级无法完成,节点不可用)
[azure microservice service fabric] start the performance monitor in the SF node and set the method of capturing the process
Matplotlib快速入门
随机推荐
php 获取图片信息的方法
强化学习-学习笔记9 | Multi-Step-TD-Target
Pre sale 179000, hengchi 5 can fire? Product power online depends on how it is sold
[open source] Net ORM accessing Firebird database
Relationship between URL and URI
operator
Use json Stringify() to realize deep copy, be careful, there may be a huge hole
Remove the default background color of chrome input input box
Paint basic graphics with custompaint
变量与常量
C development - interprocess communication - named pipeline
Unity development --- the mouse controls the camera to move, rotate and zoom
Revit secondary development - project file to family file
Revit secondary development - intercept project error / warning pop-up
Revit secondary development - get the project file path
微服务架构开源框架详情介绍
100million single men and women "online dating", supporting 13billion IPOs
Remember an experience of using selectmany
The PHP source code of the new website + remove authorization / support burning goose instead of pumping
Antd date component appears in English