当前位置:网站首页>「开源摘星计划」Loki实现Harbor日志的高效管理
「开源摘星计划」Loki实现Harbor日志的高效管理
2022-07-07 21:46:00 【51CTO】
本文已参与「 开源摘星计划」,欢迎正在阅读的你加入。
前言
在Harbor的
使用Loki实现Harbor日志的管理,在Grafana页面查询Harbor的日志。
环境:
已通过Helm在Kubernetes集群中安装Harbor,Harbor服务部署在Harbor命名空间中。
Helm部署Harbor的文档请阅读:
Helm部署Harbor,实现高可用的镜像仓库(超详细分享)~后附踩坑记录
https://blog.51cto.com/lidabai/5195706
[[email protected] harbor] # kubectl -n harbor get pods
N A M E R E A D Y S T A T U S R E S T A R T S A G E
h a r b o r - c h a r t m u s e u m - 6 8 f 6 8 d 8 8 f f - n q 9 7 s 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - c o r e - 7 5 6 5 b f b 9 c c - w 5 k 2 9 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - d a t a b a s e - 0 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - j o b s e r v i c e - 6 7 4 7 b 4 8 6 d f - v m 9 z s 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - n g i n x - 9 5 6 f f f 9 8 8 - r 8 d p x 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - n o t a r y - s e r v e r - 7 7 6 f 8 5 f 9 c 6 - 4 n l s n 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - n o t a r y - s i g n e r - 7 f 8 9 5 d 5 9 d 6 - k 5 9 5 7 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - p o r t a l - 5 7 6 6 b 7 8 4 c 7 - s n d m z 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - r e d i s - 0 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - r e g i s t r y - 6 b 8 8 c f b 4 6 5 - f c x v n 2 / 2 R u n n i n g 0 2 m 5 2 s
h a r b o r - t r i v y - 0 1 / 1 R u n n i n g 0 2 m 5 2 s
[[email protected] harbor] # kubectl -n harbor get svc
N A M E T Y P E C L U S T E R - I P E X T E R N A L - I P P O R T ( S ) A G E
h a r b o r N o d e P o r t 1 0 . 9 8 . 7 1 . 1 3 7 < n o n e > 8 0: 3 0 0 0 2 / T C P , 4 4 4 3: 3 0 0 0 4 / T C P 3 m 5 s
h a r b o r - c h a r t m u s e u m C l u s t e r I P 1 0 . 1 0 2 . 1 9 4 . 1 4 8 < n o n e > 8 0 / T C P 3 m 5 s
h a r b o r - c o r e C l u s t e r I P 1 0 . 1 0 0 . 8 7 . 1 7 4 < n o n e > 8 0 / T C P 3 m 5 s
h a r b o r - d a t a b a s e C l u s t e r I P 1 0 . 1 0 0 . 7 9 . 7 2 < n o n e > 5 4 3 2 / T C P 3 m 5 s
h a r b o r - j o b s e r v i c e C l u s t e r I P 1 0 . 1 1 1 . 3 3 . 2 3 0 < n o n e > 8 0 / T C P 3 m 5 s
h a r b o r - n o t a r y - s e r v e r C l u s t e r I P 1 0 . 9 7 . 1 4 4 . 2 2 2 < n o n e > 4 4 4 3 / T C P 3 m 5 s
h a r b o r - n o t a r y - s i g n e r C l u s t e r I P 1 0 . 1 1 0 . 1 3 1 . 6 2 < n o n e > 7 8 9 9 / T C P 3 m 5 s
h a r b o r - p o r t a l C l u s t e r I P 1 0 . 1 0 5 . 2 1 3 . 1 4 5 < n o n e > 8 0 / T C P 3 m 5 s
h a r b o r - r e d i s C l u s t e r I P 1 0 . 1 1 0 . 1 8 . 2 1 0 < n o n e > 6 3 7 9 / T C P 3 m 5 s
h a r b o r - r e g i s t r y C l u s t e r I P 1 0 . 1 1 0 . 1 2 9 . 1 6 < n o n e > 5 0 0 0 / T C P , 8 0 8 0 / T C P 3 m 5 s
h a r b o r - t r i v y C l u s t e r I P 1 0 . 1 0 9 . 0 . 1 5 5 < n o n e > 8 0 8 0 / T C P 3 m 5 s
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26.
Loki简介
Loki是 Grafana Labs 团队最新的开源项目,是一个水平可扩展,高可用性,多租户的日志聚合系统。它的设计非常经济高效且易于操作,因为它不会为日志内容编制索引,而是为每个日志流编制一组标签。
与其他日志系统不同,Loki 是围绕仅索引有关日志的元数据的想法构建的:标签(就像 Prometheus 标签一样)。然后,日志数据本身会被压缩并以块的形式存储在对象存储(例如 S3 或 GCS)中,甚至本地存储在文件系统中。小索引和高度压缩的块简化了操作并显着降低了 Loki 的成本。
使用Helm部署Loki日志平台
环境准备
1)创建命名空间
2)安装Helm
如果你已经安装好helm了,可以忽略本步骤。
[[email protected] ~] # wget https://get.helm.sh/helm-v3.7.2-linux-amd64.tar.gz
[[email protected] ~] # tar zxvf helm-v3.7.2-linux-amd64.tar.gz
[[email protected] ~] # cp linux-amd64/helm /usr/local/bin/
[[email protected] ~] # helm versio
v e r s i o n . B u i l d I n f o { V e r s i o n: " v 3 . 7 . 2 " , G i t C o m m i t: " 6 6 3 a 8 9 6 f 4 a 8 1 5 0 5 3 4 4 5 e e c 4 1 5 3 6 7 7 d d c 2 4 a 0 a 3 6 1 " ,
G i t T r e e S t a t e: " c l e a n " , G o V e r s i o n: " g o 1 . 1 6 . 1 0 " }
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
3)添加Chart仓库
[[email protected] ~] # helm repo add grafana https://grafana.github.io/helm-charts
[[email protected] ~] # helm repo list
N A M E U R L
g r a f a n a h t t p s: / / g r a f a n a . g i t h u b . i o / h e l m - c h a r t s
[[email protected] ~] # helm repo update
- 1.
- 2.
- 3.
- 4.
- 5.
4)搜索chart
[[email protected] ~] # helm search repo grafana
N A M E C H A R T V E R S I O N A P P V E R S I O N D E S C R I P T I O N
g r a f a n a / g r a f a n a 6 . 3 2 . 1 9 . 0 . 2 T h e l e a d i n g t o o l f o r q u e r y i n g a n d v i s u a l i z i n g t . . .
g r a f a n a / g r a f a n a - a g e n t - o p e r a t o r 0 . 2 . 2 0 . 2 5 . 1 A H e l m c h a r t f o r G r a f a n a A g e n t O p e r a t o r
g r a f a n a / e n t e r p r i s e - l o g s 2 . 2 . 2 v 1 . 4 . 1 G r a f a n a E n t e r p r i s e L o g s
g r a f a n a / e n t e r p r i s e - l o g s - s i m p l e 1 . 2 . 0 v 1 . 4 . 0 D E P R E C A T E D G r a f a n a E n t e r p r i s e L o g s ( S i m p l e S c a l . . .
g r a f a n a / e n t e r p r i s e - m e t r i c s 1 . 9 . 0 v 1 . 7 . 0 D E P R E C A T E D G r a f a n a E n t e r p r i s e M e t r i c s
g r a f a n a / f l u e n t - b i t 2 . 3 . 1 v 2 . 1 . 0 U s e s f l u e n t - b i t L o k i g o p l u g i n f o r g a t h e r i n g l o . . .
g r a f a n a / l o k i 2 . 1 2 . 2 v 2 . 5 . 0 L o k i: l i k e P r o m e t h e u s , b u t f o r l o g s .
g r a f a n a / l o k i - c a n a r y 0 . 8 . 1 2 . 5 . 0 H e l m c h a r t f o r G r a f a n a L o k i C a n a r y
g r a f a n a / l o k i - d i s t r i b u t e d 0 . 5 0 . 0 2 . 5 . 0 H e l m c h a r t f o r G r a f a n a L o k i i n m i c r o s e r v i c e s m o d e
g r a f a n a / l o k i - s i m p l e - s c a l a b l e 1 . 6 . 1 2 . 5 . 0 H e l m c h a r t f o r G r a f a n a L o k i i n s i m p l e , s c a l a b l e . . .
g r a f a n a / l o k i - s t a c k 2 . 6 . 5 v 2 . 4 . 2 L o k i: l i k e P r o m e t h e u s , b u t f o r l o g s .
g r a f a n a / m i m i r - d i s t r i b u t e d 2 . 1 . 0 2 . 1 . 0 G r a f a n a M i m i r
g r a f a n a / m i m i r - o p e n s h i f t - e x p e r i m e n t a l 2 . 1 . 0 2 . 0 . 0 G r a f a n a M i m i r o n O p e n S h i f t E x p e r i m e n t
g r a f a n a / o n c a l l 1 . 0 . 2 v 1 . 0 . 3 D e v e l o p e r - f r i e n d l y i n c i d e n t r e s p o n s e w i t h b r i l l . . .
g r a f a n a / p r o m t a i l 6 . 0 . 2 2 . 5 . 0 P r o m t a i l i s a n a g e n t w h i c h s h i p s t h e c o n t e n t s o . . .
g r a f a n a / r o l l o u t - o p e r a t o r 0 . 1 . 2 v 0 . 1 . 1 G r a f a n a r o l l o u t - o p e r a t o r
g r a f a n a / t e m p o 0 . 1 5 . 4 1 . 4 . 1 G r a f a n a T e m p o S i n g l e B i n a r y M o d e
g r a f a n a / t e m p o - d i s t r i b u t e d 0 . 2 0 . 3 1 . 4 . 1 G r a f a n a T e m p o i n M i c r o S e r v i c e m o d e
g r a f a n a / t e m p o - v u l t u r e 0 . 2 . 0 1 . 3 . 0 G r a f a n a T e m p o V u l t u r e - A t o o l t o m o n i t o r T e m p o . . .
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
grafana/loki-stack: 将多个进程封装到一个Pod内,未做数据持久化,适用于测试环境,单体模式。
grafana/loki-canary: 金丝雀更新模式;
grafana/loki-distributed: 微服务模式,适合生产较大规模场景;
grafana/loki-simple-scalable: 读写分离模式,简单可扩展;
下载并安装Loki Chart
- 下载解压chart
[[email protected] ~] # helm pull grafana/loki-stack --untar
[[email protected] ~] # cd loki-stack/
[[email protected] loki-stack] # ls
c h a r t s C h a r t . y a m l R E A D M E . m d r e q u i r e m e n t s . l o c k r e q u i r e m e n t s . y a m l t e m p l a t e s v a l u e s . y a m l
- 1.
- 2.
- 3.
- 4.
--untar:下载并解压Chart包;
- 修改values.yaml配置
[[email protected] loki-stack] # vim values.yaml
g r a f a n a:
e n a b l e d: t r u e # 启 用 G r a f a n a 组 件
s i d e c a r:
d a t a s o u r c e s:
e n a b l e d: t r u e
m a x L i n e s: 1 0 0 0
i m a g e:
t a g: 8 . 3 . 5
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 安装
[[email protected] loki-stack] # helm install loki-stack . -n loki
N A M E: l o k i - s t a c k
L A S T D E P L O Y E D: T h u J u l 7 1 3: 3 1: 0 2 2 0 2 2
N A M E S P A C E: l o k i
S T A T U S: d e p l o y e d
R E V I S I O N: 1
N O T E S:
T h e L o k i s t a c k h a s b e e n d e p l o y e d t o y o u r c l u s t e r .
L o k i c a n n o w b e a d d e d a s a d a t a s o u r c e i n G r a f a n a .
S e e h t t p: / / d o c s . g r a f a n a . o r g / f e a t u r e s / d a t a s o u r c e s / l o k i / f o r m o r e d e t a i l .
[[email protected] loki-stack] # helm -n loki ls
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
查看验证服务
[[email protected] loki-stack] # kubectl -n loki get pods
N A M E R E A D Y S T A T U S R E S T A R T S A G E
l o k i - s t a c k - 0 1 / 1 R u n n i n g 0 7 4 s
l o k i - s t a c k - g r a f a n a - 6 9 9 6 6 b c f 9 4 - l p 4 z 5 2 / 2 R u n n i n g 0 7 4 s
l o k i - s t a c k - p r o m t a i l - c k p s h 1 / 1 R u n n i n g 0 7 4 s
l o k i - s t a c k - p r o m t a i l - g 7 k f t 1 / 1 R u n n i n g 0 7 4 s
l o k i - s t a c k - p r o m t a i l - g f 8 6 b 1 / 1 R u n n i n g 0 7 4 s
l o k i - s t a c k - p r o m t a i l - j n z n c 1 / 1 R u n n i n g 0 7 4 s
n f s - p r o v i s i o n e r - 5 6 f d 4 7 f c 9 c - l k t r v 1 / 1 R u n n i n g 3 2 3 h
[[email protected] loki-stack] # kubectl -n loki get svc
N A M E T Y P E C L U S T E R - I P E X T E R N A L - I P P O R T ( S ) A G E
l o k i - s t a c k C l u s t e r I P 1 0 . 1 0 2 . 8 1 . 2 5 0 < n o n e > 3 1 0 0 / T C P 7 6 s
l o k i - s t a c k - g r a f a n a C l u s t e r I P 1 0 . 1 0 3 . 6 2 . 3 1 < n o n e > 8 0 / T C P 7 6 s
l o k i - s t a c k - h e a d l e s s C l u s t e r I P N o n e < n o n e > 3 1 0 0 / T C P 7 6 s
l o k i - s t a c k - m e m b e r l i s t C l u s t e r I P N o n e < n o n e > 7 9 4 6 / T C P 7 6 s
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
Pod都部署成功!
- 修改Grafana服务访问方式为NodePort
[[email protected] loki-stack] # kubectl -n loki edit svc loki-stack-grafana
s p e c:
t y p e: N o d e P o r t
- 1.
- 2.
- 3.
Grafana配置
登录Grafana
[[email protected] loki-stack] # kubectl -n loki get svc loki-stack-grafana
N A M E T Y P E C L U S T E R - I P E X T E R N A L - I P P O R T ( S ) A G E
l o k i - s t a c k - g r a f a n a N o d e P o r t 1 0 . 1 0 3 . 6 2 . 3 1 < n o n e > 8 0: 3 2 5 5 5 / T C P 4 m 9 s
- 1.
- 2.
- 3.
查看Grafana密码
浏览器登录Grafana
- 登录地址: k8s节点IP:NodePort
- 用户名:admin
- 密码: 刚才查看到的secret解析串
Grafana查看Loki日志
然后在输入: {namespace="harbor"} 即可查看Harbor命名空间的所有日志。
边栏推荐
- What does it mean to prefix a string with F?
- IP network active evaluation system -- x-vision
- The latest Android interview collection, Android video extraction audio
- Variables and constants
- operator
- Implementation method of data platform landing
- null == undefined
- UWA Q & a collection
- SAR image quality evaluation
- Kaggle-Titanic
猜你喜欢
#DAYU200体验官#MPPT光伏发电项目 DAYU200、Hi3861、华为云IotDA
What if the win11u disk does not display? Solution to failure of win11 plug-in USB flash disk
[azure microservice service fabric] how to transfer seed nodes in the service fabric cluster
PHP method of obtaining image information
Ternary expressions, generative expressions, anonymous functions
Use json Stringify() to realize deep copy, be careful, there may be a huge hole
客户案例|华律网,通过观测云大幅缩短故障定位时间
Robot autonomous exploration series papers environment code
ByteDance Android interview, summary of knowledge points + analysis of interview questions
Latest Android advanced interview questions summary, Android interview questions and answers
随机推荐
Failed to initialize rosdep after installing ROS
Tsconfig of typescript TS basics JSON configuration options
UWA Q & a collection
Overseas agent recommendation
The whole network "chases" Zhong Xuegao
Form组件常用校验规则-2(持续更新中~)
npm uninstall和rm直接删除的区别
Two methods of calling WCF service by C #
operator
PHP records the pitfalls encountered in the complete docking of Tencent cloud live broadcast and im live group chat
Robot autonomous exploration DSVP: code parsing
PHP method of obtaining image information
Welcome to CSDN markdown editor
Unity development --- the mouse controls the camera to move, rotate and zoom
Revit secondary development - project file to family file
Kaggle-Titanic
变量与常量
Programming mode - table driven programming
Oracle advanced (VI) Oracle expdp/impdp details
Typeorm automatically generates entity classes