当前位置:网站首页>Loki, the "open source star picking program", realizes the efficient management of harbor logs
Loki, the "open source star picking program", realizes the efficient management of harbor logs
2022-07-07 22:36:00 【51CTO】
This article has participated in 「 Open source star picking program 」, Welcome to join us .
Preface
stay Harbor Of
Use Loki Realization Harbor Log management , stay Grafana Page query Harbor Log .
Environmental Science :
Passed Helm stay Kubernetes Install in cluster Harbor,Harbor The service is deployed in Harbor In the namespace .
Helm Deploy Harbor Please read the document of :
Helm Deploy Harbor, Realize highly available image warehouse ( Super detailed sharing )~ The pit record is attached
https://blog.51cto.com/lidabai/5195706
[[email protected] harbor] # kubectl -n harbor get pods
N A M E R E A D Y S T A T U S R E S T A R T S A G E
h a r b o r - c h a r t m u s e u m - 6 8 f 6 8 d 8 8 f f - n q 9 7 s 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - c o r e - 7 5 6 5 b f b 9 c c - w 5 k 2 9 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - d a t a b a s e - 0 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - j o b s e r v i c e - 6 7 4 7 b 4 8 6 d f - v m 9 z s 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - n g i n x - 9 5 6 f f f 9 8 8 - r 8 d p x 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - n o t a r y - s e r v e r - 7 7 6 f 8 5 f 9 c 6 - 4 n l s n 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - n o t a r y - s i g n e r - 7 f 8 9 5 d 5 9 d 6 - k 5 9 5 7 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - p o r t a l - 5 7 6 6 b 7 8 4 c 7 - s n d m z 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - r e d i s - 0 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - r e g i s t r y - 6 b 8 8 c f b 4 6 5 - f c x v n 2 / 2 R u n n i n g 0 2 m 5 2 s
h a r b o r - t r i v y - 0 1 / 1 R u n n i n g 0 2 m 5 2 s
[[email protected] harbor] # kubectl -n harbor get svc
N A M E T Y P E C L U S T E R - I P E X T E R N A L - I P P O R T ( S ) A G E
h a r b o r N o d e P o r t 1 0 . 9 8 . 7 1 . 1 3 7 < n o n e > 8 0: 3 0 0 0 2 / T C P , 4 4 4 3: 3 0 0 0 4 / T C P 3 m 5 s
h a r b o r - c h a r t m u s e u m C l u s t e r I P 1 0 . 1 0 2 . 1 9 4 . 1 4 8 < n o n e > 8 0 / T C P 3 m 5 s
h a r b o r - c o r e C l u s t e r I P 1 0 . 1 0 0 . 8 7 . 1 7 4 < n o n e > 8 0 / T C P 3 m 5 s
h a r b o r - d a t a b a s e C l u s t e r I P 1 0 . 1 0 0 . 7 9 . 7 2 < n o n e > 5 4 3 2 / T C P 3 m 5 s
h a r b o r - j o b s e r v i c e C l u s t e r I P 1 0 . 1 1 1 . 3 3 . 2 3 0 < n o n e > 8 0 / T C P 3 m 5 s
h a r b o r - n o t a r y - s e r v e r C l u s t e r I P 1 0 . 9 7 . 1 4 4 . 2 2 2 < n o n e > 4 4 4 3 / T C P 3 m 5 s
h a r b o r - n o t a r y - s i g n e r C l u s t e r I P 1 0 . 1 1 0 . 1 3 1 . 6 2 < n o n e > 7 8 9 9 / T C P 3 m 5 s
h a r b o r - p o r t a l C l u s t e r I P 1 0 . 1 0 5 . 2 1 3 . 1 4 5 < n o n e > 8 0 / T C P 3 m 5 s
h a r b o r - r e d i s C l u s t e r I P 1 0 . 1 1 0 . 1 8 . 2 1 0 < n o n e > 6 3 7 9 / T C P 3 m 5 s
h a r b o r - r e g i s t r y C l u s t e r I P 1 0 . 1 1 0 . 1 2 9 . 1 6 < n o n e > 5 0 0 0 / T C P , 8 0 8 0 / T C P 3 m 5 s
h a r b o r - t r i v y C l u s t e r I P 1 0 . 1 0 9 . 0 . 1 5 5 < n o n e > 8 0 8 0 / T C P 3 m 5 s
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26.
Loki brief introduction
Loki yes Grafana Labs The team's latest open source project , It's a level of scalability , High availability , Multi tenant log aggregation system . It's designed to be very cost-effective and easy to operate , Because it does not index log content , Instead, label each log stream .
Unlike other logging systems ,Loki It is built around the idea of indexing only metadata about logs : label ( It's like Prometheus The label is the same ). then , The log data itself will be compressed and stored in object storage in the form of blocks ( for example S3 or GCS) in , Even stored locally in the file system . Small indexes and highly compressed blocks simplify operations and significantly reduce Loki Cost of .
Use Helm Deploy Loki Log platform
Environmental preparation
1) Create a namespace
2) install Helm
If you have installed it helm 了 , You can ignore this step .
[[email protected] ~] # wget https://get.helm.sh/helm-v3.7.2-linux-amd64.tar.gz
[[email protected] ~] # tar zxvf helm-v3.7.2-linux-amd64.tar.gz
[[email protected] ~] # cp linux-amd64/helm /usr/local/bin/
[[email protected] ~] # helm versio
v e r s i o n . B u i l d I n f o { V e r s i o n: " v 3 . 7 . 2 " , G i t C o m m i t: " 6 6 3 a 8 9 6 f 4 a 8 1 5 0 5 3 4 4 5 e e c 4 1 5 3 6 7 7 d d c 2 4 a 0 a 3 6 1 " ,
G i t T r e e S t a t e: " c l e a n " , G o V e r s i o n: " g o 1 . 1 6 . 1 0 " }
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
3) add to Chart Warehouse
[[email protected] ~] # helm repo add grafana https://grafana.github.io/helm-charts
[[email protected] ~] # helm repo list
N A M E U R L
g r a f a n a h t t p s: / / g r a f a n a . g i t h u b . i o / h e l m - c h a r t s
[[email protected] ~] # helm repo update
- 1.
- 2.
- 3.
- 4.
- 5.
4) Search for chart
[[email protected] ~] # helm search repo grafana
N A M E C H A R T V E R S I O N A P P V E R S I O N D E S C R I P T I O N
g r a f a n a / g r a f a n a 6 . 3 2 . 1 9 . 0 . 2 T h e l e a d i n g t o o l f o r q u e r y i n g a n d v i s u a l i z i n g t . . .
g r a f a n a / g r a f a n a - a g e n t - o p e r a t o r 0 . 2 . 2 0 . 2 5 . 1 A H e l m c h a r t f o r G r a f a n a A g e n t O p e r a t o r
g r a f a n a / e n t e r p r i s e - l o g s 2 . 2 . 2 v 1 . 4 . 1 G r a f a n a E n t e r p r i s e L o g s
g r a f a n a / e n t e r p r i s e - l o g s - s i m p l e 1 . 2 . 0 v 1 . 4 . 0 D E P R E C A T E D G r a f a n a E n t e r p r i s e L o g s ( S i m p l e S c a l . . .
g r a f a n a / e n t e r p r i s e - m e t r i c s 1 . 9 . 0 v 1 . 7 . 0 D E P R E C A T E D G r a f a n a E n t e r p r i s e M e t r i c s
g r a f a n a / f l u e n t - b i t 2 . 3 . 1 v 2 . 1 . 0 U s e s f l u e n t - b i t L o k i g o p l u g i n f o r g a t h e r i n g l o . . .
g r a f a n a / l o k i 2 . 1 2 . 2 v 2 . 5 . 0 L o k i: l i k e P r o m e t h e u s , b u t f o r l o g s .
g r a f a n a / l o k i - c a n a r y 0 . 8 . 1 2 . 5 . 0 H e l m c h a r t f o r G r a f a n a L o k i C a n a r y
g r a f a n a / l o k i - d i s t r i b u t e d 0 . 5 0 . 0 2 . 5 . 0 H e l m c h a r t f o r G r a f a n a L o k i i n m i c r o s e r v i c e s m o d e
g r a f a n a / l o k i - s i m p l e - s c a l a b l e 1 . 6 . 1 2 . 5 . 0 H e l m c h a r t f o r G r a f a n a L o k i i n s i m p l e , s c a l a b l e . . .
g r a f a n a / l o k i - s t a c k 2 . 6 . 5 v 2 . 4 . 2 L o k i: l i k e P r o m e t h e u s , b u t f o r l o g s .
g r a f a n a / m i m i r - d i s t r i b u t e d 2 . 1 . 0 2 . 1 . 0 G r a f a n a M i m i r
g r a f a n a / m i m i r - o p e n s h i f t - e x p e r i m e n t a l 2 . 1 . 0 2 . 0 . 0 G r a f a n a M i m i r o n O p e n S h i f t E x p e r i m e n t
g r a f a n a / o n c a l l 1 . 0 . 2 v 1 . 0 . 3 D e v e l o p e r - f r i e n d l y i n c i d e n t r e s p o n s e w i t h b r i l l . . .
g r a f a n a / p r o m t a i l 6 . 0 . 2 2 . 5 . 0 P r o m t a i l i s a n a g e n t w h i c h s h i p s t h e c o n t e n t s o . . .
g r a f a n a / r o l l o u t - o p e r a t o r 0 . 1 . 2 v 0 . 1 . 1 G r a f a n a r o l l o u t - o p e r a t o r
g r a f a n a / t e m p o 0 . 1 5 . 4 1 . 4 . 1 G r a f a n a T e m p o S i n g l e B i n a r y M o d e
g r a f a n a / t e m p o - d i s t r i b u t e d 0 . 2 0 . 3 1 . 4 . 1 G r a f a n a T e m p o i n M i c r o S e r v i c e m o d e
g r a f a n a / t e m p o - v u l t u r e 0 . 2 . 0 1 . 3 . 0 G r a f a n a T e m p o V u l t u r e - A t o o l t o m o n i t o r T e m p o . . .
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
grafana/loki-stack: Encapsulate multiple processes into one Pod Inside , No data persistence , For test environments , Monomer mode .
grafana/loki-canary: Canary update mode ;
grafana/loki-distributed: Microservice model , Suitable for large-scale production ;
grafana/loki-simple-scalable: Read write separation mode , Simple and scalable ;
Download and install Loki Chart
- Download decompression chart
[[email protected] ~] # helm pull grafana/loki-stack --untar
[[email protected] ~] # cd loki-stack/
[[email protected] loki-stack] # ls
c h a r t s C h a r t . y a m l R E A D M E . m d r e q u i r e m e n t s . l o c k r e q u i r e m e n t s . y a m l t e m p l a t e s v a l u e s . y a m l
- 1.
- 2.
- 3.
- 4.
--untar: Download and unzip Chart package ;
- modify values.yaml To configure
[[email protected] loki-stack] # vim values.yaml
g r a f a n a:
e n a b l e d: t r u e # Qi use G r a f a n a Group Pieces of
s i d e c a r:
d a t a s o u r c e s:
e n a b l e d: t r u e
m a x L i n e s: 1 0 0 0
i m a g e:
t a g: 8 . 3 . 5
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- install
[[email protected] loki-stack] # helm install loki-stack . -n loki
N A M E: l o k i - s t a c k
L A S T D E P L O Y E D: T h u J u l 7 1 3: 3 1: 0 2 2 0 2 2
N A M E S P A C E: l o k i
S T A T U S: d e p l o y e d
R E V I S I O N: 1
N O T E S:
T h e L o k i s t a c k h a s b e e n d e p l o y e d t o y o u r c l u s t e r .
L o k i c a n n o w b e a d d e d a s a d a t a s o u r c e i n G r a f a n a .
S e e h t t p: / / d o c s . g r a f a n a . o r g / f e a t u r e s / d a t a s o u r c e s / l o k i / f o r m o r e d e t a i l .
[[email protected] loki-stack] # helm -n loki ls
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
View authentication services
[[email protected] loki-stack] # kubectl -n loki get pods
N A M E R E A D Y S T A T U S R E S T A R T S A G E
l o k i - s t a c k - 0 1 / 1 R u n n i n g 0 7 4 s
l o k i - s t a c k - g r a f a n a - 6 9 9 6 6 b c f 9 4 - l p 4 z 5 2 / 2 R u n n i n g 0 7 4 s
l o k i - s t a c k - p r o m t a i l - c k p s h 1 / 1 R u n n i n g 0 7 4 s
l o k i - s t a c k - p r o m t a i l - g 7 k f t 1 / 1 R u n n i n g 0 7 4 s
l o k i - s t a c k - p r o m t a i l - g f 8 6 b 1 / 1 R u n n i n g 0 7 4 s
l o k i - s t a c k - p r o m t a i l - j n z n c 1 / 1 R u n n i n g 0 7 4 s
n f s - p r o v i s i o n e r - 5 6 f d 4 7 f c 9 c - l k t r v 1 / 1 R u n n i n g 3 2 3 h
[[email protected] loki-stack] # kubectl -n loki get svc
N A M E T Y P E C L U S T E R - I P E X T E R N A L - I P P O R T ( S ) A G E
l o k i - s t a c k C l u s t e r I P 1 0 . 1 0 2 . 8 1 . 2 5 0 < n o n e > 3 1 0 0 / T C P 7 6 s
l o k i - s t a c k - g r a f a n a C l u s t e r I P 1 0 . 1 0 3 . 6 2 . 3 1 < n o n e > 8 0 / T C P 7 6 s
l o k i - s t a c k - h e a d l e s s C l u s t e r I P N o n e < n o n e > 3 1 0 0 / T C P 7 6 s
l o k i - s t a c k - m e m b e r l i s t C l u s t e r I P N o n e < n o n e > 7 9 4 6 / T C P 7 6 s
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
Pod All deployed successfully !
- modify Grafana The service access method is NodePort
[[email protected] loki-stack] # kubectl -n loki edit svc loki-stack-grafana
s p e c:
t y p e: N o d e P o r t
- 1.
- 2.
- 3.
Grafana To configure
Sign in Grafana
[[email protected] loki-stack] # kubectl -n loki get svc loki-stack-grafana
N A M E T Y P E C L U S T E R - I P E X T E R N A L - I P P O R T ( S ) A G E
l o k i - s t a c k - g r a f a n a N o d e P o r t 1 0 . 1 0 3 . 6 2 . 3 1 < n o n e > 8 0: 3 2 5 5 5 / T C P 4 m 9 s
- 1.
- 2.
- 3.
see Grafana password
Browser login Grafana
- The login address : k8s node IP:NodePort
- user name :admin
- password : What I just saw secret Analytic string
Grafana see Loki journal
And then input : {namespace="harbor"} You can view it Harbor All logs of namespace .
边栏推荐
- [azure microservice service fabric] the service fabric cluster hangs up because the certificate expires (the upgrade cannot be completed, and the node is unavailable)
- The free styling service of Dyson's official direct store is now open for appointment. Pioneer Technology interprets the styling concept of hair care and helps consumers unlock diversified and shiny s
- 如何选择合适的自动化测试工具?
- null == undefined
- Build your own website (18)
- Redis official ORM framework is more elegant than redistemplate
- The latest Android interview collection, Android video extraction audio
- Revit secondary development - Hide occlusion elements
- How to choose the appropriate automated testing tools?
- PKPM 2020软件安装包下载及安装教程
猜你喜欢
The whole network "chases" Zhong Xuegao
OpenGL configuration vs2019
使用 CustomPaint 绘制基本图形
Where is the big data open source project, one-stop fully automated full life cycle operation and maintenance steward Chengying (background)?
Antd date component appears in English
Remember aximp once Use of exe tool
PHP method of obtaining image information
Amesim2016 and matlab2017b joint simulation environment construction
「开源摘星计划」Loki实现Harbor日志的高效管理
[environment] pycharm sets the tool to convert QRC into py file
随机推荐
Xcode modifies the default background image of launchscreen and still displays the original image
Pre sale 179000, hengchi 5 can fire? Product power online depends on how it is sold
Build your own website (18)
DNS series (I): why does the updated DNS record not take effect?
海外代理推荐
Cannot find module 'xxx' or its corresponding type declaration
Revit secondary development - modify wall thickness
Paint basic graphics with custompaint
微服务远程Debug,Nocalhost + Rainbond微服务开发第二弹
Node:504 error reporting
应用实践 | 数仓体系效率全面提升!同程数科基于 Apache Doris 的数据仓库建设
How to choose the appropriate automated testing tools?
How to realize the movement control of characters in horizontal game
【Azure微服务 Service Fabric 】因证书过期导致Service Fabric集群挂掉(升级无法完成,节点不可用)
[problem] pytorch installation
Aspose. Word operation word document (II)
Crawler (17) - Interview (2) | crawler interview question bank
Welcome to CSDN markdown editor
Add get disabled for RC form
Implementation method of data platform landing