当前位置:网站首页>Loki, the "open source star picking program", realizes the efficient management of harbor logs

Loki, the "open source star picking program", realizes the efficient management of harbor logs

2022-07-07 22:36:00 51CTO

This article has participated in 「​ ​ Open source star picking program ​​​」, Welcome to join us . ​​​​​ ​​


Preface

stay Harbor Of

Use Loki Realization Harbor Log management , stay Grafana Page query Harbor Log .


Environmental Science :

Passed Helm stay Kubernetes Install in cluster Harbor,Harbor The service is deployed in Harbor In the namespace .

Helm Deploy Harbor Please read the document of :

Helm Deploy Harbor, Realize highly available image warehouse ( Super detailed sharing )~ The pit record is attached

 ​https://blog.51cto.com/lidabai/5195706​

      
      
[[email protected] harbor] # kubectl -n harbor get pods
N A M E R E A D Y S T A T U S R E S T A R T S A G E
h a r b o r - c h a r t m u s e u m - 6 8 f 6 8 d 8 8 f f - n q 9 7 s 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - c o r e - 7 5 6 5 b f b 9 c c - w 5 k 2 9 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - d a t a b a s e - 0 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - j o b s e r v i c e - 6 7 4 7 b 4 8 6 d f - v m 9 z s 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - n g i n x - 9 5 6 f f f 9 8 8 - r 8 d p x 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - n o t a r y - s e r v e r - 7 7 6 f 8 5 f 9 c 6 - 4 n l s n 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - n o t a r y - s i g n e r - 7 f 8 9 5 d 5 9 d 6 - k 5 9 5 7 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - p o r t a l - 5 7 6 6 b 7 8 4 c 7 - s n d m z 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - r e d i s - 0 1 / 1 R u n n i n g 0 2 m 5 2 s
h a r b o r - r e g i s t r y - 6 b 8 8 c f b 4 6 5 - f c x v n 2 / 2 R u n n i n g 0 2 m 5 2 s
h a r b o r - t r i v y - 0 1 / 1 R u n n i n g 0 2 m 5 2 s
[[email protected] harbor] # kubectl -n harbor get svc
N A M E T Y P E C L U S T E R - I P E X T E R N A L - I P P O R T ( S ) A G E
h a r b o r N o d e P o r t 1 0 . 9 8 . 7 1 . 1 3 7 < n o n e > 8 0: 3 0 0 0 2 / T C P , 4 4 4 3: 3 0 0 0 4 / T C P 3 m 5 s
h a r b o r - c h a r t m u s e u m C l u s t e r I P 1 0 . 1 0 2 . 1 9 4 . 1 4 8 < n o n e > 8 0 / T C P 3 m 5 s
h a r b o r - c o r e C l u s t e r I P 1 0 . 1 0 0 . 8 7 . 1 7 4 < n o n e > 8 0 / T C P 3 m 5 s
h a r b o r - d a t a b a s e C l u s t e r I P 1 0 . 1 0 0 . 7 9 . 7 2 < n o n e > 5 4 3 2 / T C P 3 m 5 s
h a r b o r - j o b s e r v i c e C l u s t e r I P 1 0 . 1 1 1 . 3 3 . 2 3 0 < n o n e > 8 0 / T C P 3 m 5 s
h a r b o r - n o t a r y - s e r v e r C l u s t e r I P 1 0 . 9 7 . 1 4 4 . 2 2 2 < n o n e > 4 4 4 3 / T C P 3 m 5 s
h a r b o r - n o t a r y - s i g n e r C l u s t e r I P 1 0 . 1 1 0 . 1 3 1 . 6 2 < n o n e > 7 8 9 9 / T C P 3 m 5 s
h a r b o r - p o r t a l C l u s t e r I P 1 0 . 1 0 5 . 2 1 3 . 1 4 5 < n o n e > 8 0 / T C P 3 m 5 s
h a r b o r - r e d i s C l u s t e r I P 1 0 . 1 1 0 . 1 8 . 2 1 0 < n o n e > 6 3 7 9 / T C P 3 m 5 s
h a r b o r - r e g i s t r y C l u s t e r I P 1 0 . 1 1 0 . 1 2 9 . 1 6 < n o n e > 5 0 0 0 / T C P , 8 0 8 0 / T C P 3 m 5 s
h a r b o r - t r i v y C l u s t e r I P 1 0 . 1 0 9 . 0 . 1 5 5 < n o n e > 8 0 8 0 / T C P 3 m 5 s
  • 1.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
  • 22.
  • 23.
  • 24.
  • 25.
  • 26.

「 Open source star picking program 」Loki Realization Harbor Efficient management of logs _ Cloud native

Loki brief introduction

Loki yes Grafana Labs The team's latest open source project , It's a level of scalability , High availability , Multi tenant log aggregation system . It's designed to be very cost-effective and easy to operate , Because it does not index log content , Instead, label each log stream .

 

Unlike other logging systems ,Loki It is built around the idea of indexing only metadata about logs : label ( It's like Prometheus The label is the same ). then , The log data itself will be compressed and stored in object storage in the form of blocks ( for example S3 or GCS) in , Even stored locally in the file system . Small indexes and highly compressed blocks simplify operations and significantly reduce Loki Cost of .


Use Helm Deploy Loki Log platform


Environmental preparation

1) Create a namespace

      
      
[[email protected] ~] # kubectl create namespace loki
  • 1.

2) install Helm

    If you have installed it helm 了 , You can ignore this step .

      
      
[[email protected] ~] # wget https://get.helm.sh/helm-v3.7.2-linux-amd64.tar.gz
[[email protected] ~] # tar zxvf helm-v3.7.2-linux-amd64.tar.gz
[[email protected] ~] # cp linux-amd64/helm /usr/local/bin/
[[email protected] ~] # helm versio
v e r s i o n . B u i l d I n f o { V e r s i o n: " v 3 . 7 . 2 " , G i t C o m m i t: " 6 6 3 a 8 9 6 f 4 a 8 1 5 0 5 3 4 4 5 e e c 4 1 5 3 6 7 7 d d c 2 4 a 0 a 3 6 1 " ,
G i t T r e e S t a t e: " c l e a n " , G o V e r s i o n: " g o 1 . 1 6 . 1 0 " }
  • 1.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.

3) add to Chart Warehouse

      
      
[[email protected] ~] # helm repo add grafana https://grafana.github.io/helm-charts
[[email protected] ~] # helm repo list
N A M E U R L
g r a f a n a h t t p s: / / g r a f a n a . g i t h u b . i o / h e l m - c h a r t s
[[email protected] ~] # helm repo update
  • 1.
  • 2.
  • 3.
  • 4.
  • 5.


4) Search for chart

      
      
[[email protected] ~] # helm search repo grafana
N A M E C H A R T V E R S I O N A P P V E R S I O N D E S C R I P T I O N
g r a f a n a / g r a f a n a 6 . 3 2 . 1 9 . 0 . 2 T h e l e a d i n g t o o l f o r q u e r y i n g a n d v i s u a l i z i n g t . . .
g r a f a n a / g r a f a n a - a g e n t - o p e r a t o r 0 . 2 . 2 0 . 2 5 . 1 A H e l m c h a r t f o r G r a f a n a A g e n t O p e r a t o r
g r a f a n a / e n t e r p r i s e - l o g s 2 . 2 . 2 v 1 . 4 . 1 G r a f a n a E n t e r p r i s e L o g s
g r a f a n a / e n t e r p r i s e - l o g s - s i m p l e 1 . 2 . 0 v 1 . 4 . 0 D E P R E C A T E D G r a f a n a E n t e r p r i s e L o g s ( S i m p l e S c a l . . .
g r a f a n a / e n t e r p r i s e - m e t r i c s 1 . 9 . 0 v 1 . 7 . 0 D E P R E C A T E D G r a f a n a E n t e r p r i s e M e t r i c s
g r a f a n a / f l u e n t - b i t 2 . 3 . 1 v 2 . 1 . 0 U s e s f l u e n t - b i t L o k i g o p l u g i n f o r g a t h e r i n g l o . . .
g r a f a n a / l o k i 2 . 1 2 . 2 v 2 . 5 . 0 L o k i: l i k e P r o m e t h e u s , b u t f o r l o g s .
g r a f a n a / l o k i - c a n a r y 0 . 8 . 1 2 . 5 . 0 H e l m c h a r t f o r G r a f a n a L o k i C a n a r y
g r a f a n a / l o k i - d i s t r i b u t e d 0 . 5 0 . 0 2 . 5 . 0 H e l m c h a r t f o r G r a f a n a L o k i i n m i c r o s e r v i c e s m o d e
g r a f a n a / l o k i - s i m p l e - s c a l a b l e 1 . 6 . 1 2 . 5 . 0 H e l m c h a r t f o r G r a f a n a L o k i i n s i m p l e , s c a l a b l e . . .
g r a f a n a / l o k i - s t a c k 2 . 6 . 5 v 2 . 4 . 2 L o k i: l i k e P r o m e t h e u s , b u t f o r l o g s .
g r a f a n a / m i m i r - d i s t r i b u t e d 2 . 1 . 0 2 . 1 . 0 G r a f a n a M i m i r
g r a f a n a / m i m i r - o p e n s h i f t - e x p e r i m e n t a l 2 . 1 . 0 2 . 0 . 0 G r a f a n a M i m i r o n O p e n S h i f t E x p e r i m e n t
g r a f a n a / o n c a l l 1 . 0 . 2 v 1 . 0 . 3 D e v e l o p e r - f r i e n d l y i n c i d e n t r e s p o n s e w i t h b r i l l . . .
g r a f a n a / p r o m t a i l 6 . 0 . 2 2 . 5 . 0 P r o m t a i l i s a n a g e n t w h i c h s h i p s t h e c o n t e n t s o . . .
g r a f a n a / r o l l o u t - o p e r a t o r 0 . 1 . 2 v 0 . 1 . 1 G r a f a n a r o l l o u t - o p e r a t o r
g r a f a n a / t e m p o 0 . 1 5 . 4 1 . 4 . 1 G r a f a n a T e m p o S i n g l e B i n a r y M o d e
g r a f a n a / t e m p o - d i s t r i b u t e d 0 . 2 0 . 3 1 . 4 . 1 G r a f a n a T e m p o i n M i c r o S e r v i c e m o d e
g r a f a n a / t e m p o - v u l t u r e 0 . 2 . 0 1 . 3 . 0 G r a f a n a T e m p o V u l t u r e - A t o o l t o m o n i t o r T e m p o . . .
  • 1.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.

「 Open source star picking program 」Loki Realization Harbor Efficient management of logs _Harbor_02

 grafana/loki-stack:  Encapsulate multiple processes into one Pod Inside , No data persistence , For test environments , Monomer mode .

 grafana/loki-canary:  Canary update mode ;

 grafana/loki-distributed:   Microservice model , Suitable for large-scale production ;

 grafana/loki-simple-scalable:  Read write separation mode , Simple and scalable ;


Download and install Loki Chart

  • Download decompression chart
      
      
[[email protected] ~] # helm pull grafana/loki-stack --untar
[[email protected] ~] # cd loki-stack/
[[email protected] loki-stack] # ls
c h a r t s C h a r t . y a m l R E A D M E . m d r e q u i r e m e n t s . l o c k r e q u i r e m e n t s . y a m l t e m p l a t e s v a l u e s . y a m l
  • 1.
  • 2.
  • 3.
  • 4.

--untar: Download and unzip Chart package ;

  • modify values.yaml To configure
      
      
[[email protected] loki-stack] # vim values.yaml
g r a f a n a:
e n a b l e d: t r u e # Qi use G r a f a n a Group Pieces of
s i d e c a r:
d a t a s o u r c e s:
e n a b l e d: t r u e
m a x L i n e s: 1 0 0 0
i m a g e:
t a g: 8 . 3 . 5
  • 1.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.


  • install
      
      
[[email protected] loki-stack] # helm install loki-stack . -n loki
N A M E: l o k i - s t a c k
L A S T D E P L O Y E D: T h u J u l 7 1 3: 3 1: 0 2 2 0 2 2
N A M E S P A C E: l o k i
S T A T U S: d e p l o y e d
R E V I S I O N: 1
N O T E S:
T h e L o k i s t a c k h a s b e e n d e p l o y e d t o y o u r c l u s t e r .
L o k i c a n n o w b e a d d e d a s a d a t a s o u r c e i n G r a f a n a .
S e e h t t p: / / d o c s . g r a f a n a . o r g / f e a t u r e s / d a t a s o u r c e s / l o k i / f o r m o r e d e t a i l .
[[email protected] loki-stack] # helm -n loki ls
  • 1.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.

「 Open source star picking program 」Loki Realization Harbor Efficient management of logs _ Operation and maintenance _03


View authentication services

      
      
[[email protected] loki-stack] # kubectl -n loki get pods
N A M E R E A D Y S T A T U S R E S T A R T S A G E
l o k i - s t a c k - 0 1 / 1 R u n n i n g 0 7 4 s
l o k i - s t a c k - g r a f a n a - 6 9 9 6 6 b c f 9 4 - l p 4 z 5 2 / 2 R u n n i n g 0 7 4 s
l o k i - s t a c k - p r o m t a i l - c k p s h 1 / 1 R u n n i n g 0 7 4 s
l o k i - s t a c k - p r o m t a i l - g 7 k f t 1 / 1 R u n n i n g 0 7 4 s
l o k i - s t a c k - p r o m t a i l - g f 8 6 b 1 / 1 R u n n i n g 0 7 4 s
l o k i - s t a c k - p r o m t a i l - j n z n c 1 / 1 R u n n i n g 0 7 4 s
n f s - p r o v i s i o n e r - 5 6 f d 4 7 f c 9 c - l k t r v 1 / 1 R u n n i n g 3 2 3 h
[[email protected] loki-stack] # kubectl -n loki get svc
N A M E T Y P E C L U S T E R - I P E X T E R N A L - I P P O R T ( S ) A G E
l o k i - s t a c k C l u s t e r I P 1 0 . 1 0 2 . 8 1 . 2 5 0 < n o n e > 3 1 0 0 / T C P 7 6 s
l o k i - s t a c k - g r a f a n a C l u s t e r I P 1 0 . 1 0 3 . 6 2 . 3 1 < n o n e > 8 0 / T C P 7 6 s
l o k i - s t a c k - h e a d l e s s C l u s t e r I P N o n e < n o n e > 3 1 0 0 / T C P 7 6 s
l o k i - s t a c k - m e m b e r l i s t C l u s t e r I P N o n e < n o n e > 7 9 4 6 / T C P 7 6 s
  • 1.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 15.

Pod All deployed successfully !

  • modify Grafana The service access method is NodePort
      
      
[[email protected] loki-stack] # kubectl -n loki edit svc loki-stack-grafana
s p e c:
t y p e: N o d e P o r t
  • 1.
  • 2.
  • 3.

「 Open source star picking program 」Loki Realization Harbor Efficient management of logs _Harbor_04


Grafana To configure


Sign in Grafana

      
      
[[email protected] loki-stack] # kubectl -n loki get svc loki-stack-grafana
N A M E T Y P E C L U S T E R - I P E X T E R N A L - I P P O R T ( S ) A G E
l o k i - s t a c k - g r a f a n a N o d e P o r t 1 0 . 1 0 3 . 6 2 . 3 1 < n o n e > 8 0: 3 2 5 5 5 / T C P 4 m 9 s
  • 1.
  • 2.
  • 3.

see Grafana password

      
      
$ k u b e c t l - n l o k i g e t s e c r e t l o k i - s t a c k - g r a f a n a - o j s o n p a t h= " { . d a t a . a d m i n - p a s s w o r d } " | b a s e 6 4 - - d e c o d e ; e c h o
f P P Z 1 9 c A n P 0 J 7 M Y P i U 4 v k W I c M D l b x l g G J Y e 4 j K Y T
  • 1.
  • 2.

「 Open source star picking program 」Loki Realization Harbor Efficient management of logs _ Operation and maintenance _05

Browser login Grafana

  • The login address : k8s node IP:NodePort
  • user name :admin
  • password : What I just saw secret Analytic string

「 Open source star picking program 」Loki Realization Harbor Efficient management of logs _ Cloud computing _06


Grafana see Loki journal

「 Open source star picking program 」Loki Realization Harbor Efficient management of logs _ Cloud native _07

And then input : {namespace="harbor"} You can view it Harbor All logs of namespace .

「 Open source star picking program 」Loki Realization Harbor Efficient management of logs _Harbor_08



原网站

版权声明
本文为[51CTO]所创,转载请带上原文链接,感谢
https://yzsam.com/2022/188/202207072135076941.html