当前位置:网站首页>Loki, the "open source star picking program", realizes the efficient management of harbor logs
Loki, the "open source star picking program", realizes the efficient management of harbor logs
2022-07-07 22:36:00 【51CTO】
This article has participated in 「 Open source star picking program 」, Welcome to join us .
Preface
stay Harbor Of
Use Loki Realization Harbor Log management , stay Grafana Page query Harbor Log .
Environmental Science :
Passed Helm stay Kubernetes Install in cluster Harbor,Harbor The service is deployed in Harbor In the namespace .
Helm Deploy Harbor Please read the document of :
Helm Deploy Harbor, Realize highly available image warehouse ( Super detailed sharing )~ The pit record is attached
https://blog.51cto.com/lidabai/5195706
[[email protected] harbor]
# kubectl -n harbor get pods
N
A
M
E
R
E
A
D
Y
S
T
A
T
U
S
R
E
S
T
A
R
T
S
A
G
E
h
a
r
b
o
r
-
c
h
a
r
t
m
u
s
e
u
m
-
6
8
f
6
8
d
8
8
f
f
-
n
q
9
7
s
1
/
1
R
u
n
n
i
n
g
0
2
m
5
2
s
h
a
r
b
o
r
-
c
o
r
e
-
7
5
6
5
b
f
b
9
c
c
-
w
5
k
2
9
1
/
1
R
u
n
n
i
n
g
0
2
m
5
2
s
h
a
r
b
o
r
-
d
a
t
a
b
a
s
e
-
0
1
/
1
R
u
n
n
i
n
g
0
2
m
5
2
s
h
a
r
b
o
r
-
j
o
b
s
e
r
v
i
c
e
-
6
7
4
7
b
4
8
6
d
f
-
v
m
9
z
s
1
/
1
R
u
n
n
i
n
g
0
2
m
5
2
s
h
a
r
b
o
r
-
n
g
i
n
x
-
9
5
6
f
f
f
9
8
8
-
r
8
d
p
x
1
/
1
R
u
n
n
i
n
g
0
2
m
5
2
s
h
a
r
b
o
r
-
n
o
t
a
r
y
-
s
e
r
v
e
r
-
7
7
6
f
8
5
f
9
c
6
-
4
n
l
s
n
1
/
1
R
u
n
n
i
n
g
0
2
m
5
2
s
h
a
r
b
o
r
-
n
o
t
a
r
y
-
s
i
g
n
e
r
-
7
f
8
9
5
d
5
9
d
6
-
k
5
9
5
7
1
/
1
R
u
n
n
i
n
g
0
2
m
5
2
s
h
a
r
b
o
r
-
p
o
r
t
a
l
-
5
7
6
6
b
7
8
4
c
7
-
s
n
d
m
z
1
/
1
R
u
n
n
i
n
g
0
2
m
5
2
s
h
a
r
b
o
r
-
r
e
d
i
s
-
0
1
/
1
R
u
n
n
i
n
g
0
2
m
5
2
s
h
a
r
b
o
r
-
r
e
g
i
s
t
r
y
-
6
b
8
8
c
f
b
4
6
5
-
f
c
x
v
n
2
/
2
R
u
n
n
i
n
g
0
2
m
5
2
s
h
a
r
b
o
r
-
t
r
i
v
y
-
0
1
/
1
R
u
n
n
i
n
g
0
2
m
5
2
s
[[email protected] harbor]
# kubectl -n harbor get svc
N
A
M
E
T
Y
P
E
C
L
U
S
T
E
R
-
I
P
E
X
T
E
R
N
A
L
-
I
P
P
O
R
T
(
S
)
A
G
E
h
a
r
b
o
r
N
o
d
e
P
o
r
t
1
0
.
9
8
.
7
1
.
1
3
7
<
n
o
n
e
>
8
0:
3
0
0
0
2
/
T
C
P
,
4
4
4
3:
3
0
0
0
4
/
T
C
P
3
m
5
s
h
a
r
b
o
r
-
c
h
a
r
t
m
u
s
e
u
m
C
l
u
s
t
e
r
I
P
1
0
.
1
0
2
.
1
9
4
.
1
4
8
<
n
o
n
e
>
8
0
/
T
C
P
3
m
5
s
h
a
r
b
o
r
-
c
o
r
e
C
l
u
s
t
e
r
I
P
1
0
.
1
0
0
.
8
7
.
1
7
4
<
n
o
n
e
>
8
0
/
T
C
P
3
m
5
s
h
a
r
b
o
r
-
d
a
t
a
b
a
s
e
C
l
u
s
t
e
r
I
P
1
0
.
1
0
0
.
7
9
.
7
2
<
n
o
n
e
>
5
4
3
2
/
T
C
P
3
m
5
s
h
a
r
b
o
r
-
j
o
b
s
e
r
v
i
c
e
C
l
u
s
t
e
r
I
P
1
0
.
1
1
1
.
3
3
.
2
3
0
<
n
o
n
e
>
8
0
/
T
C
P
3
m
5
s
h
a
r
b
o
r
-
n
o
t
a
r
y
-
s
e
r
v
e
r
C
l
u
s
t
e
r
I
P
1
0
.
9
7
.
1
4
4
.
2
2
2
<
n
o
n
e
>
4
4
4
3
/
T
C
P
3
m
5
s
h
a
r
b
o
r
-
n
o
t
a
r
y
-
s
i
g
n
e
r
C
l
u
s
t
e
r
I
P
1
0
.
1
1
0
.
1
3
1
.
6
2
<
n
o
n
e
>
7
8
9
9
/
T
C
P
3
m
5
s
h
a
r
b
o
r
-
p
o
r
t
a
l
C
l
u
s
t
e
r
I
P
1
0
.
1
0
5
.
2
1
3
.
1
4
5
<
n
o
n
e
>
8
0
/
T
C
P
3
m
5
s
h
a
r
b
o
r
-
r
e
d
i
s
C
l
u
s
t
e
r
I
P
1
0
.
1
1
0
.
1
8
.
2
1
0
<
n
o
n
e
>
6
3
7
9
/
T
C
P
3
m
5
s
h
a
r
b
o
r
-
r
e
g
i
s
t
r
y
C
l
u
s
t
e
r
I
P
1
0
.
1
1
0
.
1
2
9
.
1
6
<
n
o
n
e
>
5
0
0
0
/
T
C
P
,
8
0
8
0
/
T
C
P
3
m
5
s
h
a
r
b
o
r
-
t
r
i
v
y
C
l
u
s
t
e
r
I
P
1
0
.
1
0
9
.
0
.
1
5
5
<
n
o
n
e
>
8
0
8
0
/
T
C
P
3
m
5
s
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26.

Loki brief introduction
Loki yes Grafana Labs The team's latest open source project , It's a level of scalability , High availability , Multi tenant log aggregation system . It's designed to be very cost-effective and easy to operate , Because it does not index log content , Instead, label each log stream .
Unlike other logging systems ,Loki It is built around the idea of indexing only metadata about logs : label ( It's like Prometheus The label is the same ). then , The log data itself will be compressed and stored in object storage in the form of blocks ( for example S3 or GCS) in , Even stored locally in the file system . Small indexes and highly compressed blocks simplify operations and significantly reduce Loki Cost of .
Use Helm Deploy Loki Log platform
Environmental preparation
1) Create a namespace
2) install Helm
If you have installed it helm 了 , You can ignore this step .
[[email protected] ~]
# wget https://get.helm.sh/helm-v3.7.2-linux-amd64.tar.gz
[[email protected] ~]
# tar zxvf helm-v3.7.2-linux-amd64.tar.gz
[[email protected] ~]
# cp linux-amd64/helm /usr/local/bin/
[[email protected] ~]
# helm versio
v
e
r
s
i
o
n
.
B
u
i
l
d
I
n
f
o
{
V
e
r
s
i
o
n:
"
v
3
.
7
.
2
"
,
G
i
t
C
o
m
m
i
t:
"
6
6
3
a
8
9
6
f
4
a
8
1
5
0
5
3
4
4
5
e
e
c
4
1
5
3
6
7
7
d
d
c
2
4
a
0
a
3
6
1
"
,
G
i
t
T
r
e
e
S
t
a
t
e:
"
c
l
e
a
n
"
,
G
o
V
e
r
s
i
o
n:
"
g
o
1
.
1
6
.
1
0
"
}
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
3) add to Chart Warehouse
[[email protected] ~]
# helm repo add grafana https://grafana.github.io/helm-charts
[[email protected] ~]
# helm repo list
N
A
M
E
U
R
L
g
r
a
f
a
n
a
h
t
t
p
s:
/
/
g
r
a
f
a
n
a
.
g
i
t
h
u
b
.
i
o
/
h
e
l
m
-
c
h
a
r
t
s
[[email protected] ~]
# helm repo update
- 1.
- 2.
- 3.
- 4.
- 5.
4) Search for chart
[[email protected] ~]
# helm search repo grafana
N
A
M
E
C
H
A
R
T
V
E
R
S
I
O
N
A
P
P
V
E
R
S
I
O
N
D
E
S
C
R
I
P
T
I
O
N
g
r
a
f
a
n
a
/
g
r
a
f
a
n
a
6
.
3
2
.
1
9
.
0
.
2
T
h
e
l
e
a
d
i
n
g
t
o
o
l
f
o
r
q
u
e
r
y
i
n
g
a
n
d
v
i
s
u
a
l
i
z
i
n
g
t
.
.
.
g
r
a
f
a
n
a
/
g
r
a
f
a
n
a
-
a
g
e
n
t
-
o
p
e
r
a
t
o
r
0
.
2
.
2
0
.
2
5
.
1
A
H
e
l
m
c
h
a
r
t
f
o
r
G
r
a
f
a
n
a
A
g
e
n
t
O
p
e
r
a
t
o
r
g
r
a
f
a
n
a
/
e
n
t
e
r
p
r
i
s
e
-
l
o
g
s
2
.
2
.
2
v
1
.
4
.
1
G
r
a
f
a
n
a
E
n
t
e
r
p
r
i
s
e
L
o
g
s
g
r
a
f
a
n
a
/
e
n
t
e
r
p
r
i
s
e
-
l
o
g
s
-
s
i
m
p
l
e
1
.
2
.
0
v
1
.
4
.
0
D
E
P
R
E
C
A
T
E
D
G
r
a
f
a
n
a
E
n
t
e
r
p
r
i
s
e
L
o
g
s
(
S
i
m
p
l
e
S
c
a
l
.
.
.
g
r
a
f
a
n
a
/
e
n
t
e
r
p
r
i
s
e
-
m
e
t
r
i
c
s
1
.
9
.
0
v
1
.
7
.
0
D
E
P
R
E
C
A
T
E
D
G
r
a
f
a
n
a
E
n
t
e
r
p
r
i
s
e
M
e
t
r
i
c
s
g
r
a
f
a
n
a
/
f
l
u
e
n
t
-
b
i
t
2
.
3
.
1
v
2
.
1
.
0
U
s
e
s
f
l
u
e
n
t
-
b
i
t
L
o
k
i
g
o
p
l
u
g
i
n
f
o
r
g
a
t
h
e
r
i
n
g
l
o
.
.
.
g
r
a
f
a
n
a
/
l
o
k
i
2
.
1
2
.
2
v
2
.
5
.
0
L
o
k
i:
l
i
k
e
P
r
o
m
e
t
h
e
u
s
,
b
u
t
f
o
r
l
o
g
s
.
g
r
a
f
a
n
a
/
l
o
k
i
-
c
a
n
a
r
y
0
.
8
.
1
2
.
5
.
0
H
e
l
m
c
h
a
r
t
f
o
r
G
r
a
f
a
n
a
L
o
k
i
C
a
n
a
r
y
g
r
a
f
a
n
a
/
l
o
k
i
-
d
i
s
t
r
i
b
u
t
e
d
0
.
5
0
.
0
2
.
5
.
0
H
e
l
m
c
h
a
r
t
f
o
r
G
r
a
f
a
n
a
L
o
k
i
i
n
m
i
c
r
o
s
e
r
v
i
c
e
s
m
o
d
e
g
r
a
f
a
n
a
/
l
o
k
i
-
s
i
m
p
l
e
-
s
c
a
l
a
b
l
e
1
.
6
.
1
2
.
5
.
0
H
e
l
m
c
h
a
r
t
f
o
r
G
r
a
f
a
n
a
L
o
k
i
i
n
s
i
m
p
l
e
,
s
c
a
l
a
b
l
e
.
.
.
g
r
a
f
a
n
a
/
l
o
k
i
-
s
t
a
c
k
2
.
6
.
5
v
2
.
4
.
2
L
o
k
i:
l
i
k
e
P
r
o
m
e
t
h
e
u
s
,
b
u
t
f
o
r
l
o
g
s
.
g
r
a
f
a
n
a
/
m
i
m
i
r
-
d
i
s
t
r
i
b
u
t
e
d
2
.
1
.
0
2
.
1
.
0
G
r
a
f
a
n
a
M
i
m
i
r
g
r
a
f
a
n
a
/
m
i
m
i
r
-
o
p
e
n
s
h
i
f
t
-
e
x
p
e
r
i
m
e
n
t
a
l
2
.
1
.
0
2
.
0
.
0
G
r
a
f
a
n
a
M
i
m
i
r
o
n
O
p
e
n
S
h
i
f
t
E
x
p
e
r
i
m
e
n
t
g
r
a
f
a
n
a
/
o
n
c
a
l
l
1
.
0
.
2
v
1
.
0
.
3
D
e
v
e
l
o
p
e
r
-
f
r
i
e
n
d
l
y
i
n
c
i
d
e
n
t
r
e
s
p
o
n
s
e
w
i
t
h
b
r
i
l
l
.
.
.
g
r
a
f
a
n
a
/
p
r
o
m
t
a
i
l
6
.
0
.
2
2
.
5
.
0
P
r
o
m
t
a
i
l
i
s
a
n
a
g
e
n
t
w
h
i
c
h
s
h
i
p
s
t
h
e
c
o
n
t
e
n
t
s
o
.
.
.
g
r
a
f
a
n
a
/
r
o
l
l
o
u
t
-
o
p
e
r
a
t
o
r
0
.
1
.
2
v
0
.
1
.
1
G
r
a
f
a
n
a
r
o
l
l
o
u
t
-
o
p
e
r
a
t
o
r
g
r
a
f
a
n
a
/
t
e
m
p
o
0
.
1
5
.
4
1
.
4
.
1
G
r
a
f
a
n
a
T
e
m
p
o
S
i
n
g
l
e
B
i
n
a
r
y
M
o
d
e
g
r
a
f
a
n
a
/
t
e
m
p
o
-
d
i
s
t
r
i
b
u
t
e
d
0
.
2
0
.
3
1
.
4
.
1
G
r
a
f
a
n
a
T
e
m
p
o
i
n
M
i
c
r
o
S
e
r
v
i
c
e
m
o
d
e
g
r
a
f
a
n
a
/
t
e
m
p
o
-
v
u
l
t
u
r
e
0
.
2
.
0
1
.
3
.
0
G
r
a
f
a
n
a
T
e
m
p
o
V
u
l
t
u
r
e
-
A
t
o
o
l
t
o
m
o
n
i
t
o
r
T
e
m
p
o
.
.
.
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.

grafana/loki-stack: Encapsulate multiple processes into one Pod Inside , No data persistence , For test environments , Monomer mode .
grafana/loki-canary: Canary update mode ;
grafana/loki-distributed: Microservice model , Suitable for large-scale production ;
grafana/loki-simple-scalable: Read write separation mode , Simple and scalable ;
Download and install Loki Chart
- Download decompression chart
[[email protected] ~]
# helm pull grafana/loki-stack --untar
[[email protected] ~]
# cd loki-stack/
[[email protected] loki-stack]
# ls
c
h
a
r
t
s
C
h
a
r
t
.
y
a
m
l
R
E
A
D
M
E
.
m
d
r
e
q
u
i
r
e
m
e
n
t
s
.
l
o
c
k
r
e
q
u
i
r
e
m
e
n
t
s
.
y
a
m
l
t
e
m
p
l
a
t
e
s
v
a
l
u
e
s
.
y
a
m
l
- 1.
- 2.
- 3.
- 4.
--untar: Download and unzip Chart package ;
- modify values.yaml To configure
[[email protected] loki-stack]
# vim values.yaml
g
r
a
f
a
n
a:
e
n
a
b
l
e
d:
t
r
u
e
#
Qi
use
G
r
a
f
a
n
a
Group
Pieces of
s
i
d
e
c
a
r:
d
a
t
a
s
o
u
r
c
e
s:
e
n
a
b
l
e
d:
t
r
u
e
m
a
x
L
i
n
e
s:
1
0
0
0
i
m
a
g
e:
t
a
g:
8
.
3
.
5
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- install
[[email protected] loki-stack]
# helm install loki-stack . -n loki
N
A
M
E:
l
o
k
i
-
s
t
a
c
k
L
A
S
T
D
E
P
L
O
Y
E
D:
T
h
u
J
u
l
7
1
3:
3
1:
0
2
2
0
2
2
N
A
M
E
S
P
A
C
E:
l
o
k
i
S
T
A
T
U
S:
d
e
p
l
o
y
e
d
R
E
V
I
S
I
O
N:
1
N
O
T
E
S:
T
h
e
L
o
k
i
s
t
a
c
k
h
a
s
b
e
e
n
d
e
p
l
o
y
e
d
t
o
y
o
u
r
c
l
u
s
t
e
r
.
L
o
k
i
c
a
n
n
o
w
b
e
a
d
d
e
d
a
s
a
d
a
t
a
s
o
u
r
c
e
i
n
G
r
a
f
a
n
a
.
S
e
e
h
t
t
p:
/
/
d
o
c
s
.
g
r
a
f
a
n
a
.
o
r
g
/
f
e
a
t
u
r
e
s
/
d
a
t
a
s
o
u
r
c
e
s
/
l
o
k
i
/
f
o
r
m
o
r
e
d
e
t
a
i
l
.
[[email protected] loki-stack]
# helm -n loki ls
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.

View authentication services
[[email protected] loki-stack]
# kubectl -n loki get pods
N
A
M
E
R
E
A
D
Y
S
T
A
T
U
S
R
E
S
T
A
R
T
S
A
G
E
l
o
k
i
-
s
t
a
c
k
-
0
1
/
1
R
u
n
n
i
n
g
0
7
4
s
l
o
k
i
-
s
t
a
c
k
-
g
r
a
f
a
n
a
-
6
9
9
6
6
b
c
f
9
4
-
l
p
4
z
5
2
/
2
R
u
n
n
i
n
g
0
7
4
s
l
o
k
i
-
s
t
a
c
k
-
p
r
o
m
t
a
i
l
-
c
k
p
s
h
1
/
1
R
u
n
n
i
n
g
0
7
4
s
l
o
k
i
-
s
t
a
c
k
-
p
r
o
m
t
a
i
l
-
g
7
k
f
t
1
/
1
R
u
n
n
i
n
g
0
7
4
s
l
o
k
i
-
s
t
a
c
k
-
p
r
o
m
t
a
i
l
-
g
f
8
6
b
1
/
1
R
u
n
n
i
n
g
0
7
4
s
l
o
k
i
-
s
t
a
c
k
-
p
r
o
m
t
a
i
l
-
j
n
z
n
c
1
/
1
R
u
n
n
i
n
g
0
7
4
s
n
f
s
-
p
r
o
v
i
s
i
o
n
e
r
-
5
6
f
d
4
7
f
c
9
c
-
l
k
t
r
v
1
/
1
R
u
n
n
i
n
g
3
2
3
h
[[email protected] loki-stack]
# kubectl -n loki get svc
N
A
M
E
T
Y
P
E
C
L
U
S
T
E
R
-
I
P
E
X
T
E
R
N
A
L
-
I
P
P
O
R
T
(
S
)
A
G
E
l
o
k
i
-
s
t
a
c
k
C
l
u
s
t
e
r
I
P
1
0
.
1
0
2
.
8
1
.
2
5
0
<
n
o
n
e
>
3
1
0
0
/
T
C
P
7
6
s
l
o
k
i
-
s
t
a
c
k
-
g
r
a
f
a
n
a
C
l
u
s
t
e
r
I
P
1
0
.
1
0
3
.
6
2
.
3
1
<
n
o
n
e
>
8
0
/
T
C
P
7
6
s
l
o
k
i
-
s
t
a
c
k
-
h
e
a
d
l
e
s
s
C
l
u
s
t
e
r
I
P
N
o
n
e
<
n
o
n
e
>
3
1
0
0
/
T
C
P
7
6
s
l
o
k
i
-
s
t
a
c
k
-
m
e
m
b
e
r
l
i
s
t
C
l
u
s
t
e
r
I
P
N
o
n
e
<
n
o
n
e
>
7
9
4
6
/
T
C
P
7
6
s
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
Pod All deployed successfully !
- modify Grafana The service access method is NodePort
[[email protected] loki-stack]
# kubectl -n loki edit svc loki-stack-grafana
s
p
e
c:
t
y
p
e:
N
o
d
e
P
o
r
t
- 1.
- 2.
- 3.

Grafana To configure
Sign in Grafana
[[email protected] loki-stack]
# kubectl -n loki get svc loki-stack-grafana
N
A
M
E
T
Y
P
E
C
L
U
S
T
E
R
-
I
P
E
X
T
E
R
N
A
L
-
I
P
P
O
R
T
(
S
)
A
G
E
l
o
k
i
-
s
t
a
c
k
-
g
r
a
f
a
n
a
N
o
d
e
P
o
r
t
1
0
.
1
0
3
.
6
2
.
3
1
<
n
o
n
e
>
8
0:
3
2
5
5
5
/
T
C
P
4
m
9
s
- 1.
- 2.
- 3.
see Grafana password

Browser login Grafana
- The login address : k8s node IP:NodePort
- user name :admin
- password : What I just saw secret Analytic string

Grafana see Loki journal

And then input : {namespace="harbor"} You can view it Harbor All logs of namespace .

边栏推荐
- It's worth seeing. Interview sites and interview skills
- Revit secondary development - project file to family file
- Customer case | China law network, through observing the cloud, greatly shortens the time of fault location
- UWA Q & a collection
- OpenGL jobs - shaders
- How to choose the appropriate automated testing tools?
- Antd date component appears in English
- Variables and constants
- 使用 CustomPaint 绘制基本图形
- Anti climbing killer
猜你喜欢

100million single men and women "online dating", supporting 13billion IPOs

双塔模型的最强出装,谷歌又开始玩起“老古董”了?

vite Unrestricted file system access to

How to quickly check whether the opening area ratio of steel mesh conforms to ipc7525

Time standard library
苹果在iOS 16中通过'虚拟卡'安全功能进一步进军金融领域

Blender exchange group, welcome to the water group ~

How to realize the movement control of characters in horizontal game
![[open source] Net ORM accessing Firebird database](/img/a2/4eff4f0af53bf3b9839a73019a212f.png)
[open source] Net ORM accessing Firebird database

海外代理推荐
随机推荐
Details of the open source framework of microservice architecture
UWA Q & a collection
Kaggle-Titanic
【Azure微服务 Service Fabric 】在SF节点中开启Performance Monitor及设置抓取进程的方式
ASP.NET Core入门五
Vs custom template - take the custom class template as an example
The difference between NPM uninstall and RM direct deletion
PDF文档签名指南
Form组件常用校验规则-2(持续更新中~)
23. Merge K ascending linked lists -c language
What is the difference between the three values of null Nan undefined in JS
Matplotlib快速入门
Application practice | the efficiency of the data warehouse system has been comprehensively improved! Data warehouse construction based on Apache Doris in Tongcheng digital Department
Record layoutrebuild Forcerebuildlayoutimmediate does not take effect
ByteDance senior engineer interview, easy to get started, fluent
Reinforcement learning - learning notes 9 | multi step TD target
Signal feature extraction +lstm to realize gear reducer fault diagnosis -matlab code
[azure microservice service fabric] the service fabric cluster hangs up because the certificate expires (the upgrade cannot be completed, and the node is unavailable)
Use json Stringify() to realize deep copy, be careful, there may be a huge hole
100million single men and women "online dating", supporting 13billion IPOs