Record a troubleshooting process - video call cannot be picked up

Preface

As the saying goes , People sit at home ,bug From the sky , Brother Kaizi's Dragon Boat Festival holiday is directly filled 10 God , His bug It's in my hands .

problem

Tangqiaoyun sdk Integrated IM、 Audio and video call and other functions .
The small program we developed IM Page is webview Inside H5 page , The applet also integrates audio and video calls and other functions .
So there are two websocket passageway ( One websocket establish IM signal communication , One websocket establish webrtc passageway < Forward signaling >).
But there are restrictions at the company level , The establishment of these two channels cannot use the same appId, So every time you hang up, jump back to the new IM webview Embedded H5 page ( Use different appId), But many official releases still have no effect , But advance is normal .

Solution flow

Advance is normal , But the formal environment is not allowed , First, let's look at the reasons why the release failed , No errors were found on the release platform , We log in to the official environment server directly , Find the corresponding project log journal , But in the release period , We didn't see any special error reports from the log .
That is to say, the packaging failure is not caused by the project error , This is basically consistent with our inference ( The advance environment is OK , There's a problem online ), That could be pm2 There is a problem , Let's see pm2 Log

pm2 log

Direct error as follows ：

node: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.14' not found (required by node)
node: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.18' not found (required by node)
node: /usr/lib64/libstdc++.so.6: version `CXXABI_1.3.5' not found (required by node)
node: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.15' not found (required by node)
node: /lib64/libc.so.6: version `GLIBC_2.16' not found (required by node)
node: /lib64/libc.so.6: version `GLIBC_2.17' not found (required by node)
node: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by node)

See this wrong guess , It could be node Environmental problems , perform node -v, It is also a mistake , That means there is no problem in guessing . Now look centos edition ,centos6.7, That could be centos Version and node Version incompatibility .
ok, At this time we go to find node Development Engineer Rego

It's settled , Is, indeed, centos Outdated Version ,node The version is too high ( Including my own server is centos7 It seems that there are some problems with the low version , See my previous article for details , upgrade centos Kernel ).
Then why is it possible to send in advance ？ Because the pre shipment uses containers , But the physical machines are used online ,ok Let's apply for two containers , No physical machines , Find a good talking O & M to review .

According to the standard items of the company , apply 2 nucleus 4G Two nodes of , Deploy the project after the application is successful . After successful deployment, there is no error message .

ok Looking for operation and maintenance Cut off flow , hold nginx Load balancing requests go to two containers + On the original host , We tail -200f xxx Look at the project log , There has been continuous flow to the container .

We continue to verify the online functions , No problem , Apply to drop the host .

Harvest

When I applied for the container , Most hosts that view other large projects are 4 nucleus 8G( More than one ), Asked the boss of the lower back end , At present, we pay attention to both front and back ends Distributed , namely Server cluster composed of low and medium configuration servers for many times . The reason for this is for better disaster recovery .

Of course , Speaking of disaster recovery , Think of our front end node ssr It is also a disaster recovery process that has been downgraded , Detect the server request volume threshold , If it is too large, it will be automatically controlled by ssr Downgrade to csr.
So for Disaster Management , Be sure to achieve multi-layer protection , This is quite important , Directly related to the company's profits .

summary

centos If the version is too low, it will be related to node Version incompatible
be familiar with linux Operation and nginx、pm2 It can really solve more problems
The distributed solution is really good , Disaster tolerant and cheap