As a programmer, is it really that important for the underlying principles?

Some time ago, I encountered a technical problem in my work ,  Finding problems , Think about the problem , In the process of solving the problem , Suddenly, I have some thoughts on the underlying principles , Let's share it with you .

background

Used in business  Redis  Database to store data , But when storing large amounts of data  string  When , Monitoring found that the memory growth was abnormal .

string  type , As  Redis  The only one “ key - Single value ” type , It's easy to operate , Easy to understand , According to the principle, only the character data is saved , How can it consume so much memory , What's going on here ？

With the spirit of getting to the bottom , I'm on my own  Mac  Hands on practice on the machine , I wrote a paragraph  Python  Code , Prepare data length  7  digit , common  100  Ten thousand data , Start running data .

Results found , Use  string  Occupy  70 MB, Use  hash ziplist  Occupy only  9 MB！ The effect is very obvious .

test result ：

begin write 100w
before: used_memory_human:1.87M
before: used_memory_human:70.90M
end write 100w, using: 72380608

This data effect , There is no difference without comparison .

When I can't understand it , I searched the Internet for other people's answers , I've read some, but I feel I haven't touched the point , Until today, in the column of zero sound education , To study the  Redis  Of  string  Bottom source .

【 Article Welfare 】 In addition, Xiaobian also sorted out some C++ Back end underlying principle development interview questions , Teaching video , Learn the road map for free , You can add what you need ：

Click to join the learning exchange group ~

  Group file sharing

Xiaobian strongly recommends C++ Back end underlying principle development free learning address ：

C/C++Linux Server development senior architect /C++ Background development architect ​

Redis  Of  string  The type and  hash ziplist  Different underlying data structures lead to different storage sizes , I just found out that , That's what happened ！

Now I call you guys .

string  type , Often regarded as  Redis  The golden oil in , But it has an obvious weakness , It consumes the most memory space when saving data .

For not knowing  Redis  People with underlying principles , It's easy to mistake the simplest type , It should consume the least resources . But this is not the case .

Redis string  The underlying structure , Additional data structures are used to store data , This creates additional overhead , Like this ：

This reminds me of a previous business problem , After an old service modifies a function and goes online , Test students' reflection , The pressure measurement time soars .

Everyone thought it was strange , The time consumption of this old service has been very stable , and , This modification is just normal business logic , People are still puzzled about it , It took a day to find out .

In the end, we found out , The culprit is that the online log data is too full , This causes the machine memory to rise instantaneously , It directly affects the time-consuming statistics of pressure measurement .

later , I find , At that time, this service was compiled with an option , Turn on  log=debug  Pattern , If we could understand the principle behind this in advance , It is estimated that this pit can be avoided .

Inner doubt

Internet industry , Update iterations are very fast , I learned this today , I may learn that again tomorrow , In the process of continuous learning , Believe that your heart will sometimes ask yourself , Is the underlying principle that important ？ Can't you just use it ？

This is the time , There may be another sound in your ear , Ten thousand Zhang tall buildings rise from the ground , Basic computer knowledge is like the foundation of the programmer's pyramid , Understand the underlying principles of computer systems , You can only go further and further on the road of writing programs .

That's right , But is this really enough ？

The deeper reason is ：

• Often use some unknown Technology , I feel uneasy ;

• In this age of data explosion , Many systems have high requirements for project performance optimization . So how to tune becomes an inevitable problem for programmers ;

• And if you don't understand the underlying principles of a computer system , Maybe all the programs you write are wrong .

For example, we always thought that the sum or product of two positive numbers must be positive , But the sum or product of positive numbers represented by binary complement is not necessarily ;

Programmers and compilers don't work （x-y<0） Instead of （x<y）, Because the former will cause overflow and so on .

Further more , Internet factory interview , Ask the source code of the bottom layer 、JVM  Structure 、TCP/IP  Three handshakes of 、 Four waves , WeChat / How to meet the high concurrency of the data of pinduoduo users ？ wait .

Questions like these , If we don't understand the basic knowledge of computer, we can't pass the test .

Why is the underlying principle so important

The opening example introduces , I mean to say , Learning the underlying principles is very important , Just as data structures and algorithms are programmers' internal skills .

I am studying after work recently 《MySQL actual combat  45 speak 》, The author shared this passage , I think it's very good , Share it here .

When I was bringing new people , Ask everyone to write  SQL  At the time of statement , In my mind, I know , Know the result of each statement execution , And what resources are consumed by this code 、 If it is slow, where will it be 、 Which locks will be occupied by the execution of each statement, and so on .

Some newcomers will ask “ Why is it so troublesome , I'll do it , See if the result is right , By the way , If not, change it , Is it possible to ？” I said no . Because if so , We will be subject to many limitations , Even if we position ourselves as business developers .

Here I say a limitation ：

This limits the ability of database based business architecture . A statement can try , A five statement transaction analysis needs to be tried many times , Database design of a complex business system , It is impossible to try .

Principle can help us prune , Eliminate those schemes that are obviously wrong in theory , Only in this way can we have the energy to really try those Limited 、 Maybe the right solution .

We don't need to  100%  Master  MySQL ( I am far from this goal myself ）, But as long as you know more about the principles , You can cut more branches , Architecture design can reduce the interference of wrong options , The designed project architecture is more likely to be correct .

I especially like the process and feeling of pruning , He said that I used my previous study time , To save working time now .

Of course ,“ principle ” It's a big concept , Some principles are closer to actual combat , Some are far away . In this column, I choose the principles related to the usual use , So that everyone can have the opportunity to learn while using .

Come on

In addition to the need to constantly explore the underlying principles , As an Internet technology practitioner , We also need to set up a correct view of the computer world ：

Here is a principle shared by Mr. wengkai of Zhejiang University ：

Learning computer must have a very powerful psychological state , Everything about computers is made by people , Others can think of it , I'm sure I can figure it out , There is no black magic in the computer world , Everything is just that I don't know now , One day I will give all the details 、 All the internal things are understood .

Okay , Said so much , Actually, I want to tell you , It's easy to make a salted fish , If you want to be a yard farmer all your life , You don't have to learn principles .

It's not just technology , test , operating , product , Master the underlying principles behind , Believe that you will gradually widen the gap with others , Because you have seen things from a different dimension .

Understand the underlying principles behind it , It is easier for us to learn by analogy , The lines .

If you are a salted fish with only dreams , People with the same ideas , People who are not determined to learn all the time , We can go together , Mutual supervision , Learn from each other .

Reference material

Recommend a zero sound education C/C++ Free open courses developed in the background , Personally, I think the teacher spoke well , Share with you ：

C/C++ Background development senior architect , The content includes Linux,Nginx,ZeroMQ,MySQL,Redis,fastdfs,MongoDB,ZK, Streaming media ,CDN,P2P,K8S,Docker,TCP/IP, coroutines ,DPDK Etc , Learn now

original text ：

Is the underlying principle that important ？