[truth] the reason why big factories are not afraid to spend money is...

The Internet giant is the company that many developers aspire to , But everyone should have heard a joke about candidates interviewed by big factories ,“ Interview to build rocket , Working screw ”. Although this is a little exaggerated , But it is also described vividly .

In the interview , Especially for interviews with top Internet companies , The examination of technology is often very deep . But when it comes to work , It may really take a lot of time to write all kinds of repetitions  CRUD  On .

Why does that happen , Is it because the big factory has nothing to do but embarrass the candidate ？

It's not , This is because solid basic skills are really important to big factories .

One business feature that distinguishes big Internet companies from small companies is the massive number of requests , Any second echelon in the industry App, It is common to see hundreds of millions of back-end interface requests every day , Not to mention wechat 、 Taobao and other head applications .

At this level of user request , Business can 7×24 It is very important to provide services steadily for hours . Even if the service fails for ten minutes , The business losses may not be underestimated .

So in big factories , The program you write doesn't just run , It must be able to operate stably .

The program may inevitably encounter various online problems during operation . Applications are all run on hardware 、 On top of the operating system , So many online problems are related to the bottom . If you have an online problem , Do you have the ability to quickly check and handle ？ For example, sometimes online access times out because TCP The full connection queue of is full . If you don't know enough about this kind of low-level knowledge , Can't cope with .

in addition , The purpose of recruiting high-level programmers in large factories may not be just to deal with problems quickly , I even hope that programmers can make predictions before writing code , So as to avoid failure .

I wonder if your team has ever done Code Review（ Code review , abbreviation CR）. Often novice programmers feel good about themselves 、 You can find a lot of problems that may occur when you look at the well written code for senior programmers .

Big factories are not afraid to spend money on recruiting people , What I fear most is that the business is unstable and unreliable . If you hire average programmers at a very low price , As a result, the business went wrong every day , Loss of business income , That's not worth the loss .

therefore , If you want to enter a large factory , Solid internal skills are indispensable .

Talk about the growth after work

Does that mean you've already worked , Or have entered the big factory , Solid internal skill 、 Ability is dispensable ？

The answer, of course, is No , Internal skill is also important after work ！

Take the back-end development post as an example . Friends who are new to backend development will feel , This direction is too easy . I had this illusion when I first came into contact with back-end development . I just graduated from college to do Windows Under the C++ When developing , The code in the project is compiled to generate several projects GB Of , But after going to the back end, I found , A server interface may 100 A line of code is done .

Because it looks like this “ simplicity ”, Many back-end developers who have worked for about three years will fall into a growth bottleneck , I feel that I am very skilled in what I have in hand , programing language 、 frame 、MySQL、Nginx、Redis They are all used very smoothly , I always feel that I have nothing new to learn .

Do they really have it all ？ It's not , When they encounter some online problems , The means of investigation and location are extremely limited , It is difficult to bear the important responsibility of emergency fire fighting for online problems . When there is a bottleneck in program performance , Just search a few posts on the Internet , Try it like a blind man or an elephant , Adjust the kernel parameters that you know only a little about , Lack of deep enough knowledge of key technologies .

In contrast, some other experienced senior technicians

, They generally have a deep understanding of the bottom . When there are problems with online services , Can quickly find the key problems . Even if it is really a thorny problem , They also have the ability to sneak into the bottom , Such as kernel source code , Find out , See how the bottom floor works , Why is this problem .

Therefore, the big factory does not only inspect the candidates during recruitment , In the internal promotion selection, we also pay attention to the developers' understanding of the bottom layer and their ability to control the performance . How deep is a person's internal skill , It determines whether he has the basic ability of troubleshooting and performance tuning . Internal skill refers to the operating system you once learned 、 The Internet 、 Knowledge of hardware, etc . Internet services are based on these infrastructures , Only you have a deep understanding of them , Then we can continuously think of new performance analysis and tuning methods .

So , Solid internal skills are not useless after passing the interview of a large factory , It will run through your entire career .

Talk about middle-aged anxiety

Before that, the Internet once made a stir in an article titled “ The Internet doesn't need middle-aged people ” The article , Crazy Render 35 The future of the year-old farmer , Create anxiety .

Originally, I thought it was just a hype by the media , But just twoorthree years ago, our team expanded , We need to recruit some high-level developers , Then I had some other ideas about this topic . During that time, I interviewed more than 70 people , Many of them have worked for more than seven to eight years .

Of the people I interviewed , There are some people who have worked for more than sevenoreight years , But all the experience focused on the business logic of the little project at hand . There is no good idea for them to ask a little more performance related questions , The technical ability has not increased with the working years ⻓ And increase ⻓. let me put it another way , They don't have sevenoreight years of experience , Instead, it took twoorthree years of experience for sevenoreight years .

After communicating with these people , I find the common reason is that they spend most of their time dealing with various business logic and bug, There is no time and energy to improve their low-level technical ability , I really don't have the patience to study online problems , Try any article you search on the Internet , Whichever one is right is the end , Or simply leave the fault to the operation and maintenance personnel to solve , As a result, the technical level has been stagnant , It doesn't increase with the working years . I did realize that , There may be middle-aged anxiety in the Manon circle .

Does that mean that this anxiety is really insoluble ？

The answer must be “ No ”. At least some of the people I interviewed were excellent , Not only have rich business experience , And the technical ability is outstanding , At present, they all play an important role . You can also look at the high-level technicians in your company , Even the technical giants in the industry , I believe they will be the mainstay of your company and even the industry for a long time .

So of these two types of people who have worked for many years , What is the reason why the difference is so great ？ I think a lot , And I have discussed this issue with many people . The final conclusion is that the technology accumulation of the big cattle gradually increases with the increase of their working years , Especially internal skill , It is a huge difference from ordinary developers .

The bulls have a deep understanding of the bottom . Deep internal skill knowledge makes them learn new skills very quickly . for instance , In the eyes of junior developers , Probably Java Of NIO and Golang Of net A bag is two completely different things , So it takes a lot of energy to learn . But in the eyes of people with deep knowledge at the bottom , Both of them are just right epoll Different packaging methods , It's like changing clothes , It is much easier to understand .

Such a benign iteration goes on , Good technology compared with ordinary developers , The gap in the overall technical level is widening . The average developer is getting more and more anxious , They even began to worry that their technical level would be overtaken by young people who had just graduated .

The benefits of cultivating internal skills

Internal skill , It doesn't help you master the latest development language , Don't teach you fashionable frames , Will not take you into the hot artificial intelligence , But I believe it is you who become “ Daniel ” The only way .

Let me briefly list the benefits of cultivating internal skills .

1） Pass the interview of big factories more smoothly .

The interview of a large factory has a low level of technical examination , However, many answers on the Internet are shallow . Take three handshakes for example , Generally, the answer on the Internet only refers to the preliminary status transfer . In fact, the three handshakes contain many key technical points , For example, full connection queue 、 Semi connected queues 、 prevent syn flood attack 、 Queue overflow packet loss 、 Deep knowledge such as overtime retransmission . Take again epoll give an example , If you are familiar with its internal implementation , Understand its red black tree and ready queue , We know that the root cause of its high performance is that the process spends most of its time dealing with user work , Instead of switching contexts frequently . If your internal functions reach these underlying principles , It will certainly add a lot of points to your interview .

2） Provide sufficient... For performance optimization “ ammunition ”.

At present, performance optimization is one of the most important indicators for the promotion of senior and above engineers in large companies . In the absence of knowledge of the kernel , Everyone's optimization methods are generally blind and elephant like , The means are very limited , The practice is one-sided . When you understand the whole process of network receiving and sending , To the network in CPU、 Memory and other aspects of the cost of understanding will be very deep . This will greatly help you to analyze the performance bottlenecks in your project , So as to provide sufficient information for your project performance optimization “ ammunition ”.

3） The technical life cycle of internal skill is long .

Linux operating system 1991 Published in , It is still in full swing . For the author  Linus, I think he also has age anxiety , But he may be worried that he can't find a successor . Looking back at some technologies in the application layer , Especially many frameworks , Life cycle can be more than ten years. I already think it's very good . If your energy is all betting on these short-lived technologies , Do you think you can stop worrying ！ So I think I should quit being impetuous , Practicing your internal skills in a down-to-earth manner is one of the antidotes to your anxiety .

4） People with deep internal skills understand new technologies very quickly .

Needless to say, everyone in the industry “ Daniel ” 了 , Take myself for example . I haven't really done much Kafka Source code , But when I finished studying how the kernel reads files 、 After the kernel processes the whole process of network packets , Just “ Seconds understand ” 了 Kafka Why is the performance of the network outstanding . also , When I understand epoll After the internal implementation of , Look back Golang Of net package , Only then really understood the extremely exquisite to the network IO Encapsulation . So you really understand Linux Kernel words , Looking at the new technologies in the application layer is like wearing a perspective mirror , See the bones directly .

5） The kernel provides examples of excellent system design .

Linux As a system that has been honed for thousands of years , It contains a large number of world-class design and implementation solutions . At ordinary times, we are in our own business development , It also needs to be designed before coding . For example, when I first worked, I was responsible for scheduling data collection tasks , The implementation of the system partially refers to the process scheduling scheme of the operating system . Another example , How to efficiently discover a connection while managing a large number of connections IO event ,epoll Inside “ Red and black trees  +  queue ” Combination can provide you with a good reference . There are many, many examples of this . All in all , If you can  Linux Some of the best implementations of are moving into your system , It will greatly improve the implementation level of your project .

Fashionable things will eventually go out of fashion , But solid internal skills will accompany you all your life . Only with a deep internal skill background , You can walk more steadily on the road of development 、 go further .

How to cultivate internal skill ？

We usually use various languages to code business logic , No matter what you use PHP、Go, still Java, Both belong to the application layer . But the application layer is built on the physical layer and the kernel layer . I call the technical ability in the application layer external skill , hold  Linux  kernel 、 The technical ability of equipment physical structure is called internal skill . As I said before , At any stage of your career , Solid internal skills are very important .

Good. , Since internal skill is so important , Then find some low-level related materials to strengthen learning . But it's a pity. , I think there are some deficiencies in the internal skill direction of the technical data on the market at present .

Let's start with the technical articles on the Internet .

The current online technical articles 、 There are many blogs . When people encounter problems, they often search first . But did you find out , Search a lot of online entry-level materials , And deep internal skill 、 There are few articles that can go deep into the underlying principles .

• such as , Nowadays, most Internet applications are through TCP Connected to work , How many can a machine support at most TCP  Connect ？ According to the principle that , The whole industry is talking about high concurrency , This should be a very basic problem . But when I had this question , I searched all over the search engine and couldn't find a satisfactory answer . Then I just did it myself , It took more than a month to do the test , Edge raking kernel source code , Only then can we fully understand the problem .

• Another example , Most developers have been engaged in network related development . How does a network packet arrive at your process from the network card ？ The problem seems simple on the surface , But in fact, many performance optimization schemes are related to the receiving process , Whether you can deeply understand this process determines how many optimization measures you have available for network performance . For example, the optimization scheme of multi queue network card is to disperse the work in multiple networks at the step of hard interrupt  CPU  Nuclear , To improve performance . A few years ago, I wanted to make this problem completely clear , Almost searched the Internet , I went through all kinds of classic books and couldn't find the answer I wanted .

• Also like , The three handshake technical articles found on the Internet are all talking about some simple content , How does the client initiate  SYN  Handshake entry  SYN_SENT  state , Server response SYN And the reply  SYNACK, Then enter  SYN_RECV…… And so on . But actually , The process of three handshakes performs many kernel operations , For example, client port selection 、 Retransmission timer start 、 Addition and deletion of semi connection queue 、 Addition and deletion of full connection queue . Many online problems are caused by a problem in one of the three handshakes , Whether you can deeply understand this process directly determines whether you have the ability to quickly eliminate or avoid such problems online . There are too few articles on the Internet that can introduce three handshakes in depth .

You might say , The articles on the Internet are not good enough , There are many classic books ？

First I have to say , Some classic computer books are really good , It's worth seeing , But there are several problems .

• First, the books at the bottom are all written in a profound and difficult way , You seem to be spending a lot of time . If you have already worked , It's hard to have so much time to chew . For example, when I first started to explore the network implementation , I bought it 《 In depth understanding of Linux kernel 》《 In depth understanding of Linux Inside of network technology 》 Wait for a few books , It took nearly a year to understand a general idea after work .

• Another problem is when you really encounter some confusion at work , You will find it difficult to have a classic book that can give you a direct answer . For example 《 In depth understanding of  Linux  Inside of network technology 》 This book introduces the various components in the kernel , Such as network card equipment 、 Neighbor subsystem 、 Routing, etc. , I talked about the relevant source code again . But after reading it, I still don't know how a package goes from a network card to an application , How many servers can a server support TCP Connect .

• Another problem is that computer technology is different from other disciplines , In addition to theory, it also has higher requirements for practice . If you just stay in the theoretical stage of classic books , In fact, many problems cannot be understood at all . These books often lack hands-on experiments related to practical work . For example, how many servers can a server support TCP Connect this problem , It was only after many experiments that I understood clearly . And if you haven't really touched it , Then you won't be able to talk about online performance optimization in the future .

in general , Reading these classic books is one way , But consider the cost of time and the precise handling of work problems , I feel that the efficiency is relatively low . So in view of this , I decided to output something , There was also

《 In depth understanding of Linux The Internet ： Cultivate the basic internal skill , Master the principle of high performance 》

The publication of this book .

Creative ideas

Although the underlying knowledge is so important , But this kind of knowledge has a common feature that it is very boring .

So how can we talk about the boring bottom well ？

I have thought about this problem many times .

2012 I worked at Tencent in , In the internal KM An article was published on the technical forum , called 《Linux Ten questions about the file system 》（ This article can now be found on the Internet , Because it was moved many times ）.

The background of writing at that time was “ The eldest brother ” Assign me a task , Download and save the image files in the data provided by all partners . I traced the source of several questions arising from my work , After finding the answer, he wrote an article and published it . For example, where does the file name exist , Does an empty file occupy disk space ,Linux What's wrong with too many subdirectories in the directory , wait . After this article was published , It has spread all over Tencent , The response was great , Finally, it became Tencent KM The hot article of the year .

Why is my article simple Linux File system articles can get such a strong response ？ Later, I found the answer in a program of logic thinking . The program says that the best way to learn is to ask yourself some questions , Take these questions to the ocean of knowledge to find answers , When the answer is found , That is, when you really master this knowledge . The knowledge acquired through this process is the most profound , The degree of integration with yourself is also the highest , Can be fully internalized into your ability system .

It is the same to consider from the perspective of readers . In fact, readers are not interested in the underlying knowledge , They are interested in solving practical problems in their work .

This article is not really about file systems , It is about the problems that may be encountered in the development process . I just use file system knowledge as a tool , Use it to solve these practical problems .

So in the process of writing this book , This idea runs through all the time ：

Focus on practical issues related to work .

In each chapter , I'm not going to instill soft interrupts in you from the beginning 、epoll、socket  Knowledge of kernel network modules such as kernel objects , I also find these very boring , Instead, each chapter throws out several practical problems related to the development work , And then explore around these issues . Yes , The word I use is not “ Study ”, It is “ seek ”. Compared with learning , Exploration emphasizes curiosity about the problem to be solved , More interesting .

Although this book will involve a lot of source code , But first of all, let's emphasize , This is not a source code analysis book . The real purpose of learning is to understand and solve problems related to project practice , And then improve the ability to control the work at hand , The source code is just a tool and a way to achieve our goals .

Applicable readers

This book is not an introduction to computer networks , Reading this book requires you to have a basic knowledge of computer network . It is suitable for the following readers ：

• Readers who want to enter a big factory by improving their network skills .

• Not satisfied with only learning network protocols , Readers who also want to understand how it is implemented .

• Although I have several years of development experience , But developers who are unsure about network overhead .

• Want to do network performance optimization , But readers without systematic theoretical guidance .

• O & M personnel who maintain various high concurrency servers .

Scan code for details of this book ！