My brave way to line -- elaborate on what happens when the browser enters the URL

Preface

In fact, I have learned this problem TCP Students can say one or two , That is, what is the general process , But if you pull it out and sort it out in detail , There are still many knowledge points that need to be sorted out . Think about it , If the interviewer asks ： You talk about browser input URL What happened after that , Then barabara finished speaking in a short meeting , Maybe we missed a good opportunity to show ourselves . So to avoid this happening , This blog was born .

I. monologue of a small packet –HTTP agreement

What did the browser do first ？ It must be first Parse your input URL, Then generate a feed to Web Sent by the server Request information .
Then let's start our analysis

1. About URL–URL Format analysis of

The first is to analyze URL, Then let's see , One URL It contains those information .

Use an example to show , for instance ：

http://    In fact, that is    Name of agreement ://
www.server.com   Namely web The server 
/dir1/file.html    Directory file path 

So a URL The composition is still very simple , Maybe a careful iron will ask , You look at ！ You said in the blue part above , Can be omitted ！

Then you omit the words , How do you know the source file path ？ ah , On this question , Look at the picture below

When we omit the file path

If there is no file path , It means to access the default folder set in advance in the root directory , for instance :/index.html Or /default.html

2. The birth of packets –HTTP Those things

Okay ,URL Finished parsing , got it Name of agreement , got it Web The address of the server , I also know what to visit file , Then is it time to send a request message to our remote server friend ？ I haven't seen you for a few days , Brother Wei needs your help , I am writing to , As a sign of urgency .

Then start writing , A letter should be written in a format , Otherwise, brother of the destination server will have a look ： Long time no see , It's not easy to write a letter. You still have this attitude ？ Who are you perfunctory ？ So our packet generation must follow HTTP agreement .
About HTTP Method name of the Protocol common There are two kinds of （ It's used a lot ！！！！ These are not the only two , Here are two ）

$pleaseseeknewspaperwriting$

Generally speaking , If you want to send some local specialties to the server brother, you can use POST, If you want my brother to send you some local specialties, you can use GET.

$ringShould\; benewspaperwriting$

There is actually a problem I want to mention here , That is, whether it is a request message or a response message , There is a message body , There are many common headers here , I really want to write something here , But considering the overall length , I dare not elaborate too much here , If you are interested, please see my next two blogs .（ It even includes HTTPS, One way and two-way authentication , Status code , There are many things to say here ）

JavaWeb–HTTP agreement ( On )
JavaWeb–HTTP agreement ( Next )

Actually, think about it carefully , About this part, do I write a blog to pull out the interesting things and hammer them separately ？ Can you can ！！ After writing this blog, I will do !!

So here we are , One of our simple , follow HTTP Agreed Data packets It's generated . So here comes the question , How can I get there ？ What if I get knocked out in a black sack and take it away in the middle of the race ？ Where is my destination ？ Is the atmosphere suddenly full ？ But even if the road ahead is stormy , Should I go or should I go , That's it , Our data package is on its own way to find relatives ！

2. Inquire about the address of relatives – About address inquiry

Browser parsing URL And after generating the data package , It is necessary to entrust the operating system to send messages to Server side Web The server . But you need to know who to send it to , So we need Query the corresponding of the server domain name IP Address , You entrust the operating system to send messages , You must know the destination IP, That is, the address of the host . So we need help from a big brother , That is, the root server DNS, It has been saved. Web Server domain name and IP Correspondence of address .

1. Jiangnan Jiangbei Street – About the hierarchical relationship of domain names

Why should I name this paragraph ？ Because I used its next sentence as the title , Of course, it doesn't matter , Return to the right topic .

DNS The domain names in are separated by periods , such as www.server.com, The period here represents different levels boundaries . In our domain name , The higher the right-hand domain name level （ Don't ask me why it's like this , You have to ask foreigners why this habit ）. The root domain is at the top , Is the next layer com Top-level domain , Until then server.com. So the domain name relationship is actually a tree

root DNS The server => Top-level domain DNS The server （com） => authority DNS The server （server.com）

Root domain DNS Server information is stored in all of the Internet DNS Server , So as long as the client finds any one DNS The server , You can find roots through it DNS The server , Then follow the steps to find one at the lower level DNS The server .

2. The vast road of looking for relatives – About domain name resolution

1. First, the current server sends a DNS Request to local DNS The server , ask ： The local DNS Server brother , Do you know www.server.com Where's brother's ？（ Quietly Mimi asked my brother IP Address , It's local DNS The server is the client TCP/IP Fill in the settings DNS Server address ）
2. Local domain name server （DNS The server ） After receiving the request from the client , If found in the cache table www.server.com（PS： LAN ）, Then put the corresponding host IP The address to return . If not found , So local DNS The server will run to ask it Root domain server Big brother ： eldest brother ,www.server.com Little brother's IP What is the address ？ Big brother is busy everyday , He won't tell you directly , It will tell you where to find .
3. root DNS The server received a message from the local DNS Server requests , See the back is .com, Just say ：www.server.com It's your second brother .com Site , You ask it , And then local DNS The server received Top-level domain name .com The address of .
4. Local DNS The server sends a request again ： Second brother , Can you tell me www.server.com The address of ？ The top-level domain name server received from the local DNS Server requests , say ： I'll give you your third brother ： Authoritative server server.com The address of , That's its territory , Where can you find the answer .
5. And then local DNS The server ran to ask server.com The authority of the DNS The server ： Three elder brother ,www.server.com Of IP What's the address ？server.com Brother, look , Roar ？ This is not my territory, that who who who ？ Then put the corresponding host IP The address told the local DNS The server .
6. Local DNS The server finally ran back ： Datagram little brother , Of your relatives IP The address is XXX. That is to put this DNS The server returned to the client .

Here we are. , The packet will know where the destination is , But I have nothing , So it's a little dangerous to go alone , It needs to find some friends to set out with itself .

3. About the author's supplement here

The search here does not mean to go directly to the root at the beginning DNS The server queries , It is

 Find the browser cache 
 Find the system cache 
 Find router cache 
 Find the operator cache 
 Finally, if you still can't find , Just go to the root DNS Server search .

3. Go on the road in groups – Packet encapsulation

Our little packet has finally reached this stage , But up to now , I haven't stepped out yet , But it doesn't matter , Keep going down , We go through DNS To obtain the IP Address , You can put the HTTP The transmission work of is given to Protocol stack , Refer to our TCP Five layer model .

First, after encapsulating the data package in the application layer , call Socket library , To delegate to the protocol stack （ Transport layer and network layer ）, There are two pieces in the top half of the protocol stack , They are responsible for sending and receiving data TCP and UDP agreement , Both of them will accept the entrustment of the application layer to perform the operation of receiving and sending data . The bottom half of the protocol stack uses IP Protocol control network packet sending and receiving operation , When uploading data on the Internet , The data will be cut into pieces of network packets , The operation of sending network packets to the other party is done by IP conscientious .
Just to mention here ,IP The agreement also includes ICMP The protocol and ARP agreement .

ICMP It is used to inform the errors and various control information in the process of network packet transmission .
ARP For use in accordance with IP Address query corresponding Ethernet MAC Address .

IP Under the agreement Network card driver （ Data link layer ） Is responsible for controlling the network card hardware , And the bottom network card Is responsible for the final actual receiving and sending operation . A look at the packet , good heavens , It's really impossible without help , So call friends , Prepare to gather and set out on the road .

1. Reliable transmission, little brother –TCP

<1> Briefly TCP agreement

HTTP The protocol is based on TCP Protocol transmitted ,, Let's talk briefly first TCP agreement

About HTTP The agreement didn't say much , here TCP The agreement cannot be bypassed

 Source port number and destination port number ： The process of the application corresponding to the source host and the process of the application corresponding to the destination host .

 Serial number ： Prevent package disorder 

 Confirmation no. ： Determine whether the other party has received 

 Status bit ： Maintain the status of both parties 

 Window size ： The guarantee of reliable transmission mechanism and efficient transmission mechanism .

ah , That won't be possible , Briefly explain the above , Here I want to steal a lazy , If there is a serious brother , I'll cheat another wave of browsing . About TCP See here for a detailed explanation of the agreement

About TCP Detailed explanation of the agreement

<2> Three handshakes

HTTP The protocol must establish a connection before sending packets , This is our TCP It's time for the agreement to work . Process of connection , That is to say, three handshakes , Three handshakes are just a state of maintaining both computers , The situation is as follows ：

1. The initial state is both Close, First, the server is in the listening state , That is to say LISTEN
2. The client sends... To the server SYN Data packets , Apply for connection . After sending the packet, the client is in SYS_SENT state .
3. After the server receives it , Returns one to the client SYN Data packets （ This packet contains SYN and ACK package , This is a piggyback response mechanism ）. After returning, the server is in SYN_RVCD state .
4. The client receives... From the server SYN After package , Return a ACK Data packets .
5. After waiting a while , Client access ESTABLISHED state . And as soon as the server receives AKC The packet immediately enters ESTABLISHED state .

The purpose of three handshakes is to ensure that both sides have the ability to send and receive messages .

Here are a few points I want to talk about

$1.Such\; asWhatcheckseeTCPevenPick\; upshapestate$

 Can be in Linux Pass through netstat-napt Command view 

$2.TCPbranchcutCountAccording\; to\; the$

What do you mean ？ It's just that there are too many things to take with you on the way , There is no way to go there at one time . Put it another way , Namely HTTP The packet message is too long , More than the MSS The length of , This is the time TCP Will the HTTP The data is split into pieces and sent , Instead of sending all the data at once .

First, let's see what a completion packet looks like （ By TCP and IP Wait for the complete data package after joint help ）

MTU： Is the maximum length of a packet , Ethernet is generally 1500 byte .
MSS： remove IP and TCP After the head , What a network packet can hold TCP Maximum length of data .

The data will be MSS Split in units of length , After being split, each piece of data will be put into a separate network packet , That is to add TCP Header information , And then to IP Module to send data .

$2.TCPnewspaperwritingrawbecome$

TCP There are two ports in the Protocol , One is the browser monitoring window （ Usually randomly generated ）, The other one is Web The port on which the server listens （HTTP The default port is 80,HTTPS The default port is 443）. After the two sides establish a connection ,TCP The data part of the message is stored in HTTP Head + data , Assembled TCP After the message , Just give it to the lower level , That is, the network layer . The data package at this time is as follows

At this time, the little packet is excited and can't help itself , Okay ！ I have a TCP This reliable little brother , Don't worry about safety anymore , So where should I go next ？

Here's an extra sentence , If yes TCP Carrying data, including other issues, can be seen in my blog below , There should be all the knowledge that readers want to know .

About what you want to know TCP knowledge , There are... Here

2. Remote location little brother –IP

TCP The module is performing the connection , Send and receive , Disconnect and other stages of operation , You have to delegate IP The module encapsulates the data into a network packet and sends it to the communication object . Let's have a look at IP Form of agreement

here IP In the protocol format, we need to pay attention to two places

 source address IP： That is, the client side IP Address 
 The goal is IP： That is, the server IP Address , That is, it is obtained through domain name resolution Web The server IP

because HTTP after TCP transmission , therefore IP Baotou Agreement No , Namely 06（ Hexadecimal ）, Said is TCP agreement . Then there is a little friend here , My purpose IP Address , But my own IP What's the address ？ Here is the bottom

About multiple network cards IP situation

If I have multiple network cards at this time , There are undoubtedly many at this moment IP Address , How do I know that is the network card I want to use ？ At this time, we will use our Routing table The rules , To judge , That network card can be used as Source IP.
At this time, we will check the routing table of the current system , for instance Linux In the system , We can order route -n Check it out. . for instance

Destination： Destination address 
GateWay： The router IP Address 
Genmask： Subnet mask 

Here we assume our purpose IP The address is 192.168.10.200 . The first and enth0 That is to say First item Subnet mask for bitwise sum

Results and Destination address Destination Dissimilarity , And then enth1 That is to say The second item Subnet mask for bitwise sum

Then find out the results and Destination equally , So just use it. eth1 NIC ID Address as source IP Address .

Here's a special explanation , If there is no one in all the entries Destination Can match on , Then it will be our third item （ Here, take the picture as an example ）, This is the default gateway , its The destination address and subnet mask are both 0.0.0.0, It will finally send the received packets to the router .

Here we are. , Our remote location work is done , At this time, our data package , ah ！ incorrect , Network package （ Little brother has upgraded ） It's like this

The packet is ecstatic at this time ！！！！！

I do IP The person protected by the little brother ！ It will show me the way , Let me have the ability of remote location , Will not be confused in the network . But ！ The destination is the destination , Where do I go next ？

3. Two point transmission, little brother –MAC

Generated IP After the head , The network package also needs to be in IP Add before the head MAC Head , The specific appearance is probably like this

our MAC The header needs the sender MAC Address and receiver's MAC Address of the , Because you want to use it for transmission between two points, not ? So in general TCP/IP In the correspondence ,MAC The protocol type of the header only uses ：

0800：IP agreement 
0806：ARP agreement 

<1> LAN

Here, it is easier for the sender

Every network card is produced , He wrote it ROM in , Read out this value , Write to MAC Just head .

But the receiver's here is more ambiguous , Because we know the purpose of the next item IP And finally IP Address . What to do here ？ It will involve our ARP agreement

ARP The protocol will shout in the Ethernet ： This IP Whose address is ？ Take yours. MAC Give me the address , Then someone will answer ： This IP The address is mine , my MAC The address is xxxxx. If the other party and oneself are in the same subnet , that MAC The address is complete .

Of course , The words here are not directly shouted as soon as they come up .
1. First query ARP cache , Among them, if it is preserved IP Address corresponding MAC Address , Then there is no need for ARP Inquire about , Use it directly ARP Address in cache .
2. If it doesn't exist , First of all, based on the current host IP And purpose IP, Perform bitwise sum with the local subnet mask , If the network number obtained is consistent , Then shout ！ That is to put the purpose MAC Set to FF:FF:FF:FF:FF:FF, Then it is sent to all connected hosts for ARP Broadcast inquiry .

<2> Wide area network

At this time, the problem comes , According to IP You can't find the address for the next jump MAC Address , It's no use shouting , Are not in the same network segment , So what happened ？ Then take the next jump first MAC The address is set to the router MAC Address , Then prepare to be received by the network card and sent to the switch , Finally, it is handed over to the router by the switch .

4. Gate exit – network card

IP The generated network packet is just a string of binary digital information stored in memory , There is no way to really give it to each other , So we need to take Digital information is converted into electrical signals , Can be transmitted on the network cable . The network card is responsible for this implementation , The network card driver is also needed to control the network card .
Network card driver from IP After the module gets the package , It will Copy To the cache of the network card , Then, the header and start frame decomposition character will be added at the beginning of it , Add at the end FCS That is to say Frame check sequence for detecting errors

Then the network card will convert the packet into an electrical signal , Send it over the Internet .

5. Brother came to see you off – Switch

First , Switch is a big brother who lives in layer 2 network equipment , After hearing that so many brothers are leaving , I must see you off before I leave , After all, deep feelings .
When the electrical signal reaches the network cable interface , The module in the switch receives , then

 The switch module converts electrical signals into digital signals 

After being converted into digital signal , Big brother, look left , Look right ： Little brothers , There is nothing with you ？

 This step is to check the error through the error check frame sequence code at the end of the packet .

If there is no problem , Then big brother can safely let them go

 After verification , If there is no problem, put it in the cache .

The above processes are somewhat similar to the operation of the network card , But there are still some differences , Because of different working methods .

 The computer network card itself has MAC Address , And check the received package MAC Address to determine whether it is sent to yourself , If not, discard .
 The port of the switch does not check the receiver MAC Address , Instead, all packets are directly received and placed in the cache .
 So the difference is ： The port of the switch does not have MAC Address 

After putting the package into the cache , Next, you need to query the recipient of this package MAC Address , Is it already in MAC There are records in the address table . Switch here MAC The address table has two main parts

The first is the equipment MAC Address
The second is that the device is connected to the port of the switch

So the switch will be based on MAC Address table lookup MAC Address , Then send the signal to the corresponding port .

If no corresponding MAC Address ？

if MAC There is no corresponding... In the address table MAC Address , How make ？ I can't find it here MAC There may be two reasons for the address

1. Probably because the device with this address has not sent a packet to the switch
2. The working address may be deleted by the switch because the packet has not been sent for a long time

In this case , The switch will send this packet to all ports except the source port , Don't worry about what problems this will cause , Because only the corresponding recipient will receive this packet , Other devices will directly ignore , Because everyone is busy , There is no time to deal with things that have nothing to do with yourself .
Don't worry about causing network congestion , There are many packets sent on the network , It won't make any difference if you have more , And this time after sending the response package, the corresponding relationship will be written into MAC Address table , So there's no problem .

Besides , If the address of the receiver is a broadcast address , The switch will also send packets to all ports

MAC Address ：FF:FF:FF:FF:FF:FF
IP Address ：255.255.255.255

At this point , The packet will leave the subnet where it was born and raised ！ About to enter a new world ！ The data package said to the big brother moved ： Brother crab, send it to the exit for me , I'm going away .

6. Exit gate – Router

1. The difference between switch and router

 After the network packet passes through the switch , It will reach the router , And it is forwarded to the next router or 
 Target device , You also need to query the table to determine the destination of packet forwarding , This is similar to the switch , however 
 Different ways of working .
 Router ： be based on IP Design , Commonly known as three-layer network equipment , Each port of the router has MAC Address and IP Address .
 Switch ： Based on Ethernet Design , Commonly known as layer 2 network equipment , No port has MAC Address , Network management switches have IP Address 

2. The principle of router

 The ports of the router are MAC Address , So it can be the receiver and sender of Ethernet ; He also has IP The earth 
 site , All from here, it is the same as the network card .
 First, it will receive Ethernet packets sent to itself , Then the router queries the forwarding target , Then the corresponding port is used as 
 The sender sends the Ethernet packet .

3. Receive router packets

 First , The electrical signal arrives at the interface part of the network line , The modules in the router will convert electrical signals into digital signals , however 
 After passing the end FCS（ Error check sequence code ） Error checking . Then check that the MAC Reception in the head 
 Fang MAC Address , Throw away if you have problems , If there is no problem, put it in the cache .

4. Query the routing table to determine the output port

 After the packet receive operation is completed , The router will take out the beginning of the packet MAC Address , After all, you MAC The function of address is 
 Is to send packets to the router , Just throw it away when you use it up （MAC The little brother was ruthlessly abandoned ）, Then route 
 The device will be based on IP The content in the header is used for packet forwarding 

Query the routing table first

 Then there is the subnet mask and purpose of each entry IP Address by bit and operation , The visible result is 
 Is the destination address of the corresponding entry , That's the first line Destination Whether the match is consistent , If you don't 
 find , Then choose the default route 

5. Sending operation

Now it's time Sending stage . First, determine the address of the other party according to the gateway column of the routing table

If the gateway is a IP Address , Then this address is the destination address we want to forward , Prove that we haven't reached the end , You need to continue forwarding
If the gateway is empty , that IP Receiver in header IP The address is the destination address to be forwarded , That is, finally find IP The destination address description in Baotou The end is coming .

Almost like this , If the address of the gateway is empty , that IP The head of the IP The address is the destination address to be forwarded .

got it IP Address , Next, the routing table will be based on ARP Protocol query MAC Address , And take the result of the query as the... Of the receiver MAC Address , It's still the same , Check the cache first , If not, proceed ARP Query request .

The sender MAC The address is the output port MAC Address
Fill in the Ethernet type field of the sender 0080（ Hexadecimal ） Express IP agreement

When it's done , It will be converted into an electrical signal and sent out through the port , Then the network packets sent reach the next router , The next router receives the cache and forwards it again , Keep forwarding, keep forwarding , Finally, we arrived at our destination . So what has been changing in the middle is MAC Address , Source IP Address and purpose IP It will never change . because MAC The address transfers packets between two devices in Ethernet .

6. About MAC Wan problem when obtaining address

What does that mean ？ It's yours according to your purpose IP, You can't find the next hop MAC Address , When you can't shout , What are you doing ？ Attention, huh , Not shouting here is not the result of no response after shouting , But getting the goal IP Address , Then according to the router IP And purpose IP The result obtained after performing bitwise sum operation with the subnet number in the current network segment and finding that the result is inconsistent .
This time is based on NAPT agreement , Put our source IP And source port (port) Into the public network . At the same time, the router based routing function , Find out about the destination IP Set the next hop while taking the nearest route MAC Address .

Why do I need to put the LAN IP And the port is set to public IP And ports ？ Because the response datagram returned by the receiving end , Destination port and destination IP Unable to use LAN IP And port .

7. Friends come from afar

Finally, the packet arrived at the server , From the bottom up , Start picking clothes （ Is it not good for me to describe it like this ？）

Chop first MAC Head , Check the purpose MAC, If it's with the server's own MAC If the address matches, put it away .
Grill again IP head , Own IP The address also conforms to , Just transfer to TCP.
Grill again TCP head , There's a serial number , If it's what I want , Just go back to one ACK Answer pack , Don't just throw away .TCP The head also has the port number ,HTTP Our server is listening for this port number . Then the server knows HTTP The process is waiting for this package , This is what it wants , therefore TCP Just send this package to HTTP process .
HTTP Take a look , Ow , Originally, this request is to visit a web page , Then we will package this web page in HTTP In the response message , And then through TCP,IP,MAC After packaging , Go home . But this time the source IP And purpose IP It was exchanged with when he came .
When you get dressed, you come to the network card , Then go to the switch and send it to the border port , That's the router , Then the router bounced back home .
When you get home , Be stripped of clothes again , Then give the data to the browser , Browser to render web pages , That's it , Browser input URL The process of is completely over .

summary

A packet , The road to visit relatives is always full of ups and downs , You have to have friends to help you on the way ： Reliable transmission TCP, Remote location IP, The next stop is for the guide MAC wait . Let's start together after we gather , With the help of switches and routers , All the way to the destination . Experience along the way , Let the data package deeply realize the vastness and order of the network world , thus , The story comes to an end , And take over ！！！