In the era of video explosion, who is supporting the high-speed operation of video ecological network?

In recent years , With the rapid development of Internet technology and video technology , The Internet has gradually entered the era of big video . Video services and traffic are growing rapidly , Regardless of the diversification of video business , Or towards Ultra HD , In fact, all these will bring more severe challenges to the carrying capacity of broadband networks of operators .

at present , The domestic video business shows the characteristics of diversification .OTT video 、 Internet homemade drama 、 Live entertainment 、 Show live 、UGC、 A hundred flowers bloom in short video and other businesses . youku 、 Iqiyi and Tencent video have different business advantages , The market share is increasing year by year , At present, it has ranked among the top three peers in China .

After Youku and Tudou officially merged , Have occupied UGC The leading position of the model . In addition to the exclusive copyright of Tencent video , Its social networks are increasingly supporting video traffic . Iqiyi has achieved remarkable results in paying members . Mango. TV Back to Hunan Satellite TV , The momentum is strong . Letv 、 Sohu videos 、PPTV、 Storm video and audio gradually form their own style characteristics . To make a long story short , The domestic online video business is in the era of wind and water .

All kinds of Ultra HD video terminals have entered thousands of households , All kinds of set-top boxes have been supported 4K（3840×2160） The resolution of the , On the market 4K Standard TV has become the mainstream ,8K Television is gradually becoming commercially available on a large scale . In the current market , The whole large video ecological chain in China has been very complete , Whether it's video content 、 application 、 Processing terminal 、 Display terminal 、 The market scale is mature , Constitute an industrial ecosystem with sound development .

Under the situation of fierce market competition , Video content service providers are eager to improve their perception of video service quality , Demand for creating high-quality video brands . To ease the pressure of Internet traffic growth , The three operators are constantly expanding capacity . At present, Chinatelecom has been in the backbone 163 Net implementation No 19 Second expansion , After the expansion, the national inter provincial bandwidth will be 100Tbit/s above . The age of video explosion , Who is supporting the promotion and operation of the video ecological chain ？

In the early days, we used traditional traffic protocols SNMP Traffic monitoring protocol . But because of SNMP The data collection of is to collect data traffic directly through the network device port , Thus, it is impossible to provide specific point-to-point session connection information , Therefore, the statistical method for the specific flow direction of network data traffic is more complex and inaccurate . be based on SNMP The system performance bottleneck exists in the traffic monitoring and analysis system ,SNMP It is difficult to deploy in a large network .

Nowadays, video traffic has exploded , In order to overcome the technical limitations of the existing network management system on the network traffic and flow analysis function . A mature and stable new technology is needed to transform and upgrade the collection and analysis methods of management information in the existing management system . New information collection and analysis technologies also need to have little impact on the operating network of operators , It can be upgraded smoothly without changing the network topology . And the new information collection and analysis technology should be able to count the bandwidth utilization of each link in the network , It can also analyze and count the traffic and flow direction of different types of services transmitted on each link . At the same time, it should be able to adapt to the rapid expansion of China's broadband Internet network , The actual situation of rapid growth of link bandwidth . The sampling mode of data collection or the mode of automatic aggregation of collected original data can be supported as required , Reduce and compress the amount of data to be processed by the management server of the network management center , Ensure that the processing capacity of the management system can adapt to the growth of network scale .

With NetFlow As a representative of the Flow Technology is a new solution to this challenge .

What is? Flow？

At the very beginning Flow It is a technical concept introduced by the network equipment manufacturer to improve the routing forwarding speed inside the network element equipment . Its original intention is to raise the CPU The consumed routing table software query matching job is transferred to the fast forwarding module implemented by the hardware （ Such as Cisco Of CEF Pattern ）. In this functional mode , Packets will be merged into a specific set through several given feature definitions , This collection is Flow Every Flow The first packet . In addition to promoting the Flow In addition to the generation of records, the layer 3 module of the ne must be driven to complete the route query and the query results . In the same period Flow In the record Flow The subsequent packets of the collection will be directly in Flow Get route forwarding information from the existing records of , Thus, the routing and forwarding efficiency of the network element device is improved .

As a by-product of the optimization of the internal routing mechanism of the network element device ,Flow Records can provide traditional SNMP MIB Incomparable wealth of information , therefore Flow Data is widely used in high-end network traffic measurement technology to provide network monitoring traffic schema analysis 、 Application business positioning 、 Network planning 、 Quick troubleshooting 、 Safety analysis such as （DDoS)、 Data mining functions such as inter domain bookkeeping .

Relative to session （ Session ） for ,Flow With more detailed identification features in the traditional TCP/IP5 On the basis of tuples, some new domain values are added, including at least the following fields 1. Source IP Address 2. Purpose IP Address 3. Source port 4. Destination port 5. IP Layer protocol type 6 .ToS Service type 7. Enter the physical port . above 7 Fields can uniquely determine which particular packet belongs to Flow, In other words, a difference in any field means a new one Flow Happen .

NetFlow Traffic analysis protocol

NetFlow yes IP Network specific stream switching protocol , At present, it is widely used in the statistical analysis of network traffic .NetFlow Adopt three-tier architecture , Including data output equipment （NDE）、 Data collectors （NFC） And data analyzer （NDA）, Respectively used for stream data collection 、 Converge 、 Output 、 receive 、 Filter 、 Storage 、 analysis .

NetFlow A one-way packet collection from a specific source host to a destination host is defined in , There are mainly the following 7 Parameter identification ： Source IP Address 、 Purpose IP Address 、 Protocol type 、 Source transport layer port 、 Destination transport layer port 、 Business types TOS And input interface of equipment . utilize NetFlow The flow direction and proportion can be counted , contain 5W1H Etc , Can solve Who（ Source IP Address ）,When,Where（ Purpose IP Address port ）,What（ Protocol type ）,Why（ Normal threshold baseline ）,How（ situation 、 Traffic size ） Other questions .

NetFlow It consists of two core components ：NetFlow Cache and NetFlow Data transmission mechanism of .NetFlow The cache is used to store the data collected from the input interface IP Data flow information ;NetFlow The data transmission mechanism of is mainly based on the predefined policy rules , Send data to the network management collector , Facilitate data acquisition .

NetFlow The traffic analysis system group is connected to the provincial core router NetFlow The traffic analysis system can perform statistical analysis on the flow direction of network traffic . in summary , utilize NetFlow The protocol can statistically analyze the size of the traffic accessing the destination in a certain period of time , That is, the traffic of video content service providers can be statistically analyzed .

How to integrate NetFlow Maximize the effectiveness of the agreement ？

  First enable on the core device NetFlow agreement , adopt NetFlow The collection server collects 、 analysis 、 Find out the user access traffic ranking in the current region TOP Of IP Address . According to IP Address , use IP Application scenarios 、DNS Traverse 、 Online tools 、 Search engine 、 Get the information of the real video content service provider by means of detection . Finally, the system is used to obtain the information of the video content service provider with the largest number of user visits under the current network .

With Internet The rapid development of , It's not just the video industry , The scale of the Internet is expanding and the environment is becoming more and more complex . A good IP Statistical methods for flow and flow analysis , It helps operators know the load status of their own network and the bandwidth occupancy of important services in time , Backbone network construction 、 Capacity expansion 、 upgrade , Improve broadband speed 、 Flow direction analysis and statistics have a great role .