On the development trend of enterprise storage: cold thoughts on open source storage

 

Many years ago ,

It has been discussed in the industry

The relationship between open source and self research ,

Which is more important ,

Which is better or worse ,

Up to now, there is no final conclusion .

Open source software is booming , Why do opportunities and challenges coexist ？

in fact , Many of our industries seem to be inseparable from open source software . Why? ？ Open source has created the development of many industries , It has become an important way for many industries to innovate .

Synopsys company 2020 According to the annual research data ,99% Our commercial software contains open source components ,75% Is directly composed of open source code . The use of open source software is increasing , In cloud computing 、 big data 、 Artificial intelligence 、 Industrial Internet and other emerging fields , Open source software has become an important innovative development model .

From the overall contribution of China to global open source software ,2021 year 7 month , Dongdajian, deputy director of the national information security development center, pointed out , China's contribution to open source code has accounted for the world 40% about , Country “ The fourteenth five year plan ” It is also the first time to add relevant development requirements for the construction of open source system . however , He also pointed out that , Open source system also has some systemic problems and risks .

Although open source software has some problems and risks , But people are still interested in open source applications and development , therefore , Both at home and abroad, great importance is attached to the security research of open source . The United States WhiteSource Issued by the company 《2019 Open source component security vulnerability status report 》 Show ,2019 The number of open source software vulnerabilities disclosed in has increased to 6000 Multiple , The increase is close to 50%; Then in its 2020 Published in 《 Open source security annual report 》 Show , The average 1000 There are... In the open source code 14 Security holes , Every time 1400 There are... In the open source code 1 A high-risk security vulnerability .

2021 end of the year , Wangxiaodong, Public Technology Service Department of the state information center, was in 《 Research on the outstanding risks and Countermeasures Faced by China's open source software industry 》 It is pointed out that , China's open source software industry suffers from supply failure 、 code safety 、 intellectual property right 、 Independent innovation and other risks .

Rational understanding of open source storage , What is good and what is bad ？

in fact , Open source led to the birth of storage start-up technology companies , Drive the development of open source ecology related to storage . Manufacturers who build storage products based on open source software , Complementary to the open source community , Promote each other , common development . Open source projects have also attracted capital interest , With the blessing of the capital and open source community , Such storage vendors have also started the commercialization of open source projects .

However , The open source commercialization of enterprise storage vendors is not easy , From the open source development and current situation of Distributed Storage Software , The current mainstream open source software mainly includes Ceph、HDFS、Swift、Lustre、GlusterFS etc. .

among , By storage type ,Ceph Belongs to the fusion block 、 object 、 Unified storage of files , HDFS、Lustre、GlusterFS Belongs to the file storage class ,Swift yes OpenStack One of the subprojects of the open source cloud computing project , Belong to OpenStack Object Storage Object storage .Ceph And GlusterFS Go hand in hand , The development of open source storage in China has been pushed to a new stage , It has contributed to the development of cloud computing and software defined storage . For now ,Ceph There are many supporters of open source storage , Community construction is even larger , Developers contribute more frequently .

From the overall development advantages of open source storage , Because community developers and development teams contribute a lot of code , Their respective functional areas of expertise , Iterations are naturally faster .

meanwhile , Because the code is open source , As long as you are a member of a community organization , Follow certain rules , Code can be used , There must be development cost advantages .

The construction of open source community is also the construction of open source ecology , This is where developers from all over the world contribute . Every community member and the community organization itself are under the joint action of the open source ecology , Continuously promote the application and external delivery of open source storage software . thus it can be seen , The overall ecosystem of open source storage software is still very strong , Industry influence is also deepening .

However , The commercialization challenge of storage software , It can't be solved in a while . Storage manufacturers who adhere to the self-development route , Like dawn 、 Huawei and others have experienced decades of technology accumulation , If open source storage software can be done in a short time , What does that mean ？ It's worth thinking calmly .

From the perspective of user needs , The correct way for the development of distributed storage is how to achieve capacity 、 performance 、 Usability 、 reliability 、 The balance of scalability . For open source storage software ,Scale-out The flexible scalability of storage capacity brought by the architecture , Plus the rapid development of flash memory and large capacity hard disk technology , So that the performance of the storage system can be high enough , At the same time, it also achieves good scalability . However, in terms of availability and reliability , Both need further optimization and improvement .

In terms of stability , The open source community is in an open ecosystem , Not code written by a group of people or a development team , The understanding of software architecture design varies , The development logic may be inconsistent , The risk of unstable factors and potential safety hazards is greater . It is inevitable that the code program and product software have defects , Defects are found and corrected at different stages , The impact on code stability varies , The costs are also different . According to relevant statistics, it is estimated that , In software testing 、 The cost of correcting defects in the release phase is the cost of finding and correcting defects in the coding phase 15-90 times , If the defect is found and solved after delivery to the user , This number will reach 50-200 times . therefore , Any potential safety hazard may cause huge losses . For open source community code , Because of its openness and complexity , As a result, it is difficult for community developers to test the open source storage code completely . Commercial products based on open source storage software , If not to 100% Source code for testing , Then this may bury many hidden dangers for users to deliver stable and reliable storage systems .

In terms of overall service quality , Because open source brings Bug The problem is increasing year by year , How to really solve the problem of Bug problem , The technical capabilities of open source storage vendors are challenged to varying degrees . The manufacturer has strong R & D strength , Naturally, the service quality for users is high , The R & D strength of the manufacturer is weaker , The service quality brought to users exists “ discount ”. so , Every time open source storage software is delivered , All are technical tests for open source storage vendors .

Besides , In large-scale deployment , There are some challenges in the stability and availability of open source storage . With Ceph For example , There are many cases of commercial storage , But use open source directly Ceph Build tens of PB Or even hundreds PB Large scale cases of , So far, there are few .

Of course , What needs to be noted here is , Self developed commercial storage software , Nor does it mean that all code modules are closed . In a huge systematic project , Will inevitably learn from 、 Improve or adopt a few open source frameworks or components , There will also be potential security vulnerabilities due to the introduction of open source components .

The independent research and development of storage software mainly focuses on four aspects ：

One is , Overall architecture design ;

Two is , Construction of core data path ;

The third is , The software design and coding implementation of each key module from the bottom to the top ;

Fourth, , System security, etc , That is, independent research and development means that the manufacturer's functions for the storage system 、 performance 、 Reliability, etc. have full control , Be independent in the integrity of the system , At the same time, it has passed the complete unit test 、 The system test 、 Product testing , Combined with white box test 、 Grey box testing 、 Black box test, etc , Deliver high quality products , Minimize the application risk of users .

Users do storage selection , Is it open source or self-developed ？

Global storage watch analysis suggests that , Open source and self-development are like the front and back of a coin , All have their value of existence . According to different scenarios and application requirements , Select according to the user's own technical development ability , This is the rational solution .

From performance 、 reliability 、 stability 、 From the perspective of backup and disaster recovery , When users are faced with the scheme selection of core business application scenarios , We have always been cautious about open source storage . Especially when the user's own operation and maintenance team is weak , Will face follow-up software maintenance 、 A series of technical challenges such as update iteration . therefore , In the core business 、 Production system and large-scale deployment , The concept that users choose storage systems remains “ conservative ”, Self developed storage system is still its first choice .

On the other hand , For a certain number of storage developers , Or users with development background in the storage software field , It has the ability to modify the code of open source storage software 、 Comprehensive ability of optimization and commissioning , Even team members may be contributors to some open source communities . under these circumstances , With storage open source software , It is convenient to cooperate with the development trend of business innovation and application , Faster access 、 Flexible application deployment . thus it can be seen , If the storage software development ability of enterprise users is strong , Choosing open source innovation is a good path in line with industrial development .

For users , Whether the storage is open source or self-developed , It is necessary to consider the comprehensive cost of the enterprise itself , Including data security requirements after application deployment , Storage software development and operation and maintenance personnel costs , Self storage service quality for solving problems in the actual application process . Of course , Data security 、 Operation and maintenance cost and service quality , For any open source storage vendor , It is also three major issues worthy of attention .

From the perspective of storage application , Open source and self research have become two major trends in the development of enterprise storage , Like the left hand and the right hand . Enterprise users will use their left hand more , Use your right hand more , Only lies in the enterprise's own technical capabilities and practical application needs . Sometimes left and right hands go together , It's not impossible . Anything is possible , Of course, the greatest possibility lies in the possibility of our own technology development ability .

about “ Open source storage + Self developed storage ” Integrated development model , How many enterprise users will really open , We can also wait and see .

（by Aming）

- END-

you

how

Well

see

？

The comments at the end of the article are welcome to add ！

【 Global storage watch ｜ Technology clearly says 】 Focus on technology company analysis , Speak with data , Take you to understand technology . This article and the author's reply only represent personal opinions , Does not constitute any investment proposal .