Green data center: comprehensive analysis of air-cooled GPU server and water-cooled GPU server

Introduction

With the Internet 、 big data 、 Cloud computing 、 The rapid development of high-performance computing , The data center is facing severe challenges . It mainly comes down to GPU Drive applications to drive , Make business and IT Growth needs to be coordinated . As the temperature inside the data center rises , Operators are scrambling to find feasible 、 More energy-saving cooling solutions . Here we will discuss the most feasible and practical cooling method , And compare and analyze air cooling and liquid cooling .

1  Operation principle of cooling technology

Air cooling

The traditional air-cooled data center brings the external air through the air inlet on the air processor . The air is conditioned by the computer room （CRAC） Unit cooling , Then it enters the server rack driven by the elevated floor " Cold aisle ". Cold air passes through and cools the server , Then leave " Hot channel ", Discharge through the inflation chamber and return to the air processor .

Liquid cooling

The working principle of liquid cooling is ： Liquid is a better thermal conductor than air .

• The server is installed vertically on horizontally oriented media （ Electrically non-conductive ） In the liquid cooling tank .

• The coolant transfers heat through direct contact with the server components .

• The heated coolant flows out of the top of the frame , And in the rack and the cooling distribution unit connected to the warm water circuit （CDU） In between . The other side contains a cooling tower or dry cooler as the final form of heat dissipation .

• The cooled liquid returns from the heat exchanger to the rack .

2  Complexity and upfront costs

Air cooling

The seemingly simple server rack system actually has many moving parts ： Elevated floor , Channel containment strategy , Cooler , Air processor , Humidity control , Some combination of filtration system and plenum . In order to support the above functions , The air-cooled data center must also operate a relatively large auxiliary infrastructure （ Especially the standby generator 、UPS And batteries ）.

These necessary complexities equate to relatively large capital expenditures （CAPEX）.

 

Liquid cooling

since 2009 Since the invention of this technology in , Has been exploring the maximum value of liquid cooling （ simplicity ）.

Consider only three moving parts ： Coolant pump , Water pump and cooling tower , Dry cooling fan , And there is no need for elevated floors , There is no need to close the space through the channel . Liquid cooling can reduce the capital expenditure of the data center 50% Even lower .

Because the racks can be closely spaced , Liquid cooling does not require air flow CFD analysis , It can even be placed on the exposed concrete floor . The electrical support system is reduced in size .

Before assuming that simplicity will affect performance , Add a little explanation GRC ICEraQ The system can be cooled easily 100 kW / frame （ Even higher ）, Far more than the best air-cooled operation ability .

3  Efficiency and operating costs

Air cooling

It is obvious that air is a more effective thermal conductor than liquid （ Low efficiency 1200 times ） . It not only reduces the efficiency of air-cooled data center in essence , And there will be a chain reaction , Have a serious impact on operating expenses .

Fans account for 20%. To improve air efficiency , Cooling components that consume energy such as coolers and air processors . These in turn will affect the scale of power infrastructure .

In view of the above , Air cooling requires extremely high operating costs in data center technology , At the same time, about 1.35 to 1.69 Of PUE.

Liquid cooling

Use three moving parts （GRC Remove fans to optimize server immersion ）, Refrigeration components , Greatly reduce infrastructure requirements , Liquid cooling reduces cooling energy 90%, The total energy consumption of the data center is reduced 50%. therefore , Can achieve <1.03 Of PUE.

From the perspective of electricity , Air cooling infrastructure not only increases costs , And it may also be accompanied by high maintenance costs . Liquid cooling does not have these problems .

4  Cooling capacity and high density performance

Air cooling

exactly , Some air-cooled data centers can be cooled 30-35 kW frame . But actually , The air-cooled data center is 15 kW It is very inefficient above the rack .

Industry trends make the situation more serious . Power consumption GPU Marching , To cope with the Internet of things and AI etc. HPC application . Use an example to illustrate , Intel's new Skylake Version consumption is as high as 250 W Energy . Put two of them on 1U Server , Add... For other electronic devices 200 W above , multiply 40 The server , be based on CPU Our system has 28 kW. Add processors and accelerators , It will far exceed the limit of air cooling .

To meet the above needs , Data center operators tend to create racks with mixed density . In terms of air cooling , Inevitably, it will lead to hot spots , This leads to hardware failure .

It is worth mentioning that , Hardware evolution will be the operator of air-cooled Data Center （ Next hardware update ） Bring real liquidation time .

Liquid cooling

GRC ICEraQ  and ICEtank  The system breaks through the thermal barrier through careful design , Bring data center computing to the next evolution , Even further . Any solution can easily cool each rack up to 100 kW Power , Theoretically, when used with coolant, it can cool up to 200 kW Power .

4  Reliability and location flexibility

Air cooling

Any air-cooled cooling technology is doomed to face the problem of hardware reliability . The reason lies in IT Assets are exposed to potentially harmful air pollutants and the adverse effects of the air itself （ Mainly refers to corrosion and oxidation ）.

The risk depends on the air quality and the natural humidity level of the unadjusted air itself . obviously , High humidity 、 Places with air pollution or wind blown particles may cause serious damage to the data center . As the demand for remote edge deployment increases , These problems will also be highlighted .

Speaking of location flexibility , The inherent complexity of air-cooled cooling and huge infrastructure requirements pose major obstacles to the placement of data centers .

in summary , Even if the best channel layout method is adopted , Traditional air-cooled cooling will also produce hot spots that lead to hardware failures .

Liquid cooling

Three main factors make liquid cooling the highest score in this category ：

• The simplest practical cooling form on the market . Fewer mistakes ： No cooler , Air processor , Humidity control, etc ; And there is no server fan to generate, which can increase MTBF（ Mean time between failures ） The vibration of .

• Immersive processing , send IT The asset is completely isolated from the outside air , So as to eliminate environmental problems .

• There are no hotspots in the data center . in fact ,ICEraQ or ICEtank Any two points in the system work within two temperature ranges .

5  Blue ocean brain water cooling GPU The server

Water cooling system based on blue ocean brain super fusion platform GPU Server and water cooling system GPU Workstation adopts Gluster Distributed architecture design provides large storage capacity , With the characteristics of open fusion and the ability of Super Computing . Support multiple blocks GPU The graphics card , Provide excellent mute effect and perfect temperature control system . With the help of NVIDIA GPU The calculation card can speed up the training and reasoning speed of neural network , Create accurate lighting and rendering effects faster , Provide high-speed video 、 Images 、 Sound and other processing capabilities , Speed up AI And bring a smoother interactive experience .

6  summary

in summary , With the increasing size of the data center , More and more racks are deployed , The requirements for server load are more stringent , The traditional air-cooled cooling technology no longer meets the cooling requirements of the data center . To solve this problem , Many enterprises begin to adopt liquid cooling technology in their data centers .