Before we learn about high-performance computing, let's take a look at its history

This article is shared from Huawei cloud community 《

High performance computing （1）—— Reading history to know the rise and fall

》, author ：  I'm a big watermelon  .

The rise of parallelism

stay 2005 Years ago , Most processors are single core , Some processors have started  

Support vectorization

( Such as X86 Processor supported MMX( Multimedia extension ) and SSE( streaming SIMD Expand ) Instruction set ), Processor manufacturer via

Increase the frequency of single core scalar processors

and

Instruction level parallel processing capability

( That is, improve the performance of the instruction pipeline ) To improve the computing performance of the processor .

stay 2005 Years ago , The performance of a single core scalar processor is basically per 18 Nearly doubled in months , This is called

Moore's law

, As shown in the figure below . The period when the performance improvement of a single core scalar processor can meet Moore's law is called the period of improving software performance “ A free lunch ” period , Because the performance of single core scalar code can meet the speed increase described by Moore's law .

stay 2005 Years later, , The performance of single core scalar processors has basically reached its peak , It's hard to go any further ( exceed 10%) Lifting performance —— Why? ？

Single core scalar processors improve performance by approximating Moore's law   can , It mainly improves performance in the following ways ：

• Increase the clock frequency of the processor
  ： The clock frequency of the processor indicates that the processor 1 How many basic operations can be run in seconds . Power consumption limits the continuous improvement of frequency ： From the point of view of physical Theorem , With the advancement of processor process , Maximum power consumption of the processor ( Mainly leakage power consumption ) More and more big ( The power consumption of the processor is approximately proportional to the third power of the processor frequency ),THINK:  cool 、 Superconducting computer 、 Quantum computers

• Improve instruction level parallelism
  ： There are many different components on a single core scalar processor , Each component performs different command operations . To make good use of the processor's instruction level parallelism, code optimizers are required 、 Compiler authors and processor designers work together . Processor design usually increases the level of hardware pipeline , Now this method has reached its limits . Hardware designers improve performance by increasing the length of hardware registers . for example , The original register length is 32 position , Now upgrade to 128 position ,THINK:  Increase the number of registers ？ Why the mainstream is 32 Bit and 64 position ？
  https://www.zhihu.com/question/21641577

therefore , Due to heat dissipation, the frequency of the processor cannot be increased , Hardware manufacturers have switched to using multiple processors on a single chip , This is called

Multicore

; To provide higher performance processors , Processor hardware manufacturers process multiple data simultaneously by increasing the width of registers and instructions , This is called

To quantify

.

Difficulties of multi-core and vectorization ：

• basis Amdahl The laws of , The proportion of serial code in the program limits the best results that parallelized code can achieve .

• Multi core vector processor ( In especial X86) In order to reduce the delay in obtaining data ,  A large number of caches are used to store repeatedly accessed data . But caching does not contribute to the native computing power of the hardware .

• Some code cannot use multi-core parallelization or Vectorization .

• We should give full play to the computing power of vectorization and multi-core , Multiple copies of code may be required , This increases the cost of code maintenance .

• In some strict application scenarios , It is limited that it cannot allow the deviation of calculation results caused by vectorization and multithreading .

The rise of heterogeneous parallelism

The rise of heterogeneous parallel era is from 2007 year NVIDIA Introduction CUDA(Computing Unified Device Architecture, Computing Unified Device Architecture ) At the beginning . Heterogeneous parallelism contains two sub concepts ：

• isomerism (Homogenerous)
  . Heterogeneous means that heterogeneous parallel computing needs to process multiple computing platforms with different architectures at the same time   The problem of , Such as the current mainstream heterogeneous parallel computing platform X86+GPU、X86+FPGA, And currently under development ARM/Power+GPU

• parallel (Parallel)
  . Parallel means that heterogeneous parallel computing mainly adopts parallel programming , Whether it's X86 It's about   Manager , still ARM and GPU Processor and DSP, All the processors here are multi-core vector processors

At present CPU Has fallen behind Moore's law , but GPU It is still developing rapidly .CPU and GPU They are all chips with computing power .  among ,CPU Not only good at command operation , And good at all kinds of numerical operations ; and GPU It's a chip specially designed for processing graphics tasks , Only good at numerical calculation of graphic functions . In terms of hardware design ,CPU It consists of several cores optimized for sequential serial processing . On the other hand , GPU By thousands of smaller 、 More efficient core components , These cores are designed for multitasking at the same time .

All in all ,GPU It is designed for the numerical calculation of matrix graphic functions . It uses a large number of repeated design operation units to establish a large number of numerical operation threads , He is good at highly parallel numerical calculation of a large number of parallel data without logical relationship .  and CPU Is based on both  

“ Instructions are executed in parallel ”

  and  

“ Data parallel operation ”

  Design according to the idea of the design , Good at handling scheduling with complex instructions 、 loop 、 Branch 、 The program tasks of logical judgment and execution, etc .

Huawei partners and developers conference 2022 The fire is coming , Heavy content can't be missed ！

【 Wonderful activities 】

March forward courageously · Be an all-around Developer →12 Technology live broadcast ,8 High energy output of the great technical treasure , And the code room 、 Many rounds of mysterious tasks such as knowledge competition are waiting for you to challenge . Break through immediately , Open the ultimate prize ！

Click to embark on the promotion of all-round developers ！

【 Technical topics 】

The future has to ,2022 Technical exploration → Huawei's cutting-edge technologies in various fields 、 Heavy open source project 、 Innovative application practice , Standing at the entrance of the intelligent world , Explore how the future shines into reality ,

Full of dry goods, click to learn

Click to follow , The first time to learn about Huawei's new cloud technology ~