After one year, the paper was finally accepted by the international summit

author | godweiyang

- BEGIN -

Just yesterday , International Summit on supercomputing SC22 Put it on the list , The papers of our group have also been accepted , score 44332（4 " ）：

Address of thesis ：
​​ ​https://arxiv.org/abs/2110.05722​​

SC What is a meeting ？

Maybe many students haven't heard of it SC What meeting is it ,SC The full name of the meeting “ High performance computing 、 The Internet 、 International Conference on storage and analysis ”（The International Conference for High Performance Computing, Networking, Storage, and Analysis）, It's high-performance computing , Top level conference in architecture field ,CCF recommend A Class meeting .SC It is difficult to deliver the meeting , Hired every year 70 About papers , The employment rate is 25% about .2022 Contribution in 320 piece , Employment 75 piece （6 Article under revision ）, The employment rate is about 23%.SC22 Will be in 11 month 13-18 Held in Dallas, USA on the th .

What did the paper say ？

The title of this paper is ：“LightSeq2: Accelerated Training for Transformer-based Models on GPUs”, This is a LightSeq The second issue of training engine related papers , In depth explanation of LightSeq Operator fusion used in the training engine 、 Video memory management and other related technologies .LightSeq Support Transformer、BERT、GPT、ViT And so on , It can speed up the task of machinetranslation 3 times . See the following article for a detailed introduction to the technical principle ：
​ Support Transformer Speed up the whole process of training , Maximum acceleration 3 times ！ Bytes to beat LightSeq Up to date

How is the submission process ？

last year 6 In the month , The first draft of this paper has already taken shape . At that time, we invested in the second half of the year PPoPP International Summit , But the score is very low , Almost all of them refuse . Summarize the reasons , One reason is that the conference and the direction of the paper don't particularly hit it off , This conference pays more attention to the achievements in the direction of distributed parallelism ; Second, the thesis is based on NLP Domain writing habits , The writing is really bad , So it is considered that there is no innovation .

After being rejected, I invited UCSB Several teachers and students have made professional changes , Reorganized the structure of the paper . The same technique is expressed in more advanced scripts , The paper suddenly got up .SC22 The process of submitting papers is very long , from 3 The month continued until 6 The results only come out in January . And the final draft will last until 8 month , The meeting was even more 11 The month begins , The whole process lasted more than half a year .

You can also travel at public expense ？

Causes of the epidemic , There is no way to go to Dallas ,UCSB A few teachers and students of can help to go to the scene to observe . Remember... Two years ago ACL20 I missed the opportunity to travel to Seattle at public expense , I missed Dallas again this year . Meetings can be missed , I hope not .

What does our group do ？

Our group is the ByteDance volcano translation team , That is, of course, machinetranslation . And I do model optimization ,LightSeq Is our main project ：
​​ ​https://github.com/bytedance/lightseq​​

I also wrote a detailed introduction of our group before ：
​​ ​ It's been almost two years since I came here , Answer some questions that interest you ​​

How to enter bytes ？

Push links inside （ Or click to read the original ）：
​​ ​https://job.toutiao.com/s/YB9MUpS​​

- END -

I am a godweiyang, Bytes to beat Algorithm engineer , sink 985 Computer Ben Shuo Major first . Qiu Zhao captured three big factories SSP offer, Good at Algorithm 、 Machine translation and Model acceleration .