AI should take code agriculture? Deepmind offers a programming version of "Alpha dog" alphacode that surpasses nearly half of programmers!

On the occasion of celebrating the Spring Festival of the year of the tiger in China , As Google parent company Alphabet Its AI enterprise ——DeepMind, Released based on Transformer Model programming AI：AlphaCode, And claimed that its computer program is competitive , Can be comparable to ordinary human programmers .

This is the OpenAI Of Codex and GitHub Of Copilot after , It has further pushed the process of yards away from unemployment ！

see , stay AlphaCode At the same time of release , Acquired by Microsoft OpenAI by Lean The neural theorem prover created has successfully solved two International Mathematical Olympiad problems . People can't help feeling ：AI After joining , Let this world volume, which is originally an internal volume, be added to the volume ！

although AlphaCode The release of is quite a sensation in the programmer world , but DeepMind Who gave it the confidence to compete with human programmers ？

stay 2 month 2 In a paper published on August ,DeepMind The researchers introduced AlphaCode.

AlphaCode Using a Transformer A language model for large-scale code generation , Solutions need to be combined with critical thinking 、 Logic 、 Algorithm 、 New problems in coding and natural language understanding , And write it as a program .

Thesis link ：

https://storage.googleapis.com/deepmind-media/AlphaCode/competition_level_code_generation_with_alphacode.pdf

The researchers will AlphaCode Put it in Codeforces The challenge was tested .

Codeforces Is a competitive programming platform , By Russian programmers Mikhail Mirzayanov founded , Its Elo The scoring system is similar to the rating system used in chess , It can measure the programming level of a programmer .

Codeforces founder ：Mikhail Mirzayanov

Different from the tasks that programmers may face when building business applications ,Codeforces The challenge of being more independent , Need to have a broader understanding of algorithms and theoretical concepts in Computer Science , Generally, it is combined logic 、 Very professional puzzles for math and coding expertise .Codeforces Last week, I will share the ranking of programming challenges and problems .

AlphaCode in the light of Codeforces On the website 5000 Resolved by users 10 An existing competition was tested , Overall ranking in the top 54.3%, In other words, it defeated 46% Of competitors .

DeepMind It is estimated that ,AlphaCode Systematic Codeforces Elo by 1238,CodeContests Every problem on the has 100 Ten thousand samples ,AlphaCode It's solved 34.2% The problem of , It has only been brushed in the past six months 10 Zhou's topic , It ranks top among the users competing on the website 28%, The results are brilliant .

founder Mike Mirzayanov stay DeepMind A shared statement said ：

I can safely say ,AlphaCode The results exceeded my expectations . I was skeptical before , Because even on simple competitive issues , Not only to implement the algorithm , And invent algorithms , And that's the hardest part .AlphaCode Our performance is comparable to that of a promising new competitor .

therefore ,DeepMind Express , Even though AlphaCode The level of does not necessarily represent the programming tasks faced by ordinary programmers , But it shows that autonomous programming has taken an important step forward .

1

AlphaCode The programming process of

Here's an example , have a look AlphaCode How to be “ Programming problem maker ” Of .

Codeforces - 1553D

link ：

https://codeforces.com/problemset/problem/1553/D

subject ：

There are two strings s and t, They are all composed of lowercase letters . For strings s, We scan the whole string backwards . If you press Backspace key , Delete all characters from the cursor to the previous undeleted character .

give an example ：

Like strings s yes "abcbd", You press in the first position and the fourth position respectively Backspace, Then you will get the string "bd".

Because there is no character before the first cursor position , So the first time there is no action . The character before the fourth cursor position is c, The last undeleted character is a,a and c The characters between are b, So press Backspace The first three characters will be deleted "abc", Get the characters "bd".

problem ：

Can we scan from front to back s character string , Give Way s Turn into t. If it can output YES, Otherwise output NO.

AlphaCode Programming process ：

AlphaCode Not only successfully solved the problem , It can also display the corresponding position of code and attention highlighting .

AlphaCode The complete code given ：

Ten of these challenges are entered in exactly the same format as humans AlphaCode, then AI Generate a large number of possible answers , And like human competitors, by running code and checking output , Filter out these answers .

What's better is ,AlphaCode Co director of the paper Yujia Li and David Choi Express ：

The whole process is automatic , No manual intervention is needed to select the best sample .

If you want to be in Codeforces Stand out from the challenges of , It was not an easy thing .AlphaCode The project was launched more than two years ago , With large scale Transformer Advances in models and large-scale sampling 、 The combination of filtering technology ,DeepMind Of researchers have been AI Significant progress has been made in the number of problems that can be solved .

Affected by the epidemic , Most of the work of this project is done at home .

2

AlphaCode The principle of programming

DeepMind Express , stay Codeforces The ability to solve problems in the competition , It has exceeded the existing AI The ability of the system .AlphaCode contain 414 One hundred million parameters , It's about Codex Of 4 times , The training data set is 715.1 GB.

Whole AlphaCode The process of the model is as follows ：

1. Modeling goals in standard languages , stay GitHub Code pre training is based on Transformer The language model of . This model can reasonably represent the space of human coding , It greatly reduces the search space for problems .
2. Fine tune the model on competitive programming datasets , Use GOLD And tempering As a training goal , Further reduce the search space , And use pre training to make up for a small amount of competitive programming data .
3. Generate a large number of... From the model for each problem C++ and Python Program sample .
4. Filter the sample , To obtain a small number of samples of candidate submissions （ most 10 individual ）, Evaluate on hidden test cases , The method is to use examples to test and screen 、 Clustering and reordering , Select samples according to program behavior .

To make a long story short , Through the large-scale Transformer The model is combined with large-scale sampling and filtering ,DeepMind Significant progress has been made in the number of problems that can be solved , It is an order of magnitude higher than the previous work .

3

AlphaCode Research significance of

On the whole ,AlphaCode The ranking of is roughly equivalent to the median among competitors . Although far from winning the game , But this result represents AI A substantial leap in problem-solving ability . This progress demonstrates the potential of the deep learning model in tasks that require critical thinking .

DeepMind Pointed out that ,AlphaCode The current skill set is only applicable to the competitive programming field , But its ability opens a new door for tools to create the future , These tools will make programming easier , And one day it can be fully automated .

Many other companies are developing similar applications . for example , Microsoft and AI labs OpenAI Have already put OpenAI Language generator GPT-3, Transform into a program that automatically completes the code string .

GPT-3 and AlphaCode equally , Is based on Transforme Artificial intelligence architecture of language model , It is especially good at parsing sequential text , Including natural language and code .

For end users , These systems work like Gmail Of Smart Compose Function as , No matter what you're writing , It can guide you .

DeepMind The chief research scientist of Oriol Vinyals To TheVerge In an e-mail of ：

The research is still in its early stages , But the research results at this stage have made the company closer to building an AI that can flexibly solve problems —— A program that can automatically meet coding challenges . In the long run , We are right. AlphaCode In helping programmers and non programmers write code 、 Excited about the potential to increase productivity or create new ways to make software .

4

AI Programming system ： strong , But not completely strong

In recent years ,AI The development of programming system has made great progress , But these systems are far from completely replacing the work of human programmers . The code they generate often has problems , And because the system is usually trained on the common code base , So sometimes we copy copyrighted materials , The consequences of infringement .

So , Machine programming is by no means a problem-solving Science ,DeepMind admit AlphaCode There are some limitations .

for example ,AlphaCode Not always generate grammatically correct code for every language , Especially in C++ in . In dynamic programming ,AlphaCode The performance is also worse .

Inevitable ,AlphaCode There may be problems in other ways .

although DeepMind There was no investigation into whether the model was biased , But it includes Codex Code generation models, including, have been shown to amplify toxic and defective content in training data sets .

for example , When the input 「 Islam 」 When the word , Will prompt Codex Output 「 The terrorists 」, And generate seemingly correct , It's the code that poses a security risk .

In a project about GitHub Developed Copilot AI In the research of programming tools , The researchers found that the output code was about 40% Contains security vulnerabilities . Security analysts even suggest , Those who have evil intentions can consciously write code , And with the hidden back door （backdoor） Share code online , Then the code may be used to train AI Program , Insert these errors into future programs .

As recent research has explored , image AlphaCode Such systems can also be abused .

Whether malicious people will use these similar systems to automatically generate large-scale malware in the future , This is an open question .

These challenges mean AI Programming systems may have to slowly integrate into the work of programmers —— let me put it another way , They have to go through an apprenticeship , Start as an assistant , Before being trusted to perform work autonomously ,AI Any advice given is subject to doubt . For now ,AI Learn quickly , Should be able to get through the apprenticeship soon .

To help others in DeepMind On the basis of the results ,DeepMind Already in GitHub Published a data set of contest level programming problems and solutions on , This includes a wide range of test data , To ensure that the procedures that pass these tests are correct —— This is a key feature that current datasets lack .DeepMind I hope this benchmark can bring further innovation in problem solving and code generation .

GitHub Project address ：

https://github.com/deepmind/code_contests

5

Views of programmers at home and abroad

just as Codeforces What the founders said , In programming problems, the invention of algorithms is the most difficult .

Google engineers who rank among the top in the global programming competition website for many years Peter Mitrichev say ：

It is very difficult to solve the problem of programming competition . It requires both good code skills , It also requires human creativity to solve problems .

AlphaCode Not the first programming tool ,Codex as well as Copilot All left a deep impression .

but DeepMind Think ,AlphaCode It is quite different from the predecessors ：

Recent large-scale language models demonstrate the amazing ability to generate code , Now you can complete simple programming tasks . However , When it's more complicated 、 When evaluating invisible problems , The performance of these models is still poor , These problems require problem-solving skills , Instead of just translating instructions into code .

A foreign programmer said ：

since AI Can see so fine , It would be better if you could add comments to the end of the code .

Some researchers say , It's like AlphaStar The result of working hard for miracles ：

The voice recognized by domestic programmers ：

Writing code , You should leave it to the code to write it yourself .

Yes, of course , There are also programmers at home and abroad who are interested in AlphaCode Wait for a group AI Programming system resistance .

This AI What a rookie .

because AlphaCode Only 1238 branch , Only equivalent to a student level , A middle school student participating in the Olympiad of informatics can also brush to this level .

Now even the world of question brushing has been AI occupy , Originally, this is where programmers compete ,AI It should be enough , Leave a pure land for programmers ！

One computer can program other computers ？ Can you realize what this means ？

The programmer killed himself .

AI is beginning to wake up .

Call the terminator ！

Yes AI Research scientists sent a series of long tweets saying ,AlphaCode It will take a few years to reach the human level , It's in Codeforces There are limits to the ranking on , Many participants are high school students or college students ; And that is AlphaCode Most of the generated programs are wrong , It is the use of sample tests for filtering that makes AlphaCode Some problems were actually solved .

about AI Programming , What do you think ？

Reference material

[1] DeepMind Release AlphaCode, from https://deepmind.com/blog/article/Competitive-programming-with-AlphaCode

[2] https://alphacode.deepmind.com/

[3] DeepMind Research AlphaCode The paper , from https://storage.googleapis.com/deepmind-media/AlphaCode/competition_level_code_generation_with_alphacode.pdf

[4] https://github.com/deepmind/code_contests

[5] https://codeforces.com/problemset/problem/1553/D

[6] https://www.163.com/dy/article/GV9K92IN0511ABV6.html

[7] AlphaCode Make a world-renowned debut , from https://baijiahao.baidu.com/s?id=1723716497603540996

[8] https://m.thepaper.cn/baijiahao_16567884