What is the experience of pairing with AI? Pilot vs alphacode, Codex, gpt-3

Deepmind Of AlphaCode By first 54% Testing among human coders has made headlines .GitHub Of Copilot Can you keep up AlphaCode Automatic programming of ？

A study conducted at Cambridge University shows that , Developers spend most of their time debugging . This time-consuming task costs the software industry about per year 3000 Billion dollars .Deepmind The latest AI based code development and analysis tools reduce such costs by automating the day-to-day and time-consuming tasks of developers .

With the suggested code GitHub Copilot comparison ,AlphaCode Be able to analyze algorithms and generate competitive and complex programs , Not only are there no mistakes , And corresponding to its description .

DeepMind Of developers tested on competitive programming sites AlphaCode The potential to test it , On these sites , Human developers encounter programming problems and rank them according to their results .

1. AlphaCode—— Independent programmer

AlphaCode It's based on Transformer The language model of , from 414 A hundred million parameters . It's a language model , Its size is GitHub Copilot The language model of Codex Four times the size of , Can only parse 120 One hundred million parameters .AlphaCode The architecture of is based on three parts ：

1. data —— AI tools are provided by public GitHub The repository provides data .
2. Study —— Then the tool trains the data set , And calibrate it according to the task requirements （ for example ,Codeforces Competitive programming ）.
   3. Sampling and evaluation —— ad locum , AI tools conduct large-scale sampling of program changes for each problem . Then through the filtering and clustering process , Arrange the programs into 10 A subset of solutions , Submit to external evaluation .

chart ：AlphaCode Work flow chart source ：deepmind.com

AlphaCode Of AI The system uses a variety of programming languages for pre training , Include C++、C#、Go、Java、JavaScript、Lua、PHP、TypeScript、Ruby、Scala、Rust and Python. The dataset contains about 715GB Code and its description .

2. AlphaCode Stand the test

AI tools participated in Codeforces Competitive coding competition on , This is a popular platform for holding coding competitions . The platform shares questions every week , And with the help of an algorithm to rank the participants , The algorithm is similar to that used to rank players Elo The rating system is similar .AlphaCode I chose 10 There are different testing problems from different development stages . Artificial intelligence tools estimate the top rank among the participants in the competition 54%, To show AlphaCode Our code generation system has achieved success in the competitive level .AlphaCode The ability to generate code is given below Codeforces It is proved in the example of one of the problems ：

chart ： AlphaCode face The problem is , Find out the possibility of converting one phrase to another by pressing the backspace key instead of writing .

chart ：AlphaCode Generated after reading the problem logic and generating the expected code Solution .

Codeforces The founder of Mike Mirzayanov Expressed his surprise , He said ：“ I'm skeptical , Because even in simple competition problems , Usually not only the algorithm needs to be implemented , and （ This is the most difficult part ） Invent it . AlphaCode Successfully reached the level of a promising new competitor .” Mike Add further ：“ I can say for sure AlphaCode The results exceeded my expectations .”

4. GPT-3 framework

It's actually Transformer Of Decoder
GPT-3 It was used 1750 100 million parameter training

4. GitHub Copilot Architecture and performance

GitHub Copilot Is in GPT-3 Built on the shoulders of Codex, Increase pair Coding Of NLP

Codex framework

Codex Is far more accurate than GPT-3

OpenAI Of AI Code suggestion tool GitHub Copilot In natural language processing (NLP) Model Codex Up operation , The model is GPT-3 Enhanced version of . Although it is built to implement and AlphaCode Similar goals , but Copilot It seems that the road ahead is difficult . Here are some differences between the two code generation tools .

1. Training ——GitHub Copilot Of AI Codex Trained to recognize 120 One hundred million parameters , and AlphaCode Based on the AI The code generation model uses 400 100 million parameters for training . This will AlphaCode Four times the performance of .
2. Suggestions and generation ： although GitHub Copilot Designed to help programmers write basic code parts , but AlphaCode Able to generate complete complex programs .
3. complexity —— Although these two AI Tools are in the early stages of development , but GitHub Copilot It is recommended to use the basic code of simple logic , and AlphaCode Tested to produce competitive complex algorithms .

Reference resources

https://analyticsindiamag.com/copilot-vs-alphacode-the-race-for-coding-supremacy/

https://arxiv.org/pdf/1907.05774.pdf

https://arxiv.org/pdf/2111.08489.pdf

https://arxiv.org/pdf/2107.03374.pdf