当前位置:网站首页>The worse the AI performance, the higher the bonus? Doctor of New York University offered a reward for the task of making the big model perform poorly
The worse the AI performance, the higher the bonus? Doctor of New York University offered a reward for the task of making the big model perform poorly
2022-07-05 18:40:00 【Zhiyuan community】
The bigger the model 、 The worse the performance, the better the prize ?
The total bonus is 25 Ten thousand dollars ( Renminbi conversion 167 ten thousand )?
such “ Out of line ” It really happened , A man named Inverse Scaling Prize( Anti scale effect Award ) The game of caused heated discussion on twitter .
The competition was organized by New York University 7 Jointly organized by researchers .
Originator Ethan Perez Express , The main purpose of this competition , It is hoped to find out which tasks will make the large model show anti scale effect , So as to find out some problems in the current large model pre training .
Now? , The competition is receiving contributions , The first round of submissions will end 2022 year 8 month 27 Japan .
Competition motivation
People seem to acquiesce , As the language model gets bigger , The operation effect will be better and better .
However , Large language models are not without flaws , For example, race 、 Gender and religious prejudice , And produce some fuzzy error messages .
The scale effect shows , With the number of parameters 、 The amount of computation used and the size of the data set increase , The language model will get better ( In terms of test losses and downstream performance ).
We assume that some tasks have the opposite trend : With the increase of language model testing loss , Task performance becomes monotonous 、 The effect becomes worse , We call this phenomenon anti scale effect , Contrary to the scale effect .
This competition aims to find more anti scale tasks , Analyze which types of tasks are prone to show anti scale effects , Especially those tasks that require high security .
meanwhile , The anti scale effect task will also help to study the potential problems in the current language model pre training and scale paradigm .
As language models are increasingly applied to real-world applications , The practical significance of this study is also increasing .
Collection of anti scale effect tasks , It will help reduce the risk of adverse consequences of large language models , And prevent harm to real users .
Netizen disputes
But for this competition , Some netizens put forward different views :
I think this is misleading . Because it assumes that the model is static , And stop after pre training .
This is more a problem of pre training on standard corpora with more parameters , Not the size of the model .
Software engineer James Agree with this view :
Yes , This whole thing is a hoax . Anything a small model can learn , Large models can also .
The deviation of the small model is larger , therefore “ Hot dogs are not hot dogs ” It may be recognized as 100% Right , When the big model realized that it could make cakes similar to hot dogs , The accuracy will drop to 98%.
James Even further proposed “ Conspiracy theories ” View of the :
Maybe the whole thing is a hoax —— Let people work hard , And show the training data when encountering difficult tasks , This experience will be absorbed by large models , Large models will eventually be better .
So they don't need to give bonuses , You will also get a better large-scale model .
Regarding this , Originator Ethan Perez Write in the comment :
Clarify it. , The focus of this award is to find language model pre training that will lead to anti scale effect , Never or rarely seen category .
This is just a way to use large models . There are many other settings that can lead to anti scale effects , Not included in our awards .
边栏推荐
- Copy the linked list with random pointer in the "Li Kou brush question plan"
- Is it safe to make fund fixed investment on access letter?
- LeetCode 6109. Number of people who know the secret
- 文章中的逻辑词
- C language makes it easy to add, delete, modify and check the linked list "suggested collection"
- ViewPager + RecyclerView的内存泄漏
- [QNX hypervisor 2.2 user manual]6.3.2 configuring VM
- 爬虫01-爬虫基本原理讲解
- Solutions contents have differences only in line separators
- Electron安装问题
猜你喜欢
MYSQL中 find_in_set() 函数用法详解
About statistical power
How to write good code defensive programming
The main thread anr exception is caused by too many binder development threads
buuctf-pwn write-ups (9)
buuctf-pwn write-ups (9)
@Extension、@SPI注解原理
The 10th global Cloud Computing Conference | Huayun data won the "special contribution award for the 10th anniversary of 2013-2022"
ViewPager + RecyclerView的内存泄漏
Let more young people from Hong Kong and Macao know about Nansha's characteristic cultural and creative products! "Nansha kylin" officially appeared
随机推荐
Memory leak of viewpager + recyclerview
Electron安装问题
Find in MySQL_ in_ Detailed explanation of set() function usage
RPC protocol details
[use electron to develop desktop on youqilin]
Einstein sum einsum
【HCIA-cloud】【1】云计算的定义、什么是云计算、云计算的架构与技术说明、华为云计算产品、华为内存DDR配置工具说明
Copy the linked list with random pointer in the "Li Kou brush question plan"
Introduction to the development function of Hanlin Youshang system of Hansheng Youpin app
Maximum artificial island [how to make all nodes of a connected component record the total number of nodes? + number the connected component]
ClickHouse(03)ClickHouse怎么安装和部署
线性表——抽象数据类型
Clickhouse (03) how to install and deploy Clickhouse
Thoroughly understand why network i/o is blocked?
解决 contents have differences only in line separators
深入底层C源码讲透Redis核心设计原理
英语句式参考
The main thread anr exception is caused by too many binder development threads
集合处理的利器
Vulnhub's darkhole_ two