当前位置：网站首页>Sorting and sharing of selected papers, systems and applications related to the most comprehensive mixed expert (MOE) model in history

Sorting and sharing of selected papers, systems and applications related to the most comprehensive mixed expert (MOE) model in history

2022-07-04 21:44:00 【lqfarmer】

sparsity （Sparsity）, It means that the model has a very large capacity , But only the model is used for a given task 、 Some parts of the sample or mark are activated . such , It can significantly increase the capacity and capacity of the model , Without proportionally increasing the amount of calculation .

2017 year , Google introduced a sparse gated expert hybrid layer （Sparsely-Gated Mixture-of-Experts Layer,MoE）, This layer shows better results in various transformation benchmarks , The calculations used at the same time are more intensive than the most advanced before LSTM There are few models 10 times .

This resource collates the mixed experts in recent years (MoE) Related papers , And classified in detail . Mark this knowledge base , Then you can keep up with the latest developments in this booming research field .

Resources are organized from the Internet , See the source address for downloading and obtaining ：https://github.com/codecaution/Awesome-Mixture-of-Experts-Papers#awesome-mixture-of-experts-papers

Catalog