Sorting and sharing of selected papers, systems and applications related to the most comprehensive mixed expert (MOE) model in history

     sparsity （Sparsity）, It means that the model has a very large capacity , But only the model is used for a given task 、 Some parts of the sample or mark are activated . such , It can significantly increase the capacity and capacity of the model , Without proportionally increasing the amount of calculation .

    2017 year , Google introduced a sparse gated expert hybrid layer （Sparsely-Gated Mixture-of-Experts Layer,MoE）, This layer shows better results in various transformation benchmarks , The calculations used at the same time are more intensive than the most advanced before LSTM There are few models 10 times .

     This resource collates the mixed experts in recent years (MoE) Related papers , And classified in detail . Mark this knowledge base , Then you can keep up with the latest developments in this booming research field .

    

     Resources are organized from the Internet , See the source address for downloading and obtaining ：https://github.com/codecaution/Awesome-Mixture-of-Experts-Papers#awesome-mixture-of-experts-papers

Catalog

Content screenshot

Recommended contents of previous boutiques

A detailed explanation baseline The paper Reproduce actual combat （NLP）

Write some suggestions to current and future doctoral students to sort out and share

2021 Sorting and sharing of the most complete selected resources for in-depth intensive learning in

《 Automatic machine learning ： Method , Systems and challenges 》- The latest version - As a free download

LeetCode selected 101 It's necessary to brush questions （C++）- Detailed classification and disintegration instructions are attached - free pdf Share

NLP Free new books -《 Overview of word vector representation algorithm in natural language processing 》 Share

Federal learning - Machine learning architecture based on distributed privacy data