Progress in architecture

Catalog

Progress in architecture

NAS An introduction to the

---

Progress in architecture

NAS An introduction to the

In terms of Architecture , One way to improve efficiency is to rely on human creativity to design . Another method is machine driven neural architecture search (NAS), Greatly reduce the workload of algorithm development .

although NAS The amount of calculation itself is very large 、 The high cost of , But on the whole, it can significantly reduce the amount of calculation in the downstream development and production environment .

Such as NAS Method search Evovled Transformer, In parameter reduction 37.6% In the case of 0.7% Improved English German translation performance .

On visual tasks ,NAS Method derived Efficientnetv2 The training speed of the model is faster than before SOTA The model improves 5-11 times .

In addition to the model architecture ,AutoML-Zero Also used NAS Methods to find new 、 More effective reinforcement learning / Supervised learning algorithm .