Or talk No.19 | Facebook Dr. Tian Yuandong: black box optimization of hidden action set based on Monte Carlo tree search

Want to make APP Same thing as WeChat , Can run small programs smoothly ？ | Experience will send you to Xinjiang 、 Huawei 、 Cherry keyboard ！>>>

Share the outline

The theme ：《 Black box optimization of hidden action set based on Monte Carlo tree search 》

The guest ： @ Tian Yuandong Doctor

Time ： Beijing time. 2020 year 11 month 7 Number （ Saturday ） Good morning! 10：00

place ：『 Operational research OR A strategy 』 Bili Bili studio

link ：live.bilibili.com/21459168

brief introduction

In the near future ,Facebook AI Lab Dr. Tian Yuandong and Wang Linnan of Brown University and his boss Rodrigo Fonseca Co published an article on black box optimization (arXiv:2007.00708), A new concept called La-MCTS (Latent Action Monte Carlo Tree Search) Black box optimization of （Black-box optimization） Method . The hidden action set here (Latent Action, La) Refer to , Select a good subspace from the current node of the search space （ The left node ）, Or bad subspaces （ Right node ）.

The goal of traditional Monte Carlo tree search is to search in a given state space （state space S）、 Action space （action space A） And state transition functions （transition matrix, S->A->S') , The traditional Monte Carlo tree search searches how many rewards there are for past behaviors , Find the best action sequence and get the biggest reward . Black box optimization starts from a good starting point to find the optimal solution , It can also be modeled in this way .

But between it and traditional reinforcement learning , There's a key difference ： Black box optimized action space can be arbitrarily specified , As long as it is conducive to the search for the optimal solution .LaMCTS It's taking advantage of this , By automatically learning the structure of action space to improve search efficiency .

LaMCTS As a meta algorithm （meta-algorithm）, We use nonlinear function to partition space , Can be superimposed on any known black box optimization algorithm , such as Bayesian Optimization（BO） above . This algorithm limits the modeling of high-dimensional Gaussian process in a relatively small range , So as to find the optimal solution in the sub region of leaf node more quickly . In practical terms , Black box optimization is often used in situations where function calls are expensive and derivative information is not available , For example, the value of a function is the average efficiency of a complex system after a day's operation , Or it's a very expensive experiment to get , wait , By reducing the sample complexity of the optimal solution , It can greatly reduce the cost .

LaMCTS Has been NeurIPS 2020 receive . The source code of the algorithm has been published in Github On .

(https://github.com/facebookresearch/LaMCTS)

This live broadcast , Dr. Tian will explain the background and content of this paper in detail .

Introduction to guests

Dr. Tian Yuandong , facebook （Facebook） Researcher and manager of the Institute of artificial intelligence , The research direction is deep reinforcement learning , Multi agent learning , And its application in games , And the theoretical analysis of deep learning model . Worked as an open source go project DarkForest And ELF OpenGo Research and engineering director and first author of the project .2013-2014 In Google The driverless team works as a software engineer .2005 Years and 08 He received his master's degree from Shanghai Jiaotong University in 1986 ,2013 He received his doctorate from the Institute of robotics, Carnegie Mellon University, USA . Have obtained 2013 International Conference on computer vision （ICCV） The Mar prize nomination （Marr Prize Honorable Mentions）.

Reference reading ：

Introduce two articles NeurIPS The article （ Two ）

Brown University and FAIR Open source LA-MCTS, And its application in neural network structure search