Su Weijie, a member of Qingyuan Association and an assistant professor at the University of Pennsylvania, won the first Siam Youth Award for data science, focusing on privacy data protection, etc

---

edit ： Good trapped

【 Introduction to new wisdom 】 In recent days, ,SIAM Announced the first 2022 Winner of the Youth Award for data science , Members of the youth source Association 、 Assistant professor at the University of Pennsylvania 、 School of Mathematical Sciences, Peking University 2011 Alumni 、 Dr. Su Weijie of Stanford University is the only winner .

 

In recent days, ,SIAM Announced the first 2022 Winner of the Youth Award for data science , School of Mathematical Sciences, Peking University 2011 Alumni 、 Dr. Su Weijie of Stanford University is the only winner .

 

SIAM The Youth Award for data science was awarded by SIAM（ American Society of industrial and Applied Mathematics ） It is issued worldwide every two years , It aims to reward a young scholar who has made outstanding contributions in the field of data science .

 

This year's Award Committee includes Germany 、 The United States 、 Belgium, Hong Kong, China and many other countries are engaged in Applied Mathematics 、 Internationally renowned scholars in the field of machine learning and optimization . The award ceremony will be held this year 9 In July in San Diego, USA SIAM At the data Science Conference . At that time, Su Weijie will be in SIAM Make a specially invited report to the Conference .

 

Su Weijie is now an assistant professor in the Department of statistics and data science and the Department of computer science at the Wharton School of business, University of Pennsylvania , as well as Facebook Visiting scientists . Professor Su Weijie is also the co director of the machine learning research center of Penn University , And served on the Executive Committee of Applied Mathematics and computational mathematics projects . In addition, he also works in Wharton intelligent business center of Penn University ,Warren Network Data Science Center , And the Qingyuan conference of Beijing Zhiyuan Artificial Intelligence Research Institute . He was in 2019 Years and 2020 In, respectively, obtained NSF CAREER Award And Sloan research award .

 

Su Weijie's study experience is also quite wonderful .

 

He participated in the Chinese Mathematics Olympiad on behalf of Zhejiang Province twice in high school , The first year of senior high school is the pre admission qualification of Tsinghua University , The third year of senior high school won the second place in the country .2007 - 2011 He studied basic mathematics at the school of Mathematical Sciences, Peking University , During this period, he won the highest scholarship for three consecutive years , The professional performance ranks first in the basic mathematics major .

 

While studying at Peking University , Su Weijie won the all-round gold medal and applied mathematics gold medal in the first Qiu Chengtong college students' Mathematics Competition , And two bronze medals in algebra and analysis .2010 Su Weijie and his teammates participated in the mathematical modeling competition of American college students , Get the choice B The first place in the direction of Chinese Mainland . Later, he won the highest level doctoral scholarship of Stanford University , stay 2016 Got a doctorate in , His graduation thesis won the first Stanford Theodore Anderson prize . Then skip the post doctoral stage and teach directly at the University of Pennsylvania .

 

Introduction to the award-winning work

 

Professor Su Weijie obtained SIAM The data Science Youth Award is based on his optimization algorithm in machine learning 、 Data privacy protection 、 Deep learning theoretical basis and important contributions made by high-dimensional statistics .

 

Add momentum （momentum） It is a common skill when optimizing machine learning models , For example, one of the three giants of deep learning 、ACM Turing prize winner Yoshua Bengio An important work points out that adding momentum can significantly accelerate the optimization of deep Neural Networks .

 

Famous in convex optimization Nesterov Acceleration algorithm , It is to add a momentum term to the general gradient descent algorithm , The effect is improved significantly , But the mechanism behind it has yet to be clarified .

Thesis link ：https://jmlr.org/papers/volume17/15-084/15-084.pdf

 

A representative work of Su Weijie is to provide an analysis and design framework for a class of optimization algorithms that add momentum , Especially for Nesterov The acceleration algorithm provides a very intuitive explanation [1]. The core of this framework is to look at discrete algorithms from the continuous point of view of dynamical systems , make the best of 「 Continuous mathematics 」 Analytical advantages .

Su Weijie and academician of the American Academy of Engineering Stephen Boyd Academician of the American Academy of Sciences Emmanuel Candes

 

This work has been used and popularized by many researchers of machine learning theory . Machine learning champion Michael Jordan stay 2018 Rio International Congress of mathematicians 1 In the hourly report , He introduced the latest achievements of his team in promoting Professor Su Weijie's work at a large length .

 

Michael Jordan stay 2018 The International Conference of mathematicians introduced how to promote Su Weijie's work on machine learning optimization algorithms

 

Another award-winning work of Su Weijie is the Gauss differential privacy developed by his team （Gaussian Differential Privacy） frame [2].

 

Thesis link ：https://rss.org.uk/RSS/media/Training-and-events/Events/2020/Dong-et-al-jrssb-final.pdf

 

Privacy in artificial intelligence has been recognized as an important and serious problem ,2006 Academicians of the American Academy of Sciences and the Academy of Engineering Cynthia Dwork The differential privacy proposed by et al. Laid the foundation of privacy data analysis .

 

Su Weijie was in Mountain View Microsoft research follows Cynthia Dwork when , Realize that this framework is inefficient in analyzing some basic privacy algorithms , It has great limitations when applied to deep learning .

 

Gauss difference privacy in the Royal Statistical Society invited report

 

Gauss difference privacy creatively describes privacy algorithms accurately from the perspective of hypothesis testing , It is strictly proved theoretically that this new framework has several optimal properties , This paper serves as Discussion Paper Invited to study at the Royal Statistical Society .

 

Su Weijie's team also successfully applied Gaussian differential privacy to the training of deep Neural Networks , Under the condition of the same degree of privacy protection, it has achieved better than Google Brain Higher prediction accuracy . This new privacy data analysis framework has been incorporated TensorFlow, Get the attention of the industry , It is expected to be applied to the products of a flagship manufacturer in Silicon Valley .

 

Su Weijie's recent contribution to deep learning theory is also the reason for his award . Deep neural network has achieved excellent performance in many scientific and engineering problems , However, there has been a lack of satisfactory theoretical explanation for its good generalization performance .

 

Su Weijie proposed the local elasticity of deep Neural Networks （local elasticity） theory , Generalize Neural Networks 、 Properties such as optimization provide a simple phenomenological theory [3].

Thesis link ：https://openreview.net/pdf?id=HJxMYANtPH

 

Besides , Su Weijie's team proposed an interlayer 「 be stripped 」 Analytical model of , Another new idea is given to the above problems [4].

Thesis link ：https://www.pnas.org/content/118/43/e2103091118

 

Strong expression ability based on Neural Network , This new model regards some layers of the network as a whole , Its output characteristics are regarded as an optimization variable that can adapt to the network training process , The interaction between features and subsequent layer parameters in network training is emphatically studied .

 

Su Weijie's team used this model to deeply analyze the performance of deep neural network when the training data is unbalanced , A novel phenomenon with important practical significance has been found . This achievement has recently been published in top journals 《 Proceedings of the National Academy of Sciences 》 On .

 

This new model also explains the academicians of the American Academy of Sciences David Donoho The team found a nervous collapse （neural collapse） The phenomenon . This phenomenon shows that the excellent performance of neural networks is largely due to geometric symmetry .

 

Interlaminar 「 be stripped 」 The analysis model shows that the neural collapse comes from the symmetry of minimizing the objective function under certain constraints , This mathematically rigorous explanation has been Donoho Academicians spoke highly of .

 

Reference material ：

[1] W. Su, S. Boyd, and E. Candes. A differential equation for modeling Nesterov’s accelerated gradient method: Theory and insights. Journal of Machine Learning Research, 17(1):5312–5354, 2016.

[2] J. Dong, A. Roth, and W. Su. Gaussian differential privacy. Journal of the Royal Statistical Society: Series B (with discussion), 2022.

[3] H. He and W. Su. The local elasticity of neural networks. In International Conference on Learning Representa tions, 2020.

[4] C. Fang, H. He, Q. Long, and W. Su. Exploring deep neural networks via layer-peeled model: Minority collapse in imbalanced training. Proceedings of the National Academy of Sciences, 118(43), 2021.

Link to the original text