当前位置:网站首页>Understanding disentangling in β- VAE paper reading notes
Understanding disentangling in β- VAE paper reading notes
2022-07-06 18:46:00 【zeronose】
List of articles
Preface
article :Understanding disentangling in β-VAE
Link to the original text : link
Understanding disentangling in β-VAE Is based on β-VAE An article from .
First ,β-VAE There are several problems in :
1.β-VAE Just by KL Item adds a super parameter β, It is found that the model has decoupling characteristics , But there is no good explanation for adding a super parameter β Will produce decoupling characteristics .
2.β-VAE Find out , When the decoupling effect is good, the reconstruction effect is not good , When the reconstruction effect is good, the decoupling effect is poor , So we need to balance decoupling and reconstruction .
Based on this ,Understanding disentangling in β-VAE adopt Information bottleneck theory given β-VAE Explanation of decoupling , And for β-VAE We need to balance decoupling and reconstruction , They put forward their own training methods ---- Gradually increase the amount of information of potential variables in the training process .
The original text also introduces VAE And β-VAE, It's just more here , Interested can see my previous article
VAE
β-VAE
One 、 What is the information bottleneck ?
The figure below is the explanation of the information bottleneck in the original text , In fact, information bottleneck describes a constrained optimization goal , The goal is to maximize potential bottlenecks Z And tasks Y Mutual information between , At the same time, discard the input X About Y All irrelevant information .
Drawing is troublesome , Let's make do with it , As shown in the figure below
You can see β-VAE The loss function of is :
L(θ, ϕ, β; x, z) = Eqϕ(z|x)ln pθ(x|z)− βKL (qϕ(z|x) || p(z))
The first term on the right of the equation is the reconstruction term , The second term is the regular term . The second is the information bottleneck of the first , Increase the weight of the second term, that is β Value , Also is to let qϕ(z|x) Closer to the p(z), because p(z) It's the standard Zhengtai distribution , At this time, the implicit variables are limited z It contains x The amount of information , Therefore, the decoupling effect is good, but the reconstruction effect is poor , On the contrary, the decoupling effect is poor , The effect of refactoring is good .
Two 、 New training goals
1. Loss function
among ,γ Fixed to a larger number 1000,C Is a variable number . In the process of training ,C Gradually increase from zero to a value large enough to produce high-quality reconstruction .
The training process here is similar to β-VAE Different ,β-VAE During training , It needs to be fixed first β Value then train , change β You need to retrain after the value . there C Is a variable parameter , It also becomes the amount of information , He is in the process of training , from 0 Gradually increasing .
summary
By controlling the increase of potential posterior coding ability in the training process , Allow the previous average KL The difference increases gradually from zero , Not the original β-VAE Fixed in the target β weighting KL Increase of items . Compared with the result of the original formula , It promotes Robust learning of disentangled representation , Combined with better reconstruction fidelity .
边栏推荐
- POJ 2208 six lengths of tetrahedron are known, and the volume is calculated
- Summary of performance knowledge points
- AcWing 3537. Tree lookup complete binary tree
- Bonecp uses data sources
- From 2022 to 2024, the list of cifar azrieli global scholars was announced, and 18 young scholars joined 6 research projects
- celery最佳实践
- None of the strongest kings in the monitoring industry!
- Reproduce ThinkPHP 2 X Arbitrary Code Execution Vulnerability
- DOM简要
- AcWing 3537.树查找 完全二叉树
猜你喜欢
C#/VB.NET 给PDF文档添加文本/图像水印
Breadth first traversal of graph
Self supervised heterogeneous graph neural network with CO comparative learning
C#/VB. Net to add text / image watermarks to PDF documents
On AAE
44所高校入选!分布式智能计算项目名单公示
十、进程管理
Oracle advanced (IV) table connection explanation
Binary search tree
Video based full link Intelligent Cloud? This article explains in detail what Alibaba cloud video cloud "intelligent media production" is
随机推荐
JDBC驱动器、C3P0、Druid和JDBCTemplate相关依赖jar包
人体骨骼点检测:自顶向下(部分理论)
Brief description of SQL optimization problems
Numerical analysis: least squares and ridge regression (pytoch Implementation)
具体说明 Flume介绍、安装和配置
MySQL查询请求的执行过程——底层原理
UFIDA OA vulnerability learning - ncfindweb directory traversal vulnerability
How are you in the first half of the year occupied by the epidemic| Mid 2022 summary
Crawling data encounters single point login problem
SQL injection Foundation
解读云原生技术
Some understandings of tree LSTM and DGL code implementation
Medical image segmentation
Shangsilicon Valley JUC high concurrency programming learning notes (3) multi thread lock
[Matlab] Simulink 同一模块的输入输出的变量不能同名
一种用于夜间和无袖测量血压手臂可穿戴设备【翻译】
Describe the process of key exchange
Introduction and case analysis of Prophet model
Jdbc driver, c3p0, druid and jdbctemplate dependent jar packages
安装及管理程序