当前位置:网站首页>Deep clustering: joint optimization of depth representation learning and clustering
Deep clustering: joint optimization of depth representation learning and clustering
2022-07-07 06:01:00 【This Livermore is not too cold】
reference :
brief introduction
Classical clustering means that data is characterized by vectorization through various representation learning techniques . As data becomes more and more complex , Shallow ( Tradition ) Clustering methods have been unable to deal with high-dimensional data types . A direct way to combine the advantages of deep learning is to first learn the depth representation , Then input it into the shallow clustering method . But there are two disadvantages :i) It means that you are not learning clustering directly , This limits clustering performance ; ii) Clustering depends on complex rather than linear relationships between instances ; iii) Clustering and representation learning are interdependent , Should be mutually reinforcing .
In order to solve the problem , The concept of deep clustering is proposed , That is, joint optimization represents learning and clustering .
Purpose
The main purpose of clustering is to group instances , Make similar samples belong to the same cluster , Different samples belong to different clusters . Sample clusters provide a global representation of data instances , This can significantly facilitate further analysis of the entire data set , For example, anomaly detection , Domain adaptation , Community detection and discrimination represent learning .
Image Representation Learning.
Learning technology through modern representation ( For example, visual converter ) The introduction of deep clustering has made progress . As one of the most popular directions , Unsupervised representation learning of image data will play a central role in deep clustering and affect other data types .
Text Representation Learning
Early attempts at text representation learning used statistical methods , Such as TF-IDF 、Word2Vec and Skip-Gram. later , Some work focuses on topic modeling and semantic distance in text representation learning , And more about unsupervised scenarios . lately , image BERT and GPT-3 Such a pre training language model gradually occupies a dominant position in the field of text representation learning .
Video Representation Learning
Video shows that learning is a challenging task , It will learn from time and space 、 Multi model learning and natural language processing ( With video summary and subtitles ) Combine into one place . Early methods used LSTM Autoencoder 、3D-ResNets and 3D-U-Net As a feature extractor .
Graph Representation Learning
The classical graph representation learning aims to learn the low dimensional representation of nodes , In order to preserve the proximity between nodes in the embedded space . Besides , Map level information also has great potential in tasks such as protein classification , This has attracted more and more attention in graph level representation learning
The data type specific representation learning mentioned above can be simple backbone for feature extraction or end-to-end unsupervised representation learning , This is the most active research direction in deep learning . As more types of data are collected , Deep clustering will grow with data type specific representation learning techniques .
Deep learning method
Multilevel deep clustering (Multistage Deep Clustering)
Multilevel deep clustering refers to the method of optimizing and connecting two modules respectively . One direct way is Using deep unsupervised representation learning technology, first learn the representation of each data instance , Then input the learned representation into the classical clustering model to obtain the final clustering result . This separation of data processing and clustering is helpful for researchers to carry out cluster analysis . More specifically , All existing clustering algorithms can serve any Research scenario .
Iterative depth clustering (Iterative Deep Clustering)
The main purpose of iterative depth clustering is Good representation can benefit clustering , The clustering results provide supervision for representation learning in reverse . Most existing iterative depth clustering pipelines update iteratively between two steps
1) Calculate the clustering result given the current representation ;
2) Update the representation given the current clustering result . Iterative depth clustering method benefits from the mutual promotion between representation learning and clustering . However , They are also affected by error propagation during iteration . Inaccurate clustering results may lead to chaotic representation , Among them, the performance is limited by the effectiveness of self marking . Besides , This in turn will affect the clustering results , Especially in the early stages of training . therefore , Existing iterative clustering methods rely heavily on the pre training of representation modules .
Generate deep clustering (Generative Deep Clustering)
Deep generation clustering model can Generate samples while clustering
The generated model can capture 、 Represent and recreate data points , Therefore, it has attracted more and more attention from academia and industry . They will make assumptions about the potential cluster structure , Then the cluster allocation is inferred by estimating the data density . The most representative model is Gaussian mixture model , It assumes that data points are generated from Gaussian mixture
There are weaknesses :
1) Training generation models usually involve Monte Carlo sampling , It may lead to unstable training and high computational complexity ;
2) The mainstream generation model is based on VAE and GAN, Inevitably inherited their shortcomings . be based on VAE Our model usually requires a priori assumption of data distribution , This may not be true in actual cases ; Even though be based on GAN Our algorithm is more flexible , But they usually have problems of mode collapse and slow convergence , Especially for data with multiple clusters .
At the same time, deep clustering (Simultaneous Deep Clustering)
The representation learning module and the clustering module are optimized simultaneously in an end-to-end manner . Although most iterative depth clustering methods also optimize two modules with a single objective , But these two modules are optimized by explicit iteration , Cannot update at the same time . Although most iterative depth clustering methods also optimize two modules with a single objective , But these two modules are optimized by explicit iteration , Cannot update at the same time .
The representation of learning is Clustering Oriented , Clustering is carried out on the discriminant space . however , It may produce an unwanted bias against the optimization focus between the representation learning module and the clustering module , At present, this bias can only be alleviated by manually setting the balance parameters . Besides , The model can easily fall into a degraded solution , All instances are assigned to a cluster
Common data set
4 Common data sets in three directions
The future direction
Initialization of Deep Clustering Module
Overlapping Deep Clustering
Degenerate Solution VS Unbalanced Data
Boosting Representation with Deep Clustering
Deep Clustering Explanation
Transfer Learning with Deep Clustering
Clustering with Anomalies
Efficient Training VS Global Modeling
边栏推荐
- R language [logic control] [mathematical operation]
- Forkjoin is the most comprehensive and detailed explanation (from principle design to use diagram)
- 【已解决】记一次EasyExcel的报错【读取xls文件时全表读不报错,指定sheet名读取报错】
- The solution of a simple algebraic problem
- The boss always asks me about my progress. Don't you trust me? (what do you think)
- JVM命令之 jinfo:实时查看和修改JVM配置参数
- Input of native applet switches between text and password types
- An example of multi module collaboration based on NCF
- EMMC print cqhci: timeout for tag 10 prompt analysis and solution
- How much do you know about clothing ERP?
猜你喜欢
如果不知道这4种缓存模式,敢说懂缓存吗?
Say sqlyog deceived me!
话说SQLyog欺骗了我!
PTA ladder game exercise set l2-004 search tree judgment
力扣102题:二叉树的层序遍历
Detailed explanation of platform device driver architecture in driver development
[cloud native] what is the microservice architecture?
Introduction to distributed transactions
cf:C. Column Swapping【排序 + 模拟】
ML之shap:基于adult人口普查收入二分类预测数据集(预测年收入是否超过50k)利用shap决策图结合LightGBM模型实现异常值检测案例之详细攻略
随机推荐
Introduction to the extension implementation of SAP Spartacus checkout process
牙齿干细胞的存储问题(未完待续)
一名普通学生的大一总结【不知我等是愚是狂,唯知一路向前奔驰】
@pathvariable 和 @Requestparam的详细区别
Dynamic memory management
zabbix_ Get test database failed
make makefile cmake qmake都是什么,有什么区别?
[daily training -- Tencent selected 50] 235 Nearest common ancestor of binary search tree
Jstat pour la commande JVM: voir les statistiques JVM
SAP ABAP BDC (batch data communication) -018
[SQL practice] a SQL statistics of epidemic distribution across the country
JVM命令之 jstat:查看JVM统计信息
深度聚类:将深度表示学习和聚类联合优化
SQLSTATE[HY000][1130] Host ‘host. docker. internal‘ is not allowed to connect to this MySQL server
Add salt and pepper noise or Gaussian noise to the picture
On the difference between FPGA and ASIC
JVM命令之 jstat:查看JVM統計信息
Flask 1.1.4 werkzeug1.0.1 analyse du code source: processus de démarrage
What is make makefile cmake qmake and what is the difference?
PTA 天梯赛练习题集 L2-002 链表去重