Paper notes: multi label learning bp-mll

Abstract : Share your understanding of the paper . See the original Zhang, M.-L., & Zhou, Z.-H. (2006). Multi-label neural networks with applications to functional genomics and text categorization. IEEE transactions on Knowledge and Data Engineering, 18, 1338–1351.

1. Contribution of thesis

The first multilabel neural network algorithm.
The first neural network algorithm for multi label learning .

2. The main idea

The labels belonging to an instance should be ranked higher than those not belonging to that instance.
example : If there is a cat but no dog in the picture , The predicted value of cat should be larger than that of dog .
A more practical example : The predicted value for the cat tag is 0.6, For dogs 0.3, The penalty is smaller ( In machine learning, the general penalty value is greater than 0); Otherwise, the punishment will be great .
Further interpretation : Pairwise comparison .

3. Symbol system

Symbol	meaning	explain
$\mathcal{X}={\mathbb{R}}^d$	$d$ Dimensional feature space
$\mathcal{Y}=\{1,2,\dots ,Q\}$	Tag space	Use integer sets instead of $\{-1,1{\}}^d$
$h:\mathcal{X}\to 2^{\mathcal{Y}}$	classifier	Use power set
$f:\mathcal{X}\times \mathcal{Y}\to \mathbb{R}$	Regressor	Predict a probability for each tag of each object

explain :

1. From regressor to classifier , Just increase the threshold .
2. There are some ( More popular ) MLL Evaluation indicators do not need to be output $+1$ or $-1$, It's the sorting of labels .

4. Algorithm is the core

chart 1. Neural network structure

$$E=\sum _{i=1}^m{E}_i=\sum _{i=1}^m\frac{1}{|Y_i||\overline{Y_i}|}\sum _{(k,l)\in Y_i\times \overline{Y_i}}\exp (-(c_k^i-c_l^i)),\text{\hspace{1em}(1)}$$
among :

• $\overline{Y_i}=\mathcal{Y}\setminus Y_i$, namely $Y_i$ The complement of .
• $c_k^i=f({\mathbf{x}}_{\mathbf{i}},k)$ Is a neural network for ${\mathbf{x}}_{\mathbf{i}}$ The first $k$ The predicted value of a tag .
• The first summation is for all objects .
• The second summation is essentially a double loop , Such as $|Y_i|=10$, $Q=100$, You need to ask for $10\times 90=900$ Yes .
• When $\exp$ The parameter of is less than 0 when , Less loss , Otherwise it will be quite large .

5. Summary

• The idea of pairwise comparison .
• A paper on machine learning , A loss function is enough .
• You can put $\exp$ Replace with another single increasing function , Or change the network structure , To make small innovations .