Interpretation of hard threshold function [easy to understand]

Hello everyone , I meet you again , I'm your friend, Quan Jun .

subject ： Hard threshold (Hard Thresholding) Function interpretation

1、 Hard threshold (Hard Thresholding) The sign of the function

Hard threshold (Hard Thresholding) There is no soft threshold (Soft Thresholding) So common , This may be because the problem solved by hard threshold is non convex . Hard threshold and soft threshold are proposed by the same literature , For the hard threshold formula, see the literature 【1】 Formula of ( 11)：

First encounter hard threshold (HardThresholding) It's in the literature 【2】 in ：

When querying soft threshold (Soft Thresholding) In the process of , Found the literature 【3】, Then we see the reference to the literature 【4】：

The literature 【4】 Mentioned in Fig 1 As shown in the figure ：

What does the symbol of hard threshold mean ？ With literature 【1】 Take symbols as an example , To be clear, that's it ：

here w It's a variable. ,λ It's the threshold .

2、 Hard threshold (HardThresholding) Function function

Figure out the hard threshold (HardThresholding) The symbol of indicates that after , Let's talk about its function . Here we mainly refer to the derivation process of soft threshold , Then after some pondering and derivation, I got .

Hard threshold (HardThresholding) The following optimization problems can be solved ：

among ：

||X||0 Is the direction is the vector X Zero norm of , The vector X The number of non-zero elements in . According to the definition of norm , The objective function of the above optimization problem can be disassembled ：

Where the symbol in the split item |x|0 It means

Now? , We can solve N An independent shape such as a function

Optimization problem , To solve this problem . take f(x) Further written as ：

about x≠0 part , We know that its minimum value is x=b Obtained at , The minimum value is λ. The problem now is λ And b2 Who is younger ？ The smallest will be the function f(x) The minimum value of . Solving inequalities b2>λ Available

At this time, the minimum value is x=0 Obtained at ;

Solving inequalities b2<λ Available

At this time, the minimum value is x=b Obtained at ;

therefore

With the previous hard threshold (Hard Thresholding) Compare the , Found? ？ If... In the above formula is used b As variables ,sqrt(λ) As a threshold , The above formula is the hard threshold (Hard Thresholding) Formula .

thus , We can get the optimization problem

The solution of is

notes ： The formula is hard threshold (Hard Thresholding) Matrix form of , there B It's a vector , The hard threshold function should be executed element by element ;.

3、 Hard threshold (HardThresholding) Deformation of

When the optimization problem becomes

Because multiplying the objective function by a constant coefficient does not affect the acquisition of the extreme point , So it can be equivalent to an optimization problem

The solution at this time is

.

4、 Hard threshold (Hard Thresholding) Of MATLAB Code

Hard threshold (Hard Thresholding) The function code can be written specifically for optimization problems

MATLAB The function code is as follows （ With reference to the literature 【5】 Last but not least 2 page ）：

function [ hard_thresh ] = hardthresholding( b,lambda )
    sel = (abs(b)>sqrt(lambda));
    hard_thresh = b.*sel;
end

Copy

Be sure to pay attention to ： This way of writing is aimed at the initial optimization problem ：

But I personally feel that it should be written in this general form ：

function [ x ] = hard( b,T )
    sel = (abs(b)>T);
    x = b.*sel;
end

Copy

After that , To solve the optimization problem

Just call hard(B, sqrt(λ)) that will do ; To solve the optimization problem

Just call hard(B, sqrt(2*λ)) that will do .

5、 Hard threshold (HardThresholding) Test code

Hard threshold (Hard Thresholding) The objective function of the optimization problem to be solved is nonconvex , Less common , There are no other functions at hand to solve this problem , Therefore, the test code can only test whether the function is written correctly ：

clear all;close all;clc; 
b = [-0.8487   -0.3349    0.5528    1.0391   -1.1176]';
lambda = 0.5;
x1=hardthresholding(b,lambda)
x2=hard(b,sqrt(lambda))
fprintf('\nError between hardthresholding and hard = %f\n',norm(x1-x2))

Copy

The output results are not given here . You can run it , Look at the output , The function is correct .

in addition , Can be in matlab Enter the following command to see a soft threshold image ：

x=-5:0.01:5;T=1;y=hard(x,T);plot(x,y);grid;

Copy

6、 Conclusion

Finally understand the hard threshold and soft threshold , In the literature 【3】 Finally, the author mentioned “ Ah , Bad math kills people , When can we reach the height of the Bulls ”, I believe many people will feel the same . But then I thought , It is impossible for us to analyze matrix 、 numerical analysis 、 Functional analysis 、 optimization 、 Combinatorics, etc （ There are so many thoughts in my mind ） Do research after learning all the basic mathematics courses , Study while studying , Where can't be filled is the most normal mode ……

Besides , If you simply learn the basics of Mathematics , You may feel very bored , You may often complain ： What's the use of learning these boring Mathematics ？

Let's move on , Want to believe in yourself , The road will be wider and wider ……

7、 reference

【1】Donoho D L, JohnstoneJ M. Ideal spatial adaptation by wavelet shrinkage[J]. Biometrika, 1994, 81(3):425-455.

【2】Wright SJ, Nowak R D, Figueiredo M A T. Sparse reconstruction by separableapproximation[J]. IEEE Transactions on Signal Processing, 2009, 57(7):2479-2493.

【3】http://blog.sina.com.cn/s/blog_6d0e97bb01015vq3.html

【4】Elad M,Figueiredo M A T, Ma Y. On the Role of Sparse and Redundant Representations inImage Processing[J]. Proceedings of the IEEE, 2010, 98(6):972-982.

【5】http://www.docin.com/p-553314466.html

Publisher ： Full stack programmer stack length , Reprint please indicate the source ：https://javaforall.cn/131512.html Link to the original text ：https://javaforall.cn