Johnson–Lindenstrauss lemma

lemma ： Given $ϵ>0$, Random vector module length varies with n Converge exponentially to 1.
$$\begin{gathered} along\; withmachinetowardsThe\; amountx\in R^{n}inOfEvery\; timeindividualsitmarkMiningsamplesinceN(0,\frac{1}{n}) \\ P(|\Vert x{\Vert }^2-1|\ge \epsilon )\le 2\exp \left(-\frac{{\epsilon }^{2}n}{8}\right) \end{gathered}$$
lemma ： Also sample two random vectors , Approximately orthogonal .
$$P(|*x_1,x_2*|\ge \epsilon )\le 4\exp \left(-\frac{{\epsilon }^{2}n}{8}\right)$$

Johnson–Lindenstrauss Lemma

Given $ϵ>0$,$x_i\in R^m(i=1,\dots ,N),Such\; asOnMiningsampleOutOneindividualalong\; withmachineMomentfrontA\in {\mathbb{R}}^{n\times m},n>\frac{24\log N}{{\epsilon }^2}$

$$(1-\epsilon )\Vert v_i-v_j{\Vert }^2\le \Vert A{v}_i-A{v}_j{\Vert }^2\le (1+\epsilon )\Vert v_i-v_j{\Vert }^2$$

application

Cosine theorem

         Calculate the similarity of two sentences , You can use it first TF-IDF Algorithm to generate word frequency vector , Then calculate the cosine angle , The smaller, the more similar .

hash function

         hash function （MD5 etc. ） Turn the article into a fixed length string , such as 32 position . In the front-end encryption, I have implemented the right “123456” The encryption .

simhash

         The traditional hash function cannot compare the similarity between the two articles .simhash technology , It is Google Algorithm invented to solve large-scale web page de duplication . Use 0,1 Represents the final calculation result , XOR operation for comparison .

        Johnson–Lindenstrauss lemma + discretization ： In European Space N A little bit , After the same random projection mapping , They will still maintain their original relative positions . Then discretize the result of random projection （ Less than 90° by 1, Greater than 90° by 0¹ Similar as 1, Otherwise 0）, Convenient for calculation and storage .

         On the basis of the above ,simhash Word segmentation of the article , For every word hash, Yes hash Result weighting , Merge word vectors ( Add in sequence ), The final result is obtained by dimension reduction and other processing . There are pairs. simhash Explanation of algorithm , It seems that the operation of calculation is different .

²:$or-1,senduse-1JustcanNotaketheYestowardsThe\; amountSetinstayOneindividuallikelimit.$

Reference resources
Reference resources
Reference resources

High dimensional random
To what extent has the theory of machine learning progressed ？
Du, S. S., Kakade, S. M., Wang, R., & Yang, L. F. (2019). Is a Good Representation Sufficient for Sample Efficient Reinforcement Learning?
Database-friendly random projections:
Johnson-Lindenstrauss with binary coins
attention Application in
DGBR Algorithm
IJCAI’21 Secure Deep Graph Generation with Link Differential Privacy

$$\begin{gathered} HorseErcanHusbandNoetc.type:P(x\ge a)\le \frac{\mathbb{E}[x]}{a} \\ cutThansnowHusbandNoetc.type:P((x-\mathbb{E}[x]{)}^2\ge a^2)\le \frac{\mathbb{E}[(x-\mathbb{E}[x]{)}^2]}{a^2}=\frac{\mathbb{V}ar[x]}{a^2} \end{gathered}$$
Bernstein inequality

---

1. 1 ︎

2. 1 ︎