A verification test of the relationship between iteration number and entropy

( A , B)---81*30*2---(1,0)(0,1)

Classify with neural network A and B, Give Way A yes mnist Of 0, Give Way B yes mnist Of 1-9. Reduce the picture to 9*9. But let B Central African 0 The number of values is equal to 14,14*0.9,14*0.8,…,14*0.2 individual . And all non 0 Value change 1. Compare the effect of binarization on the number of iterations with the previous experimental data .

Get the experimental data

	1		2		3		4		5
	The number of iterations	The number of iterations	The number of iterations	The number of iterations	The number of iterations	The number of iterations	The number of iterations	The number of iterations	The number of iterations	The number of iterations
	Two valued	normal	Two valued	normal	Two valued	normal	Two valued	normal	Two valued	normal
14	6309.538	8028.492	7593.618	7170.111	8303.518	4563.92	11198.27	6931.015	10258.35	3712.191
14*0.9	8235.688	10512.76	7083.558	7085.307	8344	3957.085	10900.14	5011.256	10004.79	4287.045
14*0.8	10822.66	12175.1	8527.307	8909.668	9144.945	5329.327	12193.12	5052.623	18225.24	3973.759
14*0.7	11132.49	14806.39	10588.36	9178.673	12194.42	3901.99	12874.62	4177.176	16052.84	3760.663
14*0.6	16670.89	18403.16	13511.03	12387.02	19269.12	4450.814	16583.43	4873.387	22452.03	3520
14*0.5	18838.25	18569.19	15671.86	16936.02	31967.7	5504	21907.32	4610	31802.59	7360.563
14*0.4	28745.89	32946.66	25975.92	21557.49	35678.28	8226.271	32413.65	6770	50814.83	8412
14*0.3	49456.88	77674.51	35543.14	32116.68	65702.13	10478.2	78774.95	8889.276	83772.57	10278.59
14*0.2	96061.47	104475.7	67555.34	59634.69	126102.6	15644.66	146888.1	15570.22	109398.6	18435.36
	6		7		8		9
	The number of iterations	The number of iterations	The number of iterations	The number of iterations	The number of iterations	The number of iterations	The number of iterations	The number of iterations
	Two valued	normal	Two valued	normal	Two valued	normal	Two valued	normal
14	10871.9	5297.548	12749.89	7865.568	14637.45	4449.216	10570.74	7524.784
14*0.9	14806.03	4568.633	10504.46	5539.92	15307.52	3349.618	11783.41	5888.221
14*0.8	15222.84	3833.618	10310.31	4698.231	16718	3198.503	17794.56	4814.271
14*0.7	20703.42	3874.573	10117.7	5672.623	18981.44	3730	16757.24	5220.161
14*0.6	30274.27	4857.91	10674.23	4257.085	25441.39	4653.035	21874.54	4988.834
14*0.5	38780.17	5293.357	15816.5	5994.452	37159.89	5058.241	30613.85	7303.035
14*0.4	44346.16	6036	23108.11	8845.538	42765.22	6021.759	58870.69	7886.734
14*0.3	83847.74	9077.477	49979.41	15703.26	76483.45	9730.201	90765.41	10279.68
14*0.2	154253.7	16665.61	90731.36	28967.59	121397.9	15053.71	149371	17749.77

For example 1 Group data

14*0.6

16670.89

18403.16

It shows that the convergence error 1e-5 Under the condition of , Use the method of taking points at intervals to 0 and 1 All reduced to 9*9, form 1 Random reservation is about 8.4 A valid value , convergence 199 The average of times is 16670.89. And if you put this 8.4 A random non 0 Value binarization , The average number of iterations is 18403.16. Because two statistics are not 0 The number and distribution of values are consistent , Therefore, the difference in the number of iterations depends only on the difference in numerical values .

Plot the data

be-all 9 In group data 1,2 The shape of is very consistent , There is little difference in the number of iterations between the two groups .

But the rest 7 Group data , The number of iterations of binarization is greater than that of non binarization , The difference is very obvious .

After binarization, the discrimination between data is reduced , Carry less information , Therefore, binarization reduces the entropy between pictures . The number of iterations of binarization is greater than that of non binarization , So the blue line is bigger than the red line . The assumption that the number of iterations is inversely proportional to entropy can well explain this 7 Group data .

0	1	2	3	4	5	6	7	8	9
44.4698	27.7134	41.7294	17.9278	15.9034	16.9892	17.8176	14.0056	19.4652	15.9438

form 1 Non - 0 Value has 27 individual , form 2 Yes 41 individual , and 3-9 Only about 16 about . form 1 And form 2 There are more valid values for , Perhaps the convergence error is too large to reflect the impact of binarization .

/***/ Today's foundation

  Hermite manifold

Whether a metric is Keller , And when moving in space , How the metric changes . Keller manifolds are a group called “ Hermite manifold ”( Hermitian manifold) Subclasses of complex manifolds of . On Hermite manifolds , You can put the origin of complex coordinates at any point , Its metric at this point looks like a standard Euclidean geometric metric . But when you leave that point , Its metric is less and less like Euclidean . More specifically , When the distance from the origin is ε when , The difference in the change of the metric coefficient itself is roughly ε times . We call such a manifold “ First order Euclidean space ”.
So if ε yes 0.001 Inch (1 Inch =2.54 centimeter ), When we leave ε Distance time , The difference between the Hermite metric coefficient and the original one will be maintained at about 0.001 Within inches . The Keller manifold is “ Second order Euclidean space ”, This means that its metric will be more stable ; When the distance from the origin is E when , The change of metric coefficient of Keller manifold is approximately ε^2 times .
    
All Hermite manifolds have this symmetry :J Transformation rotates the vector 90 degree , But keep its length unchanged . Since Keller manifold is a Hermite manifold , Of course, it also has this symmetry . besides , Keller manifold also has a “ Internal symmetry ( internal symmetry), This is when you move two points in space , Still need to maintain the same subtle symmetry , Otherwise it is not a Keller manifold .