当前位置:网站首页>20. Support vector machine - knowledge of mathematical principles
20. Support vector machine - knowledge of mathematical principles
2022-07-31 21:12:00 【WuJiaYFN】
Main content
- Inner product of vectors
- Using the properties of vector inner product to understand SVM
- SVM's method for selecting better decision boundaries
First, the inner product of vectors
1.1 Definition and geometric meaning of inner product
- If there are two vectors u and v , u^Tv is called the inner product between the vectors u and v
- Geometric meaning: The inner product of vectors is equivalent to the product of projection lengths
1.2 Euclidean length (norm)
If there is a vector u, ∥u∥ represents the norm norm of u, that is, the Euclidean length of the vector u, which is a real number
According to the Pythagorean theorem, the norm is calculated as follows:
1.3 Two ways to calculate inner product
(1) u^Tv = u1 × v1 + u2 × v2 = v^Tu
(2) First, project v to the u vector, and record its length as p (positive and negative, positive in the same direction as u, negative in the opposite direction, scalar), then the inner product of the two vectors:
u^Tv = ||u|| · ||v|| · cosθ = ||u|| · p
Note: If the angle between the two vectors is greater than 90°, then p is negative, and the inner product of the two vectors is also negative
Second, use the properties of vector inner product to understand SVM
If C is set very large and A is minimized to 0, then the cost function of SVM will be simplified as shown in the following figure:
For ease of understanding, let's simplify the function expression: let θ0 = 0, and then only have two parameters, θ1 and θ2
- **What the support vector machine does is: **minimize the square of the parameter vector norm, or the square of the length
According to the inner product calculation formula, there is θ^Tx = p · ||θ||, where p is the projection of x on θ.Use p^(i) ⋅ ∥θ∥ instead of θTx(i)
Three, SVM method to select a better decision boundary
We assume that the decision boundary is shown as the green line in the left figure above, and we can know that the parameter vector θ is perpendicular to the boundary (the proof process can be found in another blog post of mine)
It is found that for each sample x(1) and x(2), their projection lengths on θ are very small, then in order to satisfy the condition p(i)·||θ|| ≥1 or p(i)·||θ||≤-1, then ||θ|| should take a large value, which contradicts the previous minimization of the cost function (1/2||θ||2);
The support vector machine tries to make p(i) (the distance from the training sample to the decision boundary) large enough to make the norm of θ small (such as the decision boundary in the right figure above - green line)) to minimize the cost function;
This is how SVM produces the phenomenon of large-spacing classification;Simplifying by letting θ0 = 0 means that we let the decision circle pass through the origin. If θ0 ≠ 0, the decision boundary is no more than the origin , and the conclusion that SVM produces a classifier with a large gap is also true (in the case of a particularly large C).
If you think the article is helpful to you, you can give me a thumbs up and encourage me. Welcome friends to collect articles and study
Follow me, let's learn together and make progress together!!!
边栏推荐
- Linux环境redis集群搭建「建议收藏」
- ResNet的基础:残差块的原理
- 【愚公系列】2022年07月 Go教学课程 025-递归函数
- uni-app中的renderjs使用
- Three. Introduction to js
- How programmers learn open source projects, this article tells you
- 统计UTF-8字符串中的字符函数
- Linux environment redis cluster to build "recommended collection"
- The principle of ReentrantLock (to be continued)
- sqlite3 simple operation
猜你喜欢
随机推荐
Realization of character makeup
Redis综述篇:与面试官彻夜长谈Redis缓存、持久化、淘汰机制、哨兵、集群底层原理!...
Basic Grammar Introduction of Carbon Tutorial (Tutorial)
手把手教你学会部署Nestjs项目
Qualcomm cDSP simple programming example (to query Qualcomm cDSP usage, signature), RK3588 npu usage query
leetcode: 6135. The longest ring in the graph [inward base ring tree + longest ring board + timestamp]
leetcode 665. Non-decreasing Array 非递减数列(中等)
架构实战营模块 8 作业
Transfer Learning - Domain Adaptation
Shell 脚本 快速入门到实战 -02
sqlite3简单操作
Memblaze released the first enterprise-grade SSD based on long-lasting particles. What is the new value behind it?
Student management system on the first day: complete login PyQt5 + MySQL5.8 exit the operation logic
【公开课预告】:超分辨率技术在视频画质增强领域的研究与应用
linux view redis version command (linux view mysql version number)
Short-circuit characteristics and protection of SiC MOSFETs
leetcode 665. Non-decreasing Array
Linux environment redis cluster to build "recommended collection"
Tkinter 入门之旅
角色妆容的实现