当前位置:网站首页>Insight -- the application of sanet in arbitrary style transfer
Insight -- the application of sanet in arbitrary style transfer
2020-11-08 07:19:00 【Artificial intelligence meets pioneer】
author |dhwani mehta compile |Flin source |medium
Image stylization is an image processing technology studied in recent decades , This paper aims to demonstrate an efficient and novel style attention network (SANet) Method , While balancing the global and local style patterns , Keep the content structure , Synthesize high quality stylized images .
An overview of style transfer mechanism
Have you ever imagined that if you had a great artist making photos , What will the picture look like ? Arbitrary style migration through the content image ( Target image ) With style images ( Its texture is brush stroke , Angle Geometry , pattern , Images that need to be drawn to the content image, such as color transitions ) blend , And turn it into reality , To create a third image you've never seen before .
Novel SANet Style transfer method
The ultimate goal of arbitrary style transfer is to achieve generality , And maintain quality and efficiency .
Balance global and local style patterns and retain content structure for the following reasons :
-
Use the similarity kernel of learning instead of the fixed kernel
-
Use soft attention based web instead of hard attention for style decoration
-
Avoid losing features during training , To maintain content structure without losing the richness of style
Use SANet Building blocks for arbitrary style migration
The whole mechanism of style transfer can be summarized as follows :
Let's step through the architecture , Finally, get a comprehensive overview .
comprehensive SANet framework
Let's try to unravel the whole architecture , To better understand :
-
Encoder decoder module
-
Style attention module
-
Calculation of loss function
Encoder - Decoder module
The most important step to solve the style migration problem is encoder - Decoder mechanism . In the process of the training VGG-19 The network encodes an image , Form a representation , And pass it to the decoder , The decoder attempts to reconstruct the original input image back to .
Style attention module
SANet The architecture will come from VGG-19 The content and style of the encoder are input as feature maps , And standardize it , Convert to feature space , To calculate the attention between content and style feature map .
Calculation of loss function
In the process of the training VGG-19 Used to calculate the loss function , In order to train the decoder in the following way :
Complete loss calculation formula
An idea for calculating the loss of content and style :
SANet An overview of the calculation of content and style loss components in
Calculation of characteristic loss
Loss of function due to novel features ,SANet Architecture can preserve the content structure and enrich the style patterns .
SANet An overview of the calculation of characteristic loss in
Calculate the loss of the same input image without any style blank , It makes the feature loss and realizes the maintenance of content structure and style features at the same time .
Conclusion and result
The experiment clearly shows that , Use SANet The results of style transfer will analyze various styles ,
For example, global color distribution , Texture and local style , While maintaining the structure of the content . Again ,SANet It is also useful in distinguishing between the content structure and the migration style corresponding to each semantic content . So it can be inferred that ,SANet Not only is it effective in maintaining the structure of the content , And it's also very effective in retaining style and structural features , And it's easy to integrate style features , So as to enrich the global style and local style statistical information .
reference
[1] Park Daying and Lee Kwong hee .“ Any style migration through a style focused network .” IEEE Proceedings of the conference on computer vision and pattern recognition .2019.
[2] Gatys,Leon A.,Alexander S. Ecker and Matthias Bethge.“ Using convolutional neural network to transfer image style .” IEEE Conference on computer vision and pattern recognition .2016.
[3] Huang,Xun and Serge Belongie.“ Real time arbitrary style migration through adaptive instance Standardization .” IEEE Proceedings of the International Conference on computer vision .2017.
[4] Li Yijun , etc. .“ General style transfer is realized by feature transformation .” Research progress of neural information processing system .2017.
[5] Shenglu , etc. .“ Head picture network : Multi scale zero shot style transfer through feature decoration .” IEEE Proceedings of the conference on computer vision and pattern recognition .2018.
Link to the original text :https://medium.com/visionwizard/insight-on-style-attentional-networks-for-arbitrary-style-transfer-ade42e551dce
Welcome to join us AI Blog station : http://panchuang.net/
sklearn Machine learning Chinese official documents : http://sklearn123.com/
Welcome to pay attention to pan Chuang blog resource summary station : http://docs.panchuang.net/
版权声明
本文为[Artificial intelligence meets pioneer]所创,转载请带上原文链接,感谢
边栏推荐
猜你喜欢
Sum up some useful functions
sed之查找替换
NOIP 2012 提高组 复赛 第一天 第二题 国王游戏 game 数学推导 AC代码(高精度 低精度 乘 除 比较)+60代码(long long)+20分代码(全排列+深搜dfs)
数据科学面试应关注的6个要点
PCR and PTS calculation and inverse operation in TS stream
Hand tearing algorithm - handwritten singleton mode
laravel8更新之速率限制改进
Wanxin Finance
LadonGo开源全平台渗透扫描器框架
Littlest JupyterHub| 02 使用nbgitpuller分发共享文件
随机推荐
【原创】关于高版本poi autoSizeColumn方法异常的情况
Littlest JupyterHub| 02 使用nbgitpuller分发共享文件
Tail delivery
Supervisor process management installation and use
Summary of knowledge points of Jingtao project
FORTRAN 77 reads some data from the file and uses the heron iteration formula to solve the problem
Qt混合Python开发技术:Python介绍、混合过程和Demo
0.计算机简史
Wechat nickname Emoji expression, special expression causes the list not to be displayed, export excel error report and other problems solved!
A compilation bug brought by vs2015 Update1 update [existing solutions]
GET,POST,PUT,DELETE,OPTIONS用法与说明
nvm
About the promotion of the whole stack of engineers, from the introduction to give up the secret arts, do not click in to have a look?
See once to understand, graphic single chain table inversion
Adobe media encoder / me 2021 software installation package (with installation tutorial)
These core technology of object-oriented, after you master it, you can have a good interview
CPP (1) installation of cmake
Adobe Lightroom / LR 2021 software installation package (with installation tutorial)
C / C + + Programming Notes: what are the advantages of C compared with other programming languages?
异常+abstract