Detailed explanation of extended physics informedneural networks paper

author

• Ameya D. Jagtap1,∗ and George Em Karniadakis1,2

Periodical

• Communications in Computational Physics

date

• 2020

Code

• Code link

1 Abstract

Propose a more flexible decomposition domain XPINN Method , Than cPINN Domain decomposition is more flexible , And use with all equations .

2 background

cPINN Through domain decomposition , Each area uses a small network for training , So that different regions can be calculated in parallel . Proposed by the paper XPINN have cPINN The advantages of domain decomposition , There are also the following advantages

• Generalized space-time domain decomposition,XPINN The formula provides highly irregular 、 Convex / Non convex space-time domain decomposition , Because of this decomposition XPINN The formula provides highly irregular 、 Convex / Non convex space-time domain decomposition
• XPINN The formula provides highly irregular 、 Convex / Non convex space-time domain decomposition
• Simple intermediate condition , stay XPINN in , For interfaces of any shape , The interface condition is very simple , Normal direction is not required , therefore , The proposed method can be easily extended to any complex geometry , Even higher dimensional geometry .

Accurately solve complex equations , In particular, high-dimensional equations have become one of the biggest challenges of Scientific Computing .XPINN Its advantages make it a candidate for such high-dimensional complex simulation , This high-dimensional simulation usually requires a lot of training costs .

3 XPINN Method

describe ：

• Subdomains ： Subdomain ${\Omega }_q,q=1,2,\cdots N_{sd}$ Is the entire computing domain $\Omega$ Non overlapping subdomains of , Satisfy $\Omega ={\bigcup }_{q=1}^{N_{sd}}{\Omega }_q$ and ${\Omega }_i\cap {\Omega }_j=\partial {\Omega }_{ij},i\ne j$ Represents the number of decomposition fields , The intersection of subdomains is only at the boundary $\partial {\Omega }_{ij}$
• Interface ： Represents the subnet corresponding to the common boundary of two or more subdomains （sub-Nets） To communicate with each other
• sub-Net： Son PINN It refers to the individuals with their own set of optimization super parameters used in each sub domain PINN
• Interface Conditions: These conditions are used to connect the decomposed subdomains , Thus, the solution of the governing partial differential equation on the complete field is obtained , According to the properties of the governing equation , One or more interface conditions can be applied to a common interface , Such as solution continuity 、 Flux continuity, etc

Above picture X Is the solution domain , The black solid line indicates the boundary of the area , The black dotted line indicates interface.XPINN Basic interface Conditions include strong form continuity conditions and in common interface Force the average solution given by different subnets .cPINN Mentioned in the text , For stability , There is no need to add the condition of average solution , But experiments also show that it will accelerate the convergence speed .XPINN have cPINN All the advantages of , Such as parallelization ability 、 Great presentation ability 、 An optimization method 、 Activation function 、 Efficient selection of super parameters such as network depth or width . And cPINN Different ,XPINN It can be used to solve any type of partial differential equation , Not necessarily the law of conservation . stay XPINN Under the circumstances , It is not necessary to find the normal direction to adopt the normal flux continuity condition . This greatly reduces the complexity of the algorithm , Especially in large-scale problems with complex fields and mobile interface problems .

The first $q^{th}$ The neural network output of the subdomain is defined as
$u_{{\tilde{\mathbf{\Theta }}}_q}(\mathbf{z})={\mathcal{N}}^L\left(\mathbf{z};{\tilde{\mathbf{\Theta }}}_q\right)\in {\Omega }_q,q=1,2,\cdots ,N_{sd}$
The final solution is defined as
$u_{\tilde{\mathbf{\Theta }}}(\mathbf{z})={\sum }_{q=1}^{N_{sd}}{u}_{{\tilde{\mathbf{\Theta }}}_q}(\mathbf{z})\cdot 1_{{\Omega }_q}(\mathbf{z})$
among
$1_{{\Omega }_q}(\mathbf{z}):=\{\begin{array}{ll}0& \text{ if }\mathbf{z}\notin {\Omega }_q\\ 1& \text{ if }\mathbf{z}\in {\Omega }_q\backslash \text{ Common interface in the }{q}^{th}\text{ subdomain }\\ \frac{1}{\mathcal{S}}& \text{ if }\mathbf{z}\in \text{ Common interface in the }{q}^{th}\text{ subdomain }\end{array}$
$S$ Express S Indicates the number of subdomains that intersect along the public interface

3.1 just 、 The loss function of the subdomain of the inverse problem

(1) Positive problem
stay $q^{th}$ Subdomain ${\left\{{\mathbf{x}}_{u_q}^{(i)}\right\}}_{i=1}^{N_{uq}},{\left\{{\mathbf{x}}_{F_q}^{(i)}\right\}}_{i=1}^{N_{Fq}}\text{ and }{\left\{{\mathbf{x}}_{I_q}^{(i)}\right\}}_{i=1}^{N_{Iq}}$ Express training, residual, and the common interface points.$N_{u_q},N_{F_q}and{N}_{Iq}$ Respectively represent the number of corresponding points , Each subdomain uses one PINN,$u_q=u_{{\tilde{\Theta }}_t}$, The first $q^{th}$ The sub domain loss function is defined as
$\begin{array}{rl}\mathcal{J}\left({\tilde{\mathbf{\Theta }}}_q\right)=& W_{u_q}{\mathrm{MSE}}_{u_q}\left({\tilde{\mathbf{\Theta }}}_q;{\left\{{\mathbf{x}}_{u_q}^{(i)}\right\}}_{i=1}^{N_{uq}}\right)+W_{{\mathcal{F}}_q}{\mathrm{MSE}}_{{\mathcal{F}}_q}\left({\tilde{\mathbf{\Theta }}}_q;{\left\{{\mathbf{x}}_{F_q}^{(i)}\right\}}_{i=1}^{N_{Fq}}\right)\\ & +W_{I_q}\underset{\text{Interface condition }}{\underbrace{{\mathrm{MSE}}_{u_{avg}}\left({\tilde{\mathbf{\Theta }}}_q;{\left\{{\mathbf{x}}_{I_q}^{(i)}\right\}}_{i=1}^{N_{Iq}}\right)}}+W_{I_{{\mathcal{F}}_q}}\underset{\text{Interface condition }}{\underbrace{{\mathrm{MSE}}_{\mathcal{R}}\left({\tilde{\mathbf{\Theta }}}_q;{\left\{{\mathbf{x}}_{I_q}^{(i)}\right\}}_{i=1}^{N_{Iq}}\right)}}\\ & +\underset{\text{Optional }}{\underbrace{\text{ Additional Interface Condition’s }}}\end{array}$
$W_{u_q},W_{{\mathcal{F}}_q},W_{I_{{\mathcal{F}}_q}}\text{ and }{W}_{I_q}$ Parameters representing different losses ,
$\begin{array}{l}{\mathrm{MSE}}_{u_q}\left({\tilde{\mathbf{\Theta }}}_q;{\left\{{\mathbf{x}}_{u_q}^{(i)}\right\}}_{i=1}^{N_{uq}}\right)=\frac{1}{N_{u_q}}{\sum }_{i=1}^{N_{uq}}{\left|u^{(i)}-u_{{\tilde{\mathbf{\Theta }}}_q}\left({\mathbf{x}}_{u_q}^{(i)}\right)\right|}^2\\ {\mathrm{MSE}}_{{\mathcal{F}}_q}\left({\tilde{\mathbf{\Theta }}}_q;{\left\{{\mathbf{x}}_{F_q}^{(i)}\right\}}_{i=1}^{N_{Fq}}\right)=\frac{1}{N_{F_a}}{\sum }_{i=1}^{N_{Fq}}{\left|{\mathcal{F}}_{{\tilde{\mathbf{\Theta }}}_q}\left({\mathbf{x}}_{F_q}^{(i)}\right)\right|}^2\end{array}$
$\begin{array}{l}{\mathrm{MSE}}_{u_{avg}}\left({\tilde{\mathbf{\Theta }}}_q;{\left\{{\mathbf{x}}_{I_q}^{(i)}\right\}}_{i=1}^{N_{Iq}}\right)={\sum }_{\forall q^{+}}\left(\frac{1}{N_{I_q}}{\sum }_{i=1}^{N_{I_q}}{\left|u_{{\tilde{\mathbf{\Theta }}}_q}\left({\mathbf{x}}_{I_q}^{(i)}\right)-\left\{\left\{u_{{\tilde{\mathbf{\Theta }}}_q}\left({\mathbf{x}}_{I_q}^{(i)}\right)\right\}\right\}\right|}^2\right)\\ {\mathrm{MSE}}_{\mathcal{R}}\left({\tilde{\mathbf{\Theta }}}_q;{\left\{{\mathbf{x}}_{I_q}^{(i)}\right\}}_{i=1}^{N_{Iq}}\right)={\sum }_{\forall q^{+}}\left(\frac{1}{N_{I_q}}{\sum }_{i=1}^{N_{I_q}}{\left|{\mathcal{F}}_{{\tilde{\mathbf{\Theta }}}_q}\left({\mathbf{x}}_{I_q}^{(i)}\right)-{\mathcal{F}}_{{\tilde{\Theta }}_{q^{+}}}\left({\mathbf{x}}_{I_q}^{(i)}\right)\right|}^2\right)\end{array}$
The last two represent interface Condition loss , The fourth item is in the subdomain $q$ and $q^{+}$ The residual continuity condition of two different networks ,$q^{+}$ representative $q$ The field of MSER and $MS{E}_{uavg}$, Are defined in all adjacent sub domains , In the above formula $\left\{\left\{u_{{\tilde{\mathbf{\Theta }}}_q}\right\}\right\}=u_{\text{avg }}:=\frac{u_{{\tilde{\mathbf{\Theta }}}_q}+u_{{\tilde{\mathbf{\Theta }}}_{q^{+}}}}{2}$( Suppose only two subdomains intersect on the public interface ),additional interface conditions, for example flux continuity ,$c^k$ It can also be based on PDE And the type of interface The direction is added to the loss .
remark：

• interface conditions The type of determines the regularity of the solution of the whole interface , Thus affecting the convergence speed . stay interface The solution on is sufficiently continuous , So as to meet its control PDE
• Enough interface point To connect subdomains , This is very important for the convergence of the algorithm , Especially for internal

For the inverse problem :
$\begin{array}{rl}\mathcal{J}\left({\tilde{\mathbf{\Theta }}}_q,\lambda \right)=& W_{u_q}{\mathrm{MSE}}_{u_q}\left({\tilde{\mathbf{\Theta }}}_q,\lambda ;{\left\{{\mathbf{x}}_{u_q}^{(i)}\right\}}_{i=1}^{N_{u_q}}\right)+W_{{\mathcal{F}}_q}{\mathrm{MSE}}_{{\mathcal{F}}_q}\left({\tilde{\mathbf{\Theta }}}_q,\lambda ;{\left\{{\mathbf{x}}_{u_q}^{(i)}\right\}}_{i=1}^{N_{u_q}}\right)\\ & +W_{I_q}\underset{\text{Interface condition’s }}{\underbrace{\left\{{\mathrm{MSE}}_{u_{avg}}\left({\tilde{\mathbf{\Theta }}}_q,\lambda ;{\left\{{\mathbf{x}}_{I_q}^{(i)}\right\}}_{i=1}^{N_{Iq}}\right)+{\mathrm{MSE}}_{\lambda }\left({\tilde{\mathbf{\theta }}}_q,\lambda ;{\left\{{\mathbf{x}}_{I_q}^{(i)}\right\}}_{i=1}^{N_{Iq}}\right)\right\}}}\\ & +W_{I_{{\mathcal{F}}_q}}\underset{\text{Intarf }}{\underbrace{{\mathrm{MSE}}_{\mathcal{R}}\left({\tilde{\mathbf{\Theta }}}_q,\lambda ;{\left\{{\mathbf{x}}_{I_q}^{(i)}\right\}}_{i=1}^{N_{Iq}}\right)}}+\underset{\text{Optional }}{\underbrace{\text{ Additional Interface Condition’s }}}\end{array}$
among
$\begin{array}{l}{\mathrm{MSE}}_{{\mathcal{F}}_q}\left({\tilde{\mathbf{\Theta }}}_q,\lambda ;{\left\{{\mathbf{x}}_{u_q}^{(i)}\right\}}_{i=1}^{N_{u_q}}\right)=\frac{1}{N_{u_q}}{\sum }_{i=1}^{N_{u_q}}{\left|{\mathcal{F}}_{{\tilde{\mathbf{\Theta }}}_q}\left({\mathbf{x}}_{u_q}^{(i)}\right)\right|}^2\\ {\mathrm{MSE}}_{\lambda }\left({\tilde{\mathbf{\Theta }}}_q,\lambda ;{\left\{{\mathbf{x}}_{I_q}^{(i)}\right\}}_{i=1}^{N_{Iq}}\right)={\sum }_{\forall q^{+}}\left(\frac{1}{N_{I_q}}{\sum }_{i=1}^{N_{lq}}{\left|{\lambda }_q\left({\mathbf{x}}_{I_q}^{(i)}\right)-{\lambda }_{q^{+}}\left({\mathbf{x}}_{I_q}^{(i)}\right)\right|}^2\right)\end{array}$
Other residual losses are the same as positive losses .
**Remark：** It should be noted that , because XPINN Highly nonconvex loss function , It is very difficult to locate its global minimum . however , For several local minima , The value of the loss function is similar , The accuracy of the corresponding prediction solution is similar .

3.2 An optimization method

Automatic derivation

3.3 error

$\begin{array}{rl}{\mathcal{E}}_{\text{app }}q& =\Vert u_{a_q}-u_q^{ex}\Vert \\ {\mathcal{E}}_{\text{gen }}q& =\Vert u_{g_q}-u_{a_q}\Vert \\ {\mathcal{E}}_{\text{opt }}q& =\Vert u_{{\tau }_q}-u_{g_q}\Vert \end{array}$
Represent the approximation error、 generalization error as well as optimization error.

• $u_{a_q}=\arg {\min }_{f\in F_q}\Vert f-u_q^{ex}\Vert$ True solution $u_q^{ex}$ Approximation of
• $u_{g_q}=\arg {\min }_{{\tilde{\mathbf{\Theta }}}_q}\mathcal{J}\left({\tilde{\mathbf{\Theta }}}_q\right)$ Is the global optimal solution
• $u_{{\tau }_q}=\arg {\min }_{{\tilde{\mathbf{\Theta }}}_q}\mathcal{J}\left({\tilde{\mathbf{\Theta }}}_q\right)$ It is the solution obtained after the subnetwork training ,

Last XPINN The error of can be summarized as
${\mathcal{E}}_{XPINN}:=\Vert u_{\tau }-u^{ex}\Vert \le \Vert u_{\tau }-u_g\Vert +\Vert u_g-u_a\Vert +\Vert u_a-u^{ex}\Vert$
among ,$\left(u^{ex},u_{\tau },u_g,u_a\right)(\mathbf{z})={\sum }_{q=1}^{N_{sd}}\left(u_q^{ex},u_{{\tau }_q},u_{g_q},u_{a_q}\right)(\mathbf{z})\cdot 1_{{\Omega }_q}(\mathbf{z})$

Remark：

• When the estimation error decreases （ Data fitting is better ）, Generalization error will increase , This is a kind of bias variance trade-off, The two main factors affecting generalization error are the number and distribution of residual points
• The optimization error is affected by the complexity of the loss function , The network structure deeply affects the optimization error

3.4 XPINN、cPINN,PINN contrast

And PINN and cPINN Compared to the framework ,XPINN Framework has many advantages , But it also has the same limitations as the previous framework . Absolute error
PDE Solution , Not lower than the level , This is due to the inaccuracy involved in solving high-dimensional nonconvex optimization problems , May lead to a bad minimum