Bayesian network explanation

0x01 Overview of Bayesian networks

Sometimes we need to calculate the probability of an uncertain cause and give some observed evidence , In these cases, Bayesian methods can be used .
Bayesian network （bayesian network）
Is a probabilistic graphical model , It explicitly captures the known conditional dependencies of directed edges in the graphical model , It passes through a directed acyclic graph (DAG) Represents a set of variables and their conditional dependencies .

Bayesian networks are well suited for capturing events that have occurred and predicting any of several possible known causes . for example , Bayesian networks can represent the probabilistic relationship between diseases and symptoms . Given symptoms , The network can be used to calculate the probability of the existence of various diseases .

Efficient algorithms can be used for reasoning and learning in Bayesian networks .
Sequence of variables （ for example Speech signal or protein sequence ） The Bayesian network for modeling is called dynamic Bayesian network , The generalization of Bayesian networks that can represent and solve decision problems under uncertainty is called influence graph .

Unlike manual builds , Automatic learning does not require expert knowledge of the underlying domain . Bayesian networks can be automatically learned directly from databases using experience based algorithms that are usually built into appropriate software . But the disadvantage is that the automation construction requires high data .

0x02 Bayesian network model

stay 0x01 Already mentioned in , Bayesian networks are directed acyclic graphs (DAG), Its nodes represent variables in the Bayesian sense .

• There are many types of variables , There are observable quantities 、 Potential variables 、 Unknown parameters or assumptions .
• Use edges to represent conditional dependencies ; Unconnected nodes represent variables that are conditionally independent of each other .

Each node is associated with a probability function , The probability function takes a set of specific values of the node's parent variables as input , The probability of variables represented by nodes is given .

Simplest graph , It can be expressed in this way ：

Wiki gives an example

There are two ways to make grass wet ： Watering car & It's raining .
Rain will affect the travel of sprinkler , There are two results when the grass becomes wet ：T（ It means true ） and F（ On behalf of false ）.

Chain rule of probability for joint probability function ：

• G =“ Grass becomes wet （ really / false ）”
• S =“ Sprinkler on （ really / false ）”
• R =“ It's raining （ really / false ）”

The model can answer questions about the existence effect （ The so-called inverse probability ） Whether there is a problem of cause .
By using the conditional probability formula and summing all the variables ：

Use the expansion of the joint probability function Pr(G,S,R)} And the conditional probability table described in the figure (CPT) Conditional probabilities in , Each term in the numerator and denominator can be evaluated .

Then the numerical results （ Subscript by the value of the relevant variable ） yes

Infer unobserved variables

Bayesian networks are often used to answer probabilistic queries about them , When there are other variables , The network can be used to update the knowledge of variable subset states . This process of calculating the posterior distribution is called probabilistic reasoning . When selecting values for a subset of variables , Some expected loss functions can be minimized , For example, the probability of wrong decision . So Bayesian network can be considered as a complicated problem with the mechanism of automatically applying Bayesian theorem .

Parameter learning

Specified in X The parent node of is conditional X Probability distribution of ,X The distribution conditional on the parent can take any form . Discrete or Gaussian distributions are usually used , Because it simplifies the calculation . Sometimes you only know the limits of distribution ; The maximum entropy principle can then be used to determine the individual distributions , That is, the distribution with maximum entropy under given constraints .

Structural learning

The model of Bayesian network can be divided into several forms ：
Direct connection ： Indicates that the direction of a directed graph is in a straight line , And point in the same direction .
Expressed as ：G -> S -> R

Huilian ： Two independent nodes point to the same node .
Expressed as ：G -> S <- R

Successive ： One node points to two different nodes .
Expressed as ：G <- S -> R

A particularly fast and accurate BN The learning method is to transform the problem into an optimization problem , And use integer programming to solve . In the solution process, the acyclic constraint is added to the integer program in the form of cutting plane (IP) in , This method can handle up to 100 A variable problem .

Software

Useful software for processing graphical models , There are several options . The most common software packages are Genie、Hugin、BUGS etc. .