Paper reading (62):pointer networks

1 introduce

1.1 subject

  2015 Pointer network (Pointer networks)

1.2 Code

  Github：https://github.com/shirgur/PointerNet

1.3 Abstract

   A new neural architecture is introduced to Learn the conditional probability of the output sequence , Its elements are discrete markers corresponding to positions in the input sequence . Some existing methods , For example, sequence to sequence and neural Turing machines cannot deal with this problem , Because in every step of their output , The number of target classes depends on the length of the input , But the length of the input sequence is variable . Sorting of variable size sequences , And all kinds of combinatorial optimization problems belong to this kind of problem .
   In particular , Variable size output dictionary By introducing Attention mechanism To solve . It differs from previous attention in , It does not use attention to fuse the hidden units of the encoder into context vectors in each decoder step , It is Use attention as a pointer to select the members of the input sequence as the output . Such an architecture is called Pointer network (Ptr-Net).
   In order to prove Ptr-Net The effectiveness of the , Three famous geometric problems are used , For finding plane convex hull 、 Calculation Delaunay Triangulation and plane traveling salesman problem .Ptr-Net It not only improves sequence to sequence by inputting attention , And it can also be extended to variable size output dictionaries . The final results show that the generalization ability of the learning models exceeds the maximum length of their training . These results will encourage neural learning to explore a wider range of discrete problems .

1. Bib

@article{
    Vinyals:2015:pointer,
author		=	{
    Oriol Vinyals and Meire Fortunato and Navdeep Jaitly},
title		=	{
    Pointer networks},
journal		=	{
    Advances in neural information processing systems},
volume		=	{
    28},
year		=	{
    2015}
}

2 Model

   First of all, two background works are introduced , That is, sequence to sequence and input attention model . Then it leads to the Ptr-Net framework .

2.1 Sequence to sequence model

   Given a training pair $(\mathcal{P},{\mathcal{C}}^{\mathcal{P}})$, Sequence to sequence The model uses parameters $\theta$ Of RNN Calculate the conditional probability based on the probability chain rule ：
$$p({\mathcal{C}}^{\mathcal{P}}|\mathcal{P};\theta )=\prod _{i=1}^{m(\mathcal{P})}p(C_i|C_1,\dots ,C_{i-1},\mathcal{P};\theta )\text{\hspace{1em}(1)}$$ A schematic is as follows chart 1.

chart 1：