MIT-6874-Deep Learning in the Life Sciences Week4

Recurrent Neural Networks (RNNs)

Encoding time

• Autoregressive model
  • Use “delay taps” Predict the next item from a fixed number of previous items .
• Feedforward neural networks
  • The autoregressive model is extended by using one or more nonlinear hidden elements

If we give the generated model some hidden states , If we give this hidden state its own internal dynamics ,

hidden Markov model

RNN

Alternative architectures for RNNs

LSTM

• What the Internet has learned
  • It learns four different activity patterns of three hidden units . These patterns correspond to nodes in finite state automata
  • Cyclic networks can simulate finite state automata , But its function is exponential . Yes N Hidden neurons , It has 2^N Two possible binary activity vectors

Transformer

GNN

Graph neural nets

Learn how to propagate information through graphs to compute node properties

Application to “classical” network problems

Unsupervised learning with GNNs

How should we decode the graph?

Not all graphs are chemically valid