If you want to contribute a Python version of a figure, follow these steps:

• Check the list of open issues for ones with a name like "Convert foo to Python". Such issues have the tag 'Figures'. Each issue has a link to the Matlab file that was used to generate the figure, and a figure number. Figure numbers refer to this version of the book: https://github.com/probml/pml-book/releases/tag/2021-03-08. Consult the pdf file to see what the figure should look like.

• Follow these guidelines for how to contribute code.

• Test your code works in Google colab, and make a pull request. Please include the generated image(s) in your PR, and add a tag which mentions the issue you are solving.

• Here is an example of good PR.

• If you are a student and want to be considered for the Google Summer of Code, please apply here after March 29th. You should have at least 2 PRs succesfully added to this repo before applying. Please follow the style guidelines mentioned above.

In vol 1 of my book, I use several notebooks taken from the book Dive Into Deep Learning. These are written in pytorch and stored at https://github.com/probml/probml-notebooks/tree/main/notebooks-d2l. The task is to convert these notebooks into JAX/FLAX code. It's okay to keep using torch dataloaders, but you should replace torch.tensor with jax.np.array, and torch.nn with flax.linen.nn, etc. Be sure to replace idioms like np.random.seed(seed=1); torch.manual_seed(1) with proper JAX PRNGs.

You should make a JAX version of each notebook, and open a PR for each one at https://github.com/probml/probml-notebooks/tree/main/notebooks-d2l. If the original notebook is called foo_torch.ipynb, please call yours foo_jax.ipynb.

See this example for how to format your code and PR.

Before you start translating a notebook, please check that it has not already been done! (I only want one translation per notebook :)

• [x] attention_torch.ipynb - https://github.com/probml/probml-notebooks/pull/40
• [x] batchnorm_torch.ipynb - https://github.com/probml/probml-notebooks/pull/60
• [x] bert_torch.ipynb - https://github.com/probml/probml-notebooks/pull/80
• [x] cnn1d_sentiment_torch.ipynb - https://github.com/probml/probml-notebooks/pull/62
• [x] conv2d_torch.ipynb - https://github.com/probml/probml-notebooks/pull/52
• [x] densenet_torch.ipynb - https://github.com/probml/probml-notebooks/pull/46
• [x] entailment_attention_mlp_torch.ipynb - https://github.com/probml/probml-notebooks/pull/75
• [x] finetune_cnn_torch.ipynb - https://github.com/probml/probml-notebooks/pull/68
• [x] gru_torch.ipynb - https://github.com/probml/probml-notebooks/pull/66
• [x] image_augmentation_torch.ipynb - https://github.com/probml/probml-notebooks/pull/48
• [x] lenet_torch.ipynb - https://github.com/probml/probml-notebooks/pull/42
• [x] lstm_torch.ipynb - https://github.com/probml/probml-notebooks/pull/59
• [x] multi_gpu_training_torch.ipynb - https://github.com/probml/probml-notebooks/pull/77
• [x] multi_head_attention.ipynb - https://github.com/probml/probml-notebooks/pull/47
• [x] naive_bayes_mnist.ipynb - https://github.com/probml/probml-notebooks/pull/52
• [x] nmt_attention_torch.ipynb - https://github.com/probml/probml-notebooks/pull/73
• [x] nmt_torch.ipynb - https://github.com/probml/probml-notebooks/pull/72
• [x] positional_encoding.ipynb - https://github.com/probml/probml-notebooks/pull/50
• [x] resnet_torch.ipynb - https://github.com/probml/probml-notebooks/pull/45
• [x] rnn_sentiment_torch.ipynb - https://github.com/probml/probml-notebooks/pull/64
• [x] rnn_torch.ipynb - https://github.com/probml/probml-notebooks/pull/67
• [x] skipgram_torch.ipynb - https://github.com/probml/probml-notebooks/pull/76
• [x] transformers_torch.ipynb - https://github.com/probml/probml-notebooks/pull/71
• [x] word_analogies_torch.ipynb - https://github.com/probml/probml-notebooks/pull/41

After adding D2L notebooks, workflow execution time has increased from 1 hour to 2 hours 14 minutes. Ideally, the workflow should be as fast as possible so that we can know the failing components quickly. One of the solutions for this is to enable an environment variable called SMOKE_RUN for notebooks containing epoch-wise training with a logic that if SMOKE_RUN is in the environment we run only 1-2 epochs else we run the maximum epochs as mentioned in the notebook.

Hello, this is a Python implementation for mlpPriorsDemo2. The figures are slightly different from the book because NumPy generates different random vectors than Matlab.

The advi.hist (ELBO) should increase over time, but it does not seem to, even though the posterior looks sensible. Find out why. https://github.com/probml/probml-notebooks/blob/main/notebooks/beta_binom_approx_post_pymc.ipynb

Hello Kevin,

I was wondering if there is a plan to port all of the code examples from the ML book to Python?

If that's the case I'd love to help out :)

Description

Implemented SNGP JAX demo basing it on this TF demo of spectrally normalized neural GPs and using these JAX functions from edward2: random_feature.py and normalization.py .

Issue

#819

Checklist

• [x] Performed a self-review of the code
• [x] Tested on Google Colab.

Implement blocked gibbs sampling for Bayesian inference in linear Gaussian SSM . In the "E step", use the Jax SSM library for forwards-filtering backwards-sampling. In the "M step", sample from the parameter posteriors assuming conjugate priors.

Some details can be found in this paper A. Wills, T. B. Schön, F. Lindsten, and B. Ninness, “Estimation of Linear Systems using a Gibbs Sampler,” IFAC proc. vol., vol. 45, no. 16, pp. 203–208, Jul. 2012, doi: 10.3182/20120711-3-be-2027.00297. [Online]. Available: https://linkinghub.elsevier.com/retrieve/pii/S1474667015379520

I encounter the following error when running compare_results.ipynb in the vae folder.

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-2-d04a2cc14755> in <module>()
      5 import matplotlib.pyplot as plt
      6 import torchvision.transforms as transforms
----> 7 from download_celeba import celeba_dataloader
      8 from assembler import get_config, assembler
      9 from utils.plot import plot_reconstruction, plot_samples

/content/download_celeba.py in <module>()
----> 1 import superimport
      2 
      3 from absl import app
      4 from absl import flags
      5 import torchvision.transforms as transforms

I think this can be resolved by adding one line in the notebook to download superimport into the colab.

Closes #174

The code might take some time to load and this is because it is downloading the MNIST dataset from online and the delay in execution is solely due to this and not due to the inefficiency of the code.

The values and the images will differ because the data is randomly chosen from the total set of images.

I initially added +1 to all images in the training set and changed the Bernoulli function to adapt to this, but did not see any change in the execution, It was more convenient for me to use the binary image as is and use the implementation of Bernoulli distribution which is used for this mixture model.

In this implementation, I did not find the optimal k because the optimal k=20 has already been found through trial and error in the MATLAB code and no graph implementation is found in the book.

Implemented the changes suggested:

1. Vectorized code and replaced iterating over the data points to vectorized versions
2. replaced Bernoulli function with https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.bernoulli.html
3. factored out the code that does parameter initialization and image plotting into subfunctions, to simplify main()
4. changed the data source from open_ml to tensorflow.datasets

Reproduce fig 6.5 using sklearn.mutual_info_regression instead of the MIC coefficient. The 357 health and economic variables are from the Science paper Detecting Novel Associations in Large Data Sets. (Cached copy of pdf is here)

The new figure should look something like this:

Figure 6.1 b) shows a graphic depiction of a position frequency matrix, but it is not a sequence logo because all column heights are equal. My guess is the original intent was to have a sub-figure 6.1 c) showing a sequence logo and compare that to the graphic in sub-figure 6.1 b) so the reader can see how much easier it is to spot conserved positions when the column heights are scaled to equal the information content. The text corresponding to figure 6.1 does mention heights scaled to information content (although "bar" height is used instead of "column" height.

To solve https://github.com/probml/pml-book/issues/440 Tweak https://colab.research.google.com/github/probml/pyprobml/blob/master/notebooks/book1/07/gaussEvec.ipynb The exponents should be lambda^{-1/2}. I tried changing the text but the figure no longer renders properly (u_1, u_2 axis broken)

• currently, this notebook won't run in colab, due to requirement of high memory.
• Mostly prediction loops taking much memory, we can make it batch prediction to consume less memory at a time