Logging MXNet data for visualization in TensorBoard.

Overview

Logging MXNet Data for Visualization in TensorBoard

Overview

MXBoard provides a set of APIs for logging MXNet data for visualization in TensorBoard. The idea of this project comes from discussions with Zihao Zheng, the author of dmlc/tensorboard, on delivering a visualization solution for MXNet users. We aim at providing the logging APIs that can process MXNet data efficiently and supporting most of the data types for visualization in the TensorBoard GUI. We adapted the following low-level logging components from their Python and C++ implementations in TensorFlow: FileWriter, EventFileWriter, EventsWriter, RecordWriter, and _EventLoggerThread. We also adapted the user-level logging APIs defined in SummaryWriter from tensorboard-pytorch. The encoding algorithm used in writing protobuf objects into event files is directly borrowed from TeamHG-Memex/tensorboard_logger.

MXBoard supports a set of Python APIs for logging the following data types for TensorBoard to render. Logging APIs for other languages may be added in the future.

mxboard_cover

The corresponding Python APIs are accessible through a class called SummaryWriter as follows:

    mxboard.SummaryWriter.add_graph
    mxboard.SummaryWriter.add_scalar
    mxboard.SummaryWriter.add_histogram
    mxboard.SummaryWriter.add_embedding
    mxboard.SummaryWriter.add_image
    mxboard.SummaryWriter.add_text
    mxboard.SummaryWriter.add_pr_curve
    mxboard.SummaryWriter.add_audio

Installation

Install MXBoard from PyPI

pip install mxboard

Install MXBoard Python package from source

git clone https://github.com/awslabs/mxboard.git
cd mxboard/python
python setup.py install

Install TensorBoard from PyPI

MXBoard is a logger for writing MXNet data to event files. To visualize those data in browsers, users still have to install TensorBoard separately.

pip install tensorboard

Use the following to verify that the TensorBoard binary has been installed correctly.

tensorboard --help

Other required packages

MXBoard relies on the following packages for data logging.

Please note that you need to install MXNet manually before using MXBoard. The other packages will be installed automatically when you install MXBoard via pip or building from source. If you want to build from source, please make sure that protobuf compiler is installed. Check this page for downloading the protobuf compiler whose file name starts with "protoc".

Visualizing MXNet data in 30 seconds

Now that you have installed all of the required packages, let's walk through a simple visualization example. You will see how MXBoard enables visualizing MXNet NDArrays with histograms.

Step 1. Logging event data to a file.

Prepare a Python script for writing data generated by the normal operator to an event file. The data is generated ten times with decreasing standard deviation and written to the event file each time. It's expected to see the data distribution gradually become more centered around the mean value. Note that here we specify creating the event file in the folder logs under the current directory. We will need to pass this folder path to the TensorBoard binary.

import mxnet as mx
from mxboard import SummaryWriter


with SummaryWriter(logdir='./logs') as sw:
    for i in range(10):
        # create a normal distribution with fixed mean and decreasing std
        data = mx.nd.normal(loc=0, scale=10.0/(i+1), shape=(10, 3, 8, 8))
        sw.add_histogram(tag='norml_dist', values=data, bins=200, global_step=i)

Step 2. Launch TensorBoard to load the event file generated above.

Use the following command to start the TensorBoard server. It will use the logs that were generated in the current directory's logs folder.

tensorboard --logdir=./logs --host=127.0.0.1 --port=8888

Note that in some situations, the port number 8888 may be occupied by other applications and launching TensorBoard may fail. You may choose a different available port number.

Step 3. Open TensorBoard in your browser.

In the browser, enter the address 127.0.0.1:8888, and click the tab HISTOGRAMS in the TensorBoard GUI. You will see data distribution changing as time progresses.

summary_histogram_norm

More tutorials

References

  1. https://github.com/TeamHG-Memex/tensorboard_logger
  2. https://github.com/lanpa/tensorboard-pytorch
  3. https://github.com/dmlc/tensorboard
  4. https://github.com/tensorflow/tensorflow
  5. https://github.com/tensorflow/tensorboard

License

This library is licensed under the Apache 2.0 License.

Comments
  • No handlers could be found for logger

    No handlers could be found for logger "mxboard.event_file_writer"

    Hello, i install mxboard and run the demo py, but raise errors:

    No handlers could be found for logger "mxboard.event_file_writer"

    my conda list is as follows: mxboard 0.1.0 mxnet 1.2.0 tensorboard 1.12.0 tensorflow 1.12.0

    how can i solve this, thank u very much!

    bug 
    opened by HarveyGuo960817 8
  • add_embedding() does not support a list of strings as labels

    add_embedding() does not support a list of strings as labels

    The documentation states that sw.add_embedding() can take a list of elements, convertible to string as an argument to the labels parameter. However, the code prevents this both in add_embedding() and in _make_metadata_tsv() by throwing an error if labels is a List and not NDArray or NPArray.

    bug 
    opened by fhieber 7
  • fix: forcing make_image_grid to generate square sprite image

    fix: forcing make_image_grid to generate square sprite image

    Issues, if available: #35

    Description of changes: one line changed in utils/make_image_grid to make sure the produced sprite image is precisely square

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

    opened by YusongLeng 5
  • Import error

    Import error

    After pip install mxboard, when I import mxboard, it errors:

    `import mxboard as mb TypeError Traceback (most recent call last) in () ----> 1 import mxboard as mb

    /home/users/nn/code/py/mxnet/python/mxboard/init.py in () 19 20 from future import absolute_import ---> 21 from .writer import SummaryWriter 22 23 version = '0.1.0'

    /home/users/nn/code/py/mxnet/python/mxboard/writer.py in () 26 import os 27 import logging ---> 28 from .proto import event_pb2 29 from .proto import summary_pb2 30 from .event_file_writer import EventFileWriter

    /home/users/nn/code/py/mxnet/python/mxboard/proto/event_pb2.py in () 14 15 ---> 16 from mxboard.proto import summary_pb2 as mxboard_dot_proto_dot_summary__pb2 17 18

    /home/users/nn/code/py/mxnet/python/mxboard/proto/summary_pb2.py in () 14 15 ---> 16 from mxboard.proto import tensor_pb2 as mxboard_dot_proto_dot_tensor__pb2 17 18

    /home/users/nn/code/py/mxnet/python/mxboard/proto/tensor_pb2.py in () 14 15 ---> 16 from mxboard.proto import resource_handle_pb2 as mxboard_dot_proto_dot_resource__handle__pb2 17 from mxboard.proto import tensor_shape_pb2 as mxboard_dot_proto_dot_tensor__shape__pb2 18 from mxboard.proto import types_pb2 as mxboard_dot_proto_dot_types__pb2

    /home/users/nn/code/py/mxnet/python/mxboard/proto/resource_handle_pb2.py in () 39 message_type=None, enum_type=None, containing_type=None, 40 is_extension=False, extension_scope=None, ---> 41 options=None, file=DESCRIPTOR), 42 _descriptor.FieldDescriptor( 43 name='container', full_name='tensorboard.ResourceHandleProto.container', index=1,

    TypeError: new() got an unexpected keyword argument 'file'`

    opened by firestonelib 5
  • Configuring logging output

    Configuring logging output

    Is it possible to configure the logger of mxboard? If one uses frequent statements as such

    with SummarWriter(logdir=.) as sw:
      sw.add_scalar()
    

    the log of the application becomes quite cluttered with outputs like:

    [INFO:root] Successfully opened events file: events.out.tfevents.1522834055.186590d665ad.ant.amazon.com
    [INFO:root] Wrote 1 event to disk
    [INFO:root] Wrote 1 event to disk
    [INFO:root] Wrote 6 events to disk
    
    question 
    opened by fhieber 5
  • How to record histrogram of nueron network when one of its layer is set to use pre-trained weights?

    How to record histrogram of nueron network when one of its layer is set to use pre-trained weights?

    I followed this gluon's document classification tutorial and tried to visualise my training with mxboard. So far I can visualise my model architecture, training loss, and accuracy.

    However, my problem is when I want to record histograms of my model. Python throws this error:

    Runtime Error: "Cannot get gradient array for Parameter [my layer name] because grad_req='null'"

    then I checked my model initialise block and found these lines:

    model.embedding_layer.weight.set_data(vocab.embedding.idx_to_vec)
    model.embedding_layer.collect_params().setattr('grad_req', 'null')
    

    and mxnet didn't run the gradient for this layer because I want to use my pre-trained embedding.

    So, my question is how do I get gradients for the other layers except for the embedding layer?

    opened by bkktimber 3
  • add support for multi labels for embeddings

    add support for multi labels for embeddings

    This pr adds support for multi labels per data point for the tensorboard embeddings. This means you can now pass a either a 1D or 2D list/numpy.ndarray/mx.ndarray for the labels to the add_embedding for use in tensorboard. The file format for the 2D label input is a TSV with column headers based on the tensorboard documentation https://www.tensorflow.org/guide/embedding#metadata

    I tired to cover my bases with the input checking, any suggestions for improvements are appreciated.

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

    opened by jessebrizzi 3
  • import error

    import error

    [email protected]:/opt/package# pip install mxboard
    Requirement already satisfied: mxboard in /usr/local/anaconda2/lib/python2.7/site-packages (0.1.0)
    Requirement already satisfied: six in /usr/local/anaconda2/lib/python2.7/site-packages (from mxboard) (1.11.0)
    Requirement already satisfied: numpy in /usr/local/anaconda2/lib/python2.7/site-packages (from mxboard) (1.13.3)
    Requirement already satisfied: Pillow in /usr/local/anaconda2/lib/python2.7/site-packages (from mxboard) (4.2.1)
    Requirement already satisfied: protobuf>=3.0.0 in /usr/local/anaconda2/lib/python2.7/site-packages (from mxboard) (3.5.2)
    Requirement already satisfied: olefile in /usr/local/anaconda2/lib/python2.7/site-packages (from Pillow->mxboard) (0.44)
    Requirement already satisfied: setuptools in /usr/local/anaconda2/lib/python2.7/site-packages (from protobuf>=3.0.0->mxboard) (36.5.0.post20170921)
    [email protected]:/opt/package# python mxboard.py
    Traceback (most recent call last):
      File "mxboard.py", line 2, in <module>
        from mxboard import SummaryWriter
      File "/opt/package/mxboard.py", line 2, in <module>
        from mxboard import SummaryWriter
    ImportError: cannot import name SummaryWriter
    [email protected]:/opt/package#
    
    This problem has been bothering me for a long time
    
    if I only import mxboard, it does not have error. when I run the mxboard demo, it report this error.
    
    opened by as754770178 3
  • Change README: tensorboard no longer requires tensorflow

    Change README: tensorboard no longer requires tensorflow

    Issue #, if available:

    Description of changes: Tensorboard no longer requires TensorFlow, so remove tensorflow from the installation instruction for tensorboard.

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

    @reminisce

    opened by hcho3 2
  • Revert

    Revert "fix text_summary for newer version of tensorboard (#32)"

    This reverts commit 2be9b6ef174ec35165321e70636e0f73232bf384.

    Issue #, if available:

    Description of changes:

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

    opened by SamanthaFeidFischer 2
  • A tiny bug in making sprite image for embedding projector

    A tiny bug in making sprite image for embedding projector

    Hi,

    I just found a ting bug in how the current mxboard generating the sprite image for embedding projector. This bug may result in wrong embedding-thumbnail association. A simple example borrowed from here is shown as below.

    import numpy as np
    import mxnet as mx
    from mxnet import gluon
    from mxboard import SummaryWriter
    
    
    batch_size = 65
    
    
    def transformer(data, label):
        data = data.reshape((-1,)).astype(np.float32)/255
        return data, label
    
    train_data = gluon.data.DataLoader(
        gluon.data.vision.MNIST('./data', train=True, transform=transformer),
        batch_size=batch_size, shuffle=False, last_batch='discard')
    
    initialized = False
    embedding = None
    labels = None
    images = None
    
    for i, (data, label) in enumerate(train_data):
        if i >= 1:
            # only fetch the first batche of images, i.e. 65 images -> 8*9 sprite image
            break
        if initialized:
            embedding = mx.nd.concat(*(embedding, data), dim=0)
            labels = mx.nd.concat(*(labels, label), dim=0)
            images = mx.nd.concat(*(images, data.reshape(batch_size, 1, 28, 28)), dim=0)
        else:
            embedding = data
            labels = label
            images = data.reshape(batch_size, 1, 28, 28)
            initialized = True
    
    with SummaryWriter(logdir='./logs') as sw:
        sw.add_embedding(tag='mnist', embedding=embedding, labels=labels, images=images)
    

    The above code takes 65 images of handwritten digits from the MNIST dataset and log them as embedding vectors with labels and original images. The generated sprite image is of grid 8 * 9, which is not square:

    sprite

    But, the current TensorBoard requires a square sprite image for the embedding projector working correctly (see here). So, the above non-square sprite image will make the thumbnails misaligned with their embeddings:

    screenshot 2019-01-14 at 18 10 49

    (e.g. embedding at the center of label 2 is displayed with thumbnail '9')

    This issue can be easily solved by changing just one line to constrain the produced sprite image to be precisely squared:

    sprite

    I tested it locally and would like to make a pull request. ;)

    opened by YusongLeng 2
  • Support summary written to HDFS

    Support summary written to HDFS

    This PR supports use case below:

    from mxboard import SummaryWriter
    with SummaryWriter(logdir='hdfs://hdfs-nn-proxy-ns1/department/ai/user/xyz/logs') as sw:
        for i in range(10):
            # create a normal distribution with fixed mean and decreasing std
            data = mx.nd.normal(loc=0, scale=10.0/(i+1), shape=(10, 3, 8, 8))
            sw.add_histogram(tag='norml_dist', values=data, bins=200, global_step=i)
    
    opened by 372046933 0
  • Proto file duplication issue.

    Proto file duplication issue.

    TypeError: Conflict register for file "mxboard/proto/resource_handle.proto": tensorboard.ResourceHandleProto is already defined in file "tensorboard/compat/proto/resource_handle.proto". Please fix the conflict by adding package name on the proto file, or use different name for the duplication. ... File "C:\Users\selcu\anaconda3\lib\site-packages\mxboard_init_.py", line 21, in from .writer import SummaryWriter File "C:\Users\selcu\anaconda3\lib\site-packages\mxboard\writer.py", line 28, in from .proto import event_pb2 File "C:\Users\selcu\anaconda3\lib\site-packages\mxboard\proto\event_pb2.py", line 16, in from mxboard.proto import summary_pb2 as mxboard_dot_proto_dot_summary__pb2 File "C:\Users\selcu\anaconda3\lib\site-packages\mxboard\proto\summary_pb2.py", line 16, in from mxboard.proto import tensor_pb2 as mxboard_dot_proto_dot_tensor__pb2 File "C:\Users\selcu\anaconda3\lib\site-packages\mxboard\proto\tensor_pb2.py", line 16, in from mxboard.proto import resource_handle_pb2 as mxboard_dot_proto_dot_resource__handle__pb2 File "C:\Users\selcu\anaconda3\lib\site-packages\mxboard\proto\resource_handle_pb2.py", line 94, in _sym_db.RegisterMessage(ResourceHandleProto) File "C:\Users\selcu\anaconda3\lib\site-packages\google\protobuf\symbol_database.py", line 84, in RegisterMessage self.RegisterMessageDescriptor(desc) File "C:\Users\selcu\anaconda3\lib\site-packages\google\protobuf\symbol_database.py", line 95, in RegisterMessageDescriptor self.pool._AddDescriptor(message_descriptor) File "C:\Users\selcu\anaconda3\lib\site-packages\google\protobuf\descriptor_pool.py", line 238, in _AddDescriptor self._CheckConflictRegister(desc, desc.full_name, desc.file.name) File "C:\Users\selcu\anaconda3\lib\site-packages\google\protobuf\descriptor_pool.py", line 191, in _CheckConflictRegister raise TypeError(error_msg) TypeError: Conflict register for file "mxboard/proto/resource_handle.proto": tensorboard.ResourceHandleProto is already defined in file "tensorboard/compat/proto/resource_handle.proto". Please fix the conflict by adding package name on the proto file, or use different name for the duplication.

    Process finished with exit code 1

    opened by develooper1994 2
  • Networks with BatchNorm and gradients

    Networks with BatchNorm and gradients

    I have set the i.grad != "null" which I found in here but now I receive error conv0_weight' was not initialized on context cpu(0). It was only initialized on [gpu(0), gpu(1)], but if I run it on just one gpu I receive the assertion error len(grads) == len(param_names) this occurs on all nets with a BatchNorm ones I have written and one in gluon's model zoo tried Inception, Densenet121 error:

    grads = [i.grad() for i in net.collect_params().values() if i.grad_req != 'null'] 50 assert len(grads) == len(param_names) 51

    in (.0) 47 sw.add_graph(net) 48 ---> 49 grads = [i.grad() for i in net.collect_params().values() if i.grad_req != 'null'] 50 assert len(grads) == len(param_names) 51

    /usr/local/lib/python3.6/dist-packages/mxnet/gluon/parameter.py in grad(self, ctx) 570 "Cannot get gradient array for Parameter '%s' "
    571 "because grad_req='null'"%(self.name)) --> 572 return self._check_and_get(self._grad, ctx) 573 574 def list_grad(self):

    /usr/local/lib/python3.6/dist-packages/mxnet/gluon/parameter.py in _check_and_get(self, arr_list, ctx) 225 "Parameter '%s' was not initialized on context %s. " 226 "It was only initialized on %s."%( --> 227 self.name, str(ctx), str(self._ctx_list))) 228 if self._deferred_init: 229 raise DeferredInitializationError(

    RuntimeError: Parameter 'densenet0_conv0_weight' was not initialized on context cpu(0). It was only initialized on [gpu(0), gpu(1)].

    opened by hskramer 0
  •  Appending to event file

    Appending to event file

    Is there a way to make SummaryWriter append to old event file, instead of creating new one each time it's called?

    Or is there any other method to prepare log files, so that if I interrupt and then continue training later, get a desired Tensorboard output (and not just graph with duplicated values)?.

    good first issue question 
    opened by lambdaofgod 1
Releases(v0.1.0)
  • v0.1.0(May 22, 2018)

    MXBoard Change Log

    v0.1.0 (first release)

    Supports logging MXNet data and visualizing them as Graph, Scalar, Histogram, Embedding, Image, Text, PR-Curve, or Audio in TensorBoard.

    Source code(tar.gz)
    Source code(zip)
Owner
Amazon Web Services - Labs
AWS Labs
Amazon Web Services - Labs
JittorVis - Visual understanding of deep learning model.

JittorVis - Visual understanding of deep learning model.

182 Jan 06, 2023
GNNLens2 is an interactive visualization tool for graph neural networks (GNN).

GNNLens2 is an interactive visualization tool for graph neural networks (GNN).

Distributed (Deep) Machine Learning Community 143 Jan 07, 2023
Visualizer for neural network, deep learning, and machine learning models

Netron is a viewer for neural network, deep learning and machine learning models. Netron supports ONNX (.onnx, .pb, .pbtxt), Keras (.h5, .keras), Tens

Lutz Roeder 20.9k Dec 28, 2022
Convolutional neural network visualization techniques implemented in PyTorch.

This repository contains a number of convolutional neural network visualization techniques implemented in PyTorch.

1 Nov 06, 2021
Summary Explorer is a tool to visually explore the state-of-the-art in text summarization.

Summary Explorer is a tool to visually explore the state-of-the-art in text summarization.

Webis 42 Aug 14, 2022
L2X - Code for replicating the experiments in the paper Learning to Explain: An Information-Theoretic Perspective on Model Interpretation.

L2X Code for replicating the experiments in the paper Learning to Explain: An Information-Theoretic Perspective on Model Interpretation at ICML 2018,

Jianbo Chen 113 Sep 06, 2022
Bias and Fairness Audit Toolkit

The Bias and Fairness Audit Toolkit Aequitas is an open-source bias audit toolkit for data scientists, machine learning researchers, and policymakers

Data Science for Social Good 513 Jan 06, 2023
A ultra-lightweight 3D renderer of the Tensorflow/Keras neural network architectures

A ultra-lightweight 3D renderer of the Tensorflow/Keras neural network architectures

Souvik Pratiher 16 Nov 17, 2021
PyTorch implementation of DeepDream algorithm

neural-dream This is a PyTorch implementation of DeepDream. The code is based on neural-style-pt. Here we DeepDream a photograph of the Golden Gate Br

121 Nov 05, 2022
A collection of infrastructure and tools for research in neural network interpretability.

Lucid Lucid is a collection of infrastructure and tools for research in neural network interpretability. We're not currently supporting tensorflow 2!

4.5k Jan 07, 2023
Visual Computing Group (Ulm University) 99 Nov 30, 2022
Visualize a molecule and its conformations in Jupyter notebooks/lab using py3dmol

Mol Viewer This is a simple package wrapping py3dmol for a single command visualization of a RDKit molecule and its conformations (embed as Conformer

Benoît BAILLIF 1 Feb 11, 2022
Contrastive Explanation (Foil Trees), developed at TNO/Utrecht University

Contrastive Explanation (Foil Trees) Contrastive and counterfactual explanations for machine learning (ML) Marcel Robeer (2018-2020), TNO/Utrecht Univ

M.J. Robeer 41 Aug 29, 2022
A library that implements fairness-aware machine learning algorithms

Themis ML themis-ml is a Python library built on top of pandas and sklearnthat implements fairness-aware machine learning algorithms. Fairness-aware M

Niels Bantilan 105 Dec 30, 2022
🎆 A visualization of the CapsNet layers to better understand how it works

CapsNet-Visualization For more information on capsule networks check out my Medium articles here and here. Setup Use pip to install the required pytho

Nick Bourdakos 387 Dec 06, 2022
Visualization toolkit for neural networks in PyTorch! Demo -->

FlashTorch A Python visualization toolkit, built with PyTorch, for neural networks in PyTorch. Neural networks are often described as "black box". The

Misa Ogura 692 Dec 29, 2022
Many Class Activation Map methods implemented in Pytorch for CNNs and Vision Transformers. Including Grad-CAM, Grad-CAM++, Score-CAM, Ablation-CAM and XGrad-CAM

Class Activation Map methods implemented in Pytorch pip install grad-cam ⭐ Comprehensive collection of Pixel Attribution methods for Computer Vision.

Jacob Gildenblat 6.5k Jan 01, 2023
Python Library for Model Interpretation/Explanations

Skater Skater is a unified framework to enable Model Interpretation for all forms of model to help one build an Interpretable machine learning system

Oracle 1k Dec 27, 2022
Delve is a Python package for analyzing the inference dynamics of your PyTorch model.

Delve is a Python package for analyzing the inference dynamics of your PyTorch model.

Delve 73 Dec 12, 2022
Algorithms for monitoring and explaining machine learning models

Alibi is an open source Python library aimed at machine learning model inspection and interpretation. The focus of the library is to provide high-qual

Seldon 1.9k Dec 30, 2022