On Anytime Learning At Macroscale

Learning from sequential data dumps

(key) Requirements

Python 3.7
Pytorch 1.9.0
Hydra 1.1.0 (pip install hydra-core & pip install hydra-submitit-launcher)

Structure

├── crlapi           
  ├── benchmark.py    # Creates the data stream, feeds it to the model and evaluates it
  ├── core.py         # Abstract classes for 
  ├── logger.py   
  ├── sl
    ├── architectures
      ├── ...         # NN architectures used in this project
    ├── clmodels
      ├── ...         # Models (e.g. Single, gEns, ..., )
    ├── streams
      ├── ...         # CIFAR and MNIST stream implementatins

Running Experiments

To run experiments, you need to call the dataset specific run file, and you need to pass the configuration of the run. We have place the configurations in the previous directory (../configs). The config structure is as follows

    ├── configs
        ├── mnist
           ├── run.py                 # run file
           ├── test_usage_gmoe.yaml   # This is the "gMoE" model
           ├── test_finetune_mlp.yaml # This is the "Single Model"
           ... 
        ├── cifar
           ├── run.py                 # run file
           ├── test_finetune_vgg.yaml # This is the "Single Model"
           ├── test_usage_gmoe.yaml   # This is the "gMoE" model
           ...

To run an e.g. mnist gMoE run, the command is (launched from the directory just above (so cd ..)

PYTHONPATH=./ python configs/mnist/run.py -cn test_usage_gmoe n_megabatches=2 replay=1 clmodel.max_epochs=200

Important arguments

n_megabatches : controls the number of megabatches. So n_megabatches=1 is your regular full dataset training
replay : whether to use replay or not
clmodel.init_from_scratch : whether to reinitialize the model at every MB. Should only be used when replay=1
device : use cuda or cpu depending on your hardware

License

alma is released under the MIT license. See LICENSE for additional details about it. See also our Terms of Use and Privacy Policy.

Anytime Learning At Macroscale

Related tags

Overview

On Anytime Learning At Macroscale

(key) Requirements

Structure

Running Experiments

Important arguments

License

Owner

Meta Research

MooGBT is a library for Multi-objective optimization in Gradient Boosted Trees.

Simplify stop motion animation with machine learning.

The easy way to combine mlflow, hydra and optuna into one machine learning pipeline.

Price Prediction model is used to develop an LSTM model to predict the future market price of Bitcoin and Ethereum.

Both social media sentiment and stock market data are crucial for stock price prediction

A machine learning model for Covid case prediction

Pyomo is an object-oriented algebraic modeling language in Python for structured optimization problems.

This is the code repository for LRM Stochastic watershed model.

QML: A Python Toolkit for Quantum Machine Learning

Decision tree is the most powerful and popular tool for classification and prediction

GroundSeg Clustering Optimized Kdtree

Price forecasting of SGB and IRFC Bonds and comparing there returns

Learn Machine Learning Algorithms by doing projects in Python and R Programming Language

A library of extension and helper modules for Python's data analysis and machine learning libraries.

Interactive Web App with Streamlit and Scikit-learn that applies different Classification algorithms to popular datasets

Extended Isolation Forest for Anomaly Detection

nn-Meter is a novel and efficient system to accurately predict the inference latency of DNN models on diverse edge devices

A basic Ray Tracer that exploits numpy arrays and functions to work fast.

Fit interpretable models. Explain blackbox machine learning.

A python library for easy manipulation and forecasting of time series.