On Anytime Learning At Macroscale

Learning from sequential data dumps

(key) Requirements

Python 3.7
Pytorch 1.9.0
Hydra 1.1.0 (pip install hydra-core & pip install hydra-submitit-launcher)

Structure

├── crlapi           
  ├── benchmark.py    # Creates the data stream, feeds it to the model and evaluates it
  ├── core.py         # Abstract classes for 
  ├── logger.py   
  ├── sl
    ├── architectures
      ├── ...         # NN architectures used in this project
    ├── clmodels
      ├── ...         # Models (e.g. Single, gEns, ..., )
    ├── streams
      ├── ...         # CIFAR and MNIST stream implementatins

Running Experiments

To run experiments, you need to call the dataset specific run file, and you need to pass the configuration of the run. We have place the configurations in the previous directory (../configs). The config structure is as follows

    ├── configs
        ├── mnist
           ├── run.py                 # run file
           ├── test_usage_gmoe.yaml   # This is the "gMoE" model
           ├── test_finetune_mlp.yaml # This is the "Single Model"
           ... 
        ├── cifar
           ├── run.py                 # run file
           ├── test_finetune_vgg.yaml # This is the "Single Model"
           ├── test_usage_gmoe.yaml   # This is the "gMoE" model
           ...

To run an e.g. mnist gMoE run, the command is (launched from the directory just above (so cd ..)

PYTHONPATH=./ python configs/mnist/run.py -cn test_usage_gmoe n_megabatches=2 replay=1 clmodel.max_epochs=200

Important arguments

n_megabatches : controls the number of megabatches. So n_megabatches=1 is your regular full dataset training
replay : whether to use replay or not
clmodel.init_from_scratch : whether to reinitialize the model at every MB. Should only be used when replay=1
device : use cuda or cpu depending on your hardware

License

alma is released under the MIT license. See LICENSE for additional details about it. See also our Terms of Use and Privacy Policy.

Anytime Learning At Macroscale

Related tags

Overview

On Anytime Learning At Macroscale

(key) Requirements

Structure

Running Experiments

Important arguments

License

Owner

Meta Research

Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

Skforecast is a python library that eases using scikit-learn regressors as multi-step forecasters

Little Ball of Fur - A graph sampling extension library for NetworKit and NetworkX (CIKM 2020)

A data preprocessing package for time series data. Design for machine learning and deep learning.

Bodywork deploys machine learning projects developed in Python, to Kubernetes.

Mosec is a high-performance and flexible model serving framework for building ML model-enabled backend and microservices

PySurvival is an open source python package for Survival Analysis modeling

使用数学和计算机知识投机倒把

Apple-voice-recognition - Machine Learning

Iris-Heroku - Putting a Machine Learning Model into Production with Flask and Heroku

A complete guide to start and improve in machine learning (ML)

YouTube Spam Detection with python

Interactive Parallel Computing in Python

Responsible AI Workshop: a series of tutorials & walkthroughs to illustrate how put responsible AI into practice

Predicting Keystrokes using an Audio Side-Channel Attack and Machine Learning

High performance Python GLMs with all the features!

This jupyter notebook project was completed by me and my friend using the dataset from Kaggle

MLFlow in a Dockercontainer based on Azurite and Postgres

DoWhy is a Python library for causal inference that supports explicit modeling and testing of causal assumptions. DoWhy is based on a unified language for causal inference, combining causal graphical models and potential outcomes frameworks.

A single Python file with some tools for visualizing machine learning in the terminal.