Code for the paper "Next Generation Reservoir Computing"

Overview

Next Generation Reservoir Computing

This is the code for the results and figures in our paper "Next Generation Reservoir Computing". They are written in Python, and require recent versions of NumPy, SciPy, and matplotlib. If you are using a Python environment like Anaconda, these are likely already installed.

Python Virtual Environment

If you are not using Anaconda, or want to run this code on the command line in vanilla Python, you can create a virtual environment with the required dependencies by running:

python3 -m venv env
./env/bin/pip install -r requirements.txt

This will install the most recent version of the requirements available to you. If you wish to use the exact versions we used, use requirements-exact.txt instead.

You can then run the individual scripts, for example:

./env/bin/python DoubleScrollNVAR-RK23.py
You might also like...
Code for the paper Learning the Predictability of the Future

Learning the Predictability of the Future Code from the paper Learning the Predictability of the Future. Website of the project in hyperfuture.cs.colu

PyTorch code for the paper: FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning
PyTorch code for the paper: FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning

FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning This is the PyTorch implementation of our paper: FeatMatch: Feature-Based Augmentat

Code for the paper A Theoretical Analysis of the Repetition Problem in Text Generation
Code for the paper A Theoretical Analysis of the Repetition Problem in Text Generation

A Theoretical Analysis of the Repetition Problem in Text Generation This repository share the code for the paper "A Theoretical Analysis of the Repeti

Code for our ICASSP 2021 paper: SA-Net: Shuffle Attention for Deep Convolutional Neural Networks
Code for our ICASSP 2021 paper: SA-Net: Shuffle Attention for Deep Convolutional Neural Networks

SA-Net: Shuffle Attention for Deep Convolutional Neural Networks (paper) By Qing-Long Zhang and Yu-Bin Yang [State Key Laboratory for Novel Software T

Open source repository for the code accompanying the paper 'Non-Rigid Neural Radiance Fields Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video'.
Open source repository for the code accompanying the paper 'Non-Rigid Neural Radiance Fields Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video'.

Non-Rigid Neural Radiance Fields This is the official repository for the project "Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synt

Code for the Shortformer model, from the paper by Ofir Press, Noah A. Smith and Mike Lewis.

Shortformer This repository contains the code and the final checkpoint of the Shortformer model. This file explains how to run our experiments on the

PyTorch code for ICLR 2021 paper Unbiased Teacher for Semi-Supervised Object Detection
PyTorch code for ICLR 2021 paper Unbiased Teacher for Semi-Supervised Object Detection

Unbiased Teacher for Semi-Supervised Object Detection This is the PyTorch implementation of our paper: Unbiased Teacher for Semi-Supervised Object Detection

Official code for paper "Optimization for Oriented Object Detection via Representation Invariance Loss".

Optimization for Oriented Object Detection via Representation Invariance Loss By Qi Ming, Zhiqiang Zhou, Lingjuan Miao, Xue Yang, and Yunpeng Dong. Th

Code for our CVPR 2021 paper
Code for our CVPR 2021 paper "MetaCam+DSCE"

Joint Noise-Tolerant Learning and Meta Camera Shift Adaptation for Unsupervised Person Re-Identification (CVPR'21) Introduction Code for our CVPR 2021

Comments
  • Generalized Performance

    Generalized Performance

    I modified the code given in this repo to what I think is a more generalized version (below) where the input is an array containing points generated by any sort of process. It gives a perfect result on predicting sin functions, but on a constant linear trend gives absolutely terrible, nonsense performance. By my understanding, that is simply the nature of reservoir computing, it can't handle a trend. Is that correct?

    I would also appreciate any other insight you might have on the generalization of this function. Thanks!

    import numpy as np
    import pandas as pd
    
    
    def load_linear(long=False, shape=None, start_date: str = "2021-01-01"):
        """Create a dataset of just zeroes for testing edge case."""
        if shape is None:
            shape = (500, 5)
        df_wide = pd.DataFrame(
            np.ones(shape), index=pd.date_range(start_date, periods=shape[0], freq="D")
        )
        df_wide = (df_wide * list(range(0, shape[1]))).cumsum()
        if not long:
            return df_wide
        else:
            df_wide.index.name = "datetime"
            df_long = df_wide.reset_index(drop=False).melt(
                id_vars=['datetime'], var_name='series_id', value_name='value'
            )
            return df_long
    
    
    def load_sine(long=False, shape=None, start_date: str = "2021-01-01"):
        """Create a dataset of just zeroes for testing edge case."""
        if shape is None:
            shape = (500, 5)
        df_wide = pd.DataFrame(
            np.ones(shape),
            index=pd.date_range(start_date, periods=shape[0], freq="D"),
            columns=range(shape[1])
        )
        X = pd.to_numeric(df_wide.index, errors='coerce', downcast='integer').values
    
        def sin_func(a, X):
            return a * np.sin(1 * X) + a
        for column in df_wide.columns:
            df_wide[column] = sin_func(column, X)
        if not long:
            return df_wide
        else:
            df_wide.index.name = "datetime"
            df_long = df_wide.reset_index(drop=False).melt(
                id_vars=['datetime'], var_name='series_id', value_name='value'
            )
            return df_long
    
    
    def predict_reservoir(df, forecast_length, warmup_pts, k=2, ridge_param=2.5e-6):
        # k =  # number of time delay taps
        # pass in traintime_pts to limit as .tail() for huge datasets?
    
        n_pts = df.shape[1]
        # handle short data edge case
        min_train_pts = 10
        max_warmup_pts = n_pts - min_train_pts
        if warmup_pts >= max_warmup_pts:
            warmup_pts = max_warmup_pts if max_warmup_pts > 0 else 0
    
        traintime_pts = n_pts - warmup_pts   # round(traintime / dt)
        warmtrain_pts = warmup_pts + traintime_pts
        testtime_pts = forecast_length + 1  # round(testtime / dt)
        maxtime_pts = n_pts  # round(maxtime / dt)
    
        # input dimension
        d = df.shape[0]
        # size of the linear part of the feature vector
        dlin = k * d
        # size of nonlinear part of feature vector
        dnonlin = int(dlin * (dlin + 1) / 2)
        # total size of feature vector: constant + linear + nonlinear
        dtot = 1 + dlin + dnonlin
    
        # create an array to hold the linear part of the feature vector
        x = np.zeros((dlin, maxtime_pts))
    
        # fill in the linear part of the feature vector for all times
        for delay in range(k):
            for j in range(delay, maxtime_pts):
                x[d * delay : d * (delay + 1), j] = df[:, j - delay]
    
        # create an array to hold the full feature vector for training time
        # (use ones so the constant term is already 1)
        out_train = np.ones((dtot, traintime_pts))
    
        # copy over the linear part (shift over by one to account for constant)
        out_train[1 : dlin + 1, :] = x[:, warmup_pts - 1 : warmtrain_pts - 1]
    
        # fill in the non-linear part
        cnt = 0
        for row in range(dlin):
            for column in range(row, dlin):
                # shift by one for constant
                out_train[dlin + 1 + cnt] = (
                    x[row, warmup_pts - 1 : warmtrain_pts - 1]
                    * x[column, warmup_pts - 1 : warmtrain_pts - 1]
                )
                cnt += 1
    
        # ridge regression: train W_out to map out_train to Lorenz[t] - Lorenz[t - 1]
        W_out = (
            (x[0:d, warmup_pts:warmtrain_pts] - x[0:d, warmup_pts - 1 : warmtrain_pts - 1])
            @ out_train[:, :].T
            @ np.linalg.pinv(
                out_train[:, :] @ out_train[:, :].T + ridge_param * np.identity(dtot)
            )
        )
    
        # create a place to store feature vectors for prediction
        out_test = np.ones(dtot)  # full feature vector
        x_test = np.zeros((dlin, testtime_pts))  # linear part
    
        # copy over initial linear feature vector
        x_test[:, 0] = x[:, warmtrain_pts - 1]
    
        # do prediction
        for j in range(testtime_pts - 1):
            # copy linear part into whole feature vector
            out_test[1 : dlin + 1] = x_test[:, j]  # shift by one for constant
            # fill in the non-linear part
            cnt = 0
            for row in range(dlin):
                for column in range(row, dlin):
                    # shift by one for constant
                    out_test[dlin + 1 + cnt] = x_test[row, j] * x_test[column, j]
                    cnt += 1
            # fill in the delay taps of the next state
            x_test[d:dlin, j + 1] = x_test[0 : (dlin - d), j]
            # do a prediction
            x_test[0:d, j + 1] = x_test[0:d, j] + W_out @ out_test[:]
        return x_test[0:d, 1:]
    
    
    # note transposed from the opposite of my usual shape
    data_pts = 7000
    series = 3
    forecast_length = 10
    df_sine = load_sine(long=False, shape=(data_pts, series)).transpose().to_numpy()
    df_sine_train = df_sine[:, :-10]
    df_sine_test = df_sine[:, -10:]
    prediction_sine = predict_reservoir(df_sine_train, forecast_length=forecast_length, warmup_pts=150, k=2, ridge_param=2.5e-6)
    print(f"sine MAE {np.mean(np.abs(df_sine_test - prediction_sine))}")
    
    df_linear = load_linear(long=False, shape=(data_pts, series)).transpose().to_numpy()
    df_linear_train = df_linear[:, :-10]
    df_linear_test = df_linear[:, -10:]
    prediction_linear = predict_reservoir(df_linear_train, forecast_length=forecast_length, warmup_pts=150, k=2, ridge_param=2.5e-6)
    print(f"linear MAE {np.mean(np.abs(df_linear_test - prediction_linear))}")
    
    
    opened by winedarksea 2
  • Link to your paper

    Link to your paper

    I'm documenting here the link to your paper. I couldn't find it in the readme:


    Next generation reservoir computing

    Daniel J. Gauthier, Erik Bollt, Aaron Griffith & Wendson A. S. Barbosa 
    

    Nature Communications volume 12, Article number: 5564 (2021) https://www.nature.com/articles/s41467-021-25801-2

    opened by impredicative 1
Releases(v1.0)
Owner
OSU QuantInfo Lab
Daniel Gauthier's Research Group
OSU QuantInfo Lab
Implementation of the HMAX model of vision in PyTorch

PyTorch implementation of HMAX PyTorch implementation of the HMAX model that closely follows that of the MATLAB implementation of The Laboratory for C

Marijn van Vliet 52 Oct 13, 2022
DeepLab2: A TensorFlow Library for Deep Labeling

DeepLab2 is a TensorFlow library for deep labeling, aiming to provide a unified and state-of-the-art TensorFlow codebase for dense pixel labeling tasks.

Google Research 845 Jan 04, 2023
Official implementation of Rethinking Graph Neural Architecture Search from Message-passing (CVPR2021)

Rethinking Graph Neural Architecture Search from Message-passing Intro The GNAS can automatically learn better architecture with the optimal depth of

Shaofei Cai 48 Sep 30, 2022
Super-Fast-Adversarial-Training - A PyTorch Implementation code for developing super fast adversarial training

Super-Fast-Adversarial-Training This is a PyTorch Implementation code for develo

LBK 26 Dec 02, 2022
Instance-level Image Retrieval using Reranking Transformers

Instance-level Image Retrieval using Reranking Transformers Fuwen Tan, Jiangbo Yuan, Vicente Ordonez, ICCV 2021. Abstract Instance-level image retriev

UVA Computer Vision 87 Jan 03, 2023
TCNN Temporal convolutional neural network for real-time speech enhancement in the time domain

TCNN Pandey A, Wang D L. TCNN: Temporal convolutional neural network for real-time speech enhancement in the time domain[C]//ICASSP 2019-2019 IEEE Int

凌逆战 16 Dec 30, 2022
😇A pyTorch implementation of the DeepMoji model: state-of-the-art deep learning model for analyzing sentiment, emotion, sarcasm etc

------ Update September 2018 ------ It's been a year since TorchMoji and DeepMoji were released. We're trying to understand how it's being used such t

Hugging Face 865 Dec 24, 2022
Deep-Learning-Image-Captioning - Implementing convolutional and recurrent neural networks in Keras to generate sentence descriptions of images

Deep Learning - Image Captioning with Convolutional and Recurrent Neural Nets ========================================================================

23 Apr 06, 2022
Real-time multi-object tracker using YOLO v5 and deep sort

This repository contains a two-stage-tracker. The detections generated by YOLOv5, a family of object detection architectures and models pretrained on the COCO dataset, are passed to a Deep Sort algor

Mike 3.6k Jan 05, 2023
Code for Understanding Pooling in Graph Neural Networks

Select, Reduce, Connect This repository contains the code used for the experiments of: "Understanding Pooling in Graph Neural Networks" Setup Install

Daniele Grattarola 37 Dec 13, 2022
source code the paper Fast and Robust Iterative Closet Point.

Fast-Robust-ICP This repository includes the source code the paper Fast and Robust Iterative Closet Point. Authors: Juyong Zhang, Yuxin Yao, Bailin De

yaoyuxin 320 Dec 28, 2022
Doods2 - API for detecting objects in images and video streams using Tensorflow

DOODS2 - Return of DOODS Dedicated Open Object Detection Service - Yes, it's a b

Zach 101 Jan 04, 2023
auto-tuning momentum SGD optimizer

YellowFin YellowFin is an auto-tuning optimizer based on momentum SGD which requires no manual specification of learning rate and momentum. It measure

Jian Zhang 288 Nov 19, 2022
Code for Graph-to-Tree Learning for Solving Math Word Problems (ACL 2020)

Graph-to-Tree Learning for Solving Math Word Problems PyTorch implementation of Graph based Math Word Problem solver described in our ACL 2020 paper G

Jipeng Zhang 66 Nov 23, 2022
The code for our CVPR paper PISE: Person Image Synthesis and Editing with Decoupled GAN, Project Page, supp.

PISE The code for our CVPR paper PISE: Person Image Synthesis and Editing with Decoupled GAN, Project Page, supp. Requirement conda create -n pise pyt

jinszhang 110 Nov 21, 2022
Crowd-sourced Annotation of Human Motion.

Motion Annotation Tool Live: https://motion-annotation.humanoids.kit.edu Paper: The KIT Motion-Language Dataset Installation Start by installing all P

Matthias Plappert 4 May 25, 2020
Attention-based Transformation from Latent Features to Point Clouds (AAAI 2022)

Attention-based Transformation from Latent Features to Point Clouds This repository contains a PyTorch implementation of the paper: Attention-based Tr

12 Nov 11, 2022
VLG-Net: Video-Language Graph Matching Networks for Video Grounding

VLG-Net: Video-Language Graph Matching Networks for Video Grounding Introduction Official repository for VLG-Net: Video-Language Graph Matching Networ

Mattia Soldan 25 Dec 04, 2022
Code for reproducible experiments presented in KSD Aggregated Goodness-of-fit Test.

Code for KSDAgg: a KSD aggregated goodness-of-fit test This GitHub repository contains the code for the reproducible experiments presented in our pape

Antonin Schrab 5 Dec 15, 2022