Video-Music Transformer

Last update: Jul 13, 2022

Related tags

Deep Learning VMT

Overview

VMT

Video-Music Transformer (VMT) is an attention-based multi-modal model, which generates piano music for a given video.

Paper

https://arxiv.org/abs/2112.15320

Demo

Here is an example video fragments from our dataset. Note that we do not do any post-production. Each file is made from the original video with a WAVE file converted from the MIDI of the model output.

Original

The original music of the videos.

100-001_original.mp4

VMT

The music generated by our VMT model.

100-001_vmt.mp4

Seq2Seq

The musics generated by the baseline Seq2Seq model.

100-001_seq2seq.mp4

Owner

Chin-Tung Lin

GitHub Repository

Jarvis Project is a basic virtual assistant that uses TensorFlow for learning.

Jarvis_proyect Jarvis Project is a basic virtual assistant that uses TensorFlow for learning. Latest version 0.1 Features: Good morning protocol Tell

3 Aug 31, 2022

The Rich Get Richer: Disparate Impact of Semi-Supervised Learning

The Rich Get Richer: Disparate Impact of Semi-Supervised Learning Preprocess file of the dataset used in implicit sub-populations: (Demographic groups

[email protected]"> 4 Oct 14, 2022

A novel pipeline framework for multi-hop complex KGQA task. About the paper title: Improving Multi-hop Embedded Knowledge Graph Question Answering by Introducing Relational Chain Reasoning

Rce-KGQA A novel pipeline framework for multi-hop complex KGQA task. This framework mainly contains two modules, answering_filtering_module and relati

16 Nov 18, 2022

Effective Use of Transformer Networks for Entity Tracking

Effective Use of Transformer Networks for Entity Tracking (EMNLP19) This is a PyTorch implementation of our EMNLP paper on the effectiveness of pre-tr

5 Nov 06, 2021

The official repository for paper ''Domain Generalization for Vision-based Driving Trajectory Generation'' submitted to ICRA 2022

DG-TrajGen The official repository for paper ''Domain Generalization for Vision-based Driving Trajectory Generation'' submitted to ICRA 2022. Our Meth

25 Sep 26, 2022

Simulation of the solar system using various nummerical methods

solar-system Simulation of the solar system using various nummerical methods Download the repo Make shure matplotlib, scipy etc. are installed execute

7 Jul 15, 2022

Classify bird species based on their songs using SIamese Networks and 1D dilated convolutions.

The goal is to classify different birds species based on their songs/calls. Spectrograms have been extracted from the audio samples and used as features for classification.

9 Dec 27, 2022

Supervised Sliding Window Smoothing Loss Function Based on MS-TCN for Video Segmentation

SSWS-loss_function_based_on_MS-TCN Supervised Sliding Window Smoothing Loss Function Based on MS-TCN for Video Segmentation Supervised Sliding Window

3 Aug 03, 2022

banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services.

banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services. This library is developed by Bandit ML and ex-authors of Facebook's app

51 Dec 22, 2022

UMT is a unified and flexible framework which can handle different input modality combinations, and output video moment retrieval and/or highlight detection results.

Unified Multi-modal Transformers This repository maintains the official implementation of the paper UMT: Unified Multi-modal Transformers for Joint Vi

84 Jan 04, 2023

Video-Music Transformer

Related tags

Overview

VMT

Paper

Demo

Original

VMT

Seq2Seq

Owner

Chin-Tung Lin

Jarvis Project is a basic virtual assistant that uses TensorFlow for learning.

The Rich Get Richer: Disparate Impact of Semi-Supervised Learning

A novel pipeline framework for multi-hop complex KGQA task. About the paper title: Improving Multi-hop Embedded Knowledge Graph Question Answering by Introducing Relational Chain Reasoning

Effective Use of Transformer Networks for Entity Tracking

The official repository for paper ''Domain Generalization for Vision-based Driving Trajectory Generation'' submitted to ICRA 2022

Simulation of the solar system using various nummerical methods

Classify bird species based on their songs using SIamese Networks and 1D dilated convolutions.

Supervised Sliding Window Smoothing Loss Function Based on MS-TCN for Video Segmentation

banditml is a lightweight contextual bandit & reinforcement learning library designed to be used in production Python services.

A multilingual version of MS MARCO passage ranking dataset

Code for "FPS-Net: A convolutional fusion network for large-scale LiDAR point cloud segmentation".

Variational Attention: Propagating Domain-Specific Knowledge for Multi-Domain Learning in Crowd Counting (ICCV, 2021)

Synthetic LiDAR sequential point cloud dataset with point-wise annotations

converts nominal survey data into a numerical value based on a dictionary lookup.

EASY - Ensemble Augmented-Shot Y-shaped Learning: State-Of-The-Art Few-Shot Classification with Simple Ingredients.

Accepted at ICCV-2021: Workshop on Computer Vision for Automated Medical Diagnosis (CVAMD)

PyTorch implementation of the TTC algorithm

Unofficial pytorch implementation for Self-critical Sequence Training for Image Captioning. and others.

Leveraging Two Types of Global Graph for Sequential Fashion Recommendation, ICMR 2021

UMT is a unified and flexible framework which can handle different input modality combinations, and output video moment retrieval and/or highlight detection results.