李云龙二次元风格化!打滚卖萌,使用了animeGANv2进行了视频的风格迁移

Overview

李云龙二次元风格化!一键star、fork,你也可以生成这样的团长!

打滚卖萌求star求fork!

0.效果展示

1.模型简介

1.1AnimeGANv2

本文使用了animeGANv2进行了视频的风格迁移。
animeGANv2,顾名思义,是其前作AnimeGAN的改进版,改进方向主要在以下4点:

  • 解决了生成的图像中的高频伪影问题。
  • 它易于训练,并能直接达到论文所述的效果。
  • 进一步减少生成器网络的参数数量。(现在生成器大小 8.17Mb)
  • 尽可能多地使用来自BD电影的新的高质量的风格数据。
    效果图参考:
    animeGANv2
    本文则是使用了paddlepaddle预训练好的animeGANv2模型对李云龙名场面视频进行了风格化迁移,详情请看下文分解。

2.实现思路

flow

3.素材准备

首先要找到自己要操作的视频素材,将视频的音频单独提取出来备用
我自己找的资源放在了codes/videos/liyunlong文件夹下,是李云龙名场面:
你咋不敢跟旅长干一架呢!→旅长我给你跪下了 名场面

4.代码实操:

话不多说,首先是环境的基本配置

  • 安装基本环境
!pip install -r codes/PaddleGAN-develop/requirements.txt
  • 导入基本环境
import paddle 
import os 
import sys 
sys.path.insert(0,'codes/PaddleGAN-develop')
from ppgan.apps import AnimeGANPredictor

5.GAN它!

友情提示:此处最好使用GPU环境,cpu推理属实是有点点慢
进行模型的推理:

使用paddlepaddle预训练好的animeGANv2模型对视频进行风格迁移:
from ppgan.apps import AnimeGANPredictor
import cv2

predictor = AnimeGANPredictor('',None,)
video_src = 'codes/videos/liyunlong/格式工厂混流 亮剑-03+亮剑-03+亮剑-04 00_00_23-.mp4'
video_ = cv2.VideoCapture(video_src)
video_name_ = os.path.basename(video_src)
total_frames = video_.get(cv2.CAP_PROP_FRAME_COUNT)
fps_ = video_.get(cv2.CAP_PROP_FPS)
print("video {}, fps:{}, total frames:{}...".format(video_name_, fps_, total_frames))
frame_count_ = 0
save_per_frames = 1
dst_dir = 'codes/videos/liyunlong/'
out_video = cv2.VideoWriter('{}/hayao_{}'.format(dst_dir, video_name_),
                                cv2.VideoWriter_fourcc(*'DIVX'), int(fps_),
                                (int(video_.get(3)), int(video_.get(4))))
print('now begin...')
while True:
    ret_, frame_ = video_.read()
    if not ret_:  # or len(fps_list_) == 0:
        print('end of video...')
        break
    result_frame = predictor.anime_image_only(frame_)
    if frame_count_ % save_per_frames == 0:
        out_video.write(result_frame)
    frame_count_ = frame_count_ + 1
    if frame_count_ % 100 == 0:
        print("{}/{} processed...".format(frame_count_, int(total_frames)), flush=False)

6.最终视频

合成最终所需要的视频:

# 合并生成的视频和之前分离的音频:
!ffmpeg -i codes/videos/liyunlong/hayao_格式工厂混流 亮剑-03+亮剑-03+亮剑-04 00_00_23-.mp4 -i codes/videos/liyunlong/音频1.aac -c:v copy -c:a aac -strict experimental codes/videos/liyunlong/李云龙二次元化.mp4

这样就大功告成啦~~~
你可以在此基础上:

  • 更换你喜欢的视频
  • 更换其他paddle预训练好的模型
  • 甚至可以尝试自己动手训练定制化的模型!

打滚卖萌求star、fork!

PaddleGAN 的基础上做了些微小的改动,鸣谢.

Owner
oukohou
Hello there.
oukohou
PyTorch Lightning + Hydra. A feature-rich template for rapid, scalable and reproducible ML experimentation with best practices. ⚡🔥⚡

Lightning-Hydra-Template A clean and scalable template to kickstart your deep learning project 🚀 ⚡ 🔥 Click on Use this template to initialize new re

Łukasz Zalewski 2.1k Jan 09, 2023
[AAAI2022] Source code for our paper《Suppressing Static Visual Cues via Normalizing Flows for Self-Supervised Video Representation Learning》

SSVC The source code for paper [Suppressing Static Visual Cues via Normalizing Flows for Self-Supervised Video Representation Learning] samples of the

7 Oct 26, 2022
A collection of metrics for evaluating timbre dissimilarity using the TorchMetrics API

Timbre Dissimilarity Metrics A collection of metrics for evaluating timbre dissimilarity using the TorchMetrics API Installation pip install -e . Usag

Ben Hayes 21 Jan 05, 2022
Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

Fairseq(-py) is a sequence modeling toolkit that allows researchers and developers to train custom models for translation, summarization, language mod

20.5k Jan 08, 2023
Pgn2tex - Scripts to convert pgn files to latex document. Useful to build books or pdf from pgn studies

Pgn2Latex (WIP) A simple script to make pdf from pgn files and studies. It's sti

12 Jul 23, 2022
Real-time face detection and emotion/gender classification using fer2013/imdb datasets with a keras CNN model and openCV.

Real-time face detection and emotion/gender classification using fer2013/imdb datasets with a keras CNN model and openCV.

Octavio Arriaga 5.3k Dec 30, 2022
Controlling Hill Climb Racing with Hand Tacking

Controlling Hill Climb Racing with Hand Tacking Opened Palm for Gas Closed Palm for Brake

Rohit Ingole 3 Jan 18, 2022
Train SN-GAN with AdaBelief

SNGAN-AdaBelief Train a state-of-the-art spectral normalization GAN with AdaBelief https://github.com/juntang-zhuang/Adabelief-Optimizer Acknowledgeme

Juntang Zhuang 10 Jun 11, 2022
TorchOk - The toolkit for fast Deep Learning experiments in Computer Vision

TorchOk - The toolkit for fast Deep Learning experiments in Computer Vision

52 Dec 23, 2022
Physical Anomalous Trajectory or Motion (PHANTOM) Dataset

Physical Anomalous Trajectory or Motion (PHANTOM) Dataset Description This dataset contains the six different classes as described in our paper[]. The

0 Dec 16, 2021
The implementation of "Bootstrapping Semantic Segmentation with Regional Contrast".

ReCo - Regional Contrast This repository contains the source code of ReCo and baselines from the paper, Bootstrapping Semantic Segmentation with Regio

Shikun Liu 128 Dec 30, 2022
GNN4Traffic - This is the repository for the collection of Graph Neural Network for Traffic Forecasting

GNN4Traffic - This is the repository for the collection of Graph Neural Network for Traffic Forecasting

564 Jan 02, 2023
[CIKM 2021] Enhancing Aspect-Based Sentiment Analysis with Supervised Contrastive Learning

Enhancing Aspect-Based Sentiment Analysis with Supervised Contrastive Learning. This repo contains the PyTorch code and implementation for the paper E

Akuchi 18 Dec 22, 2022
Keras-1D-NN-Classifier

Keras-1D-NN-Classifier This code is based on the reference codes linked below. reference 1, reference 2 This code is for 1-D array data classification

Jae-Hoon Shim 6 May 18, 2021
Source codes for the paper "Local Additivity Based Data Augmentation for Semi-supervised NER"

LADA This repo contains codes for the following paper: Jiaao Chen*, Zhenghui Wang*, Ran Tian, Zichao Yang, Diyi Yang: Local Additivity Based Data Augm

GT-SALT 36 Dec 02, 2022
Copy Paste positive polyp using poisson image blending for medical image segmentation

Copy Paste positive polyp using poisson image blending for medical image segmentation According poisson image blending I've completely used it for bio

Phạm Vũ Hùng 2 Oct 19, 2021
This PyTorch package implements MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation (NAACL 2022).

MoEBERT This PyTorch package implements MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation (NAACL 2022). Installation Create an

Simiao Zuo 34 Dec 24, 2022
Alpha-Zero - Telegram Group Manager Bot Written In Python Using Pyrogram

✨ Alpha Zero Bot ✨ Telegram Group Manager Bot + Userbot Written In Python Using

1 Feb 17, 2022
Training and Evaluation Code for Neural Volumes

Neural Volumes This repository contains training and evaluation code for the paper Neural Volumes. The method learns a 3D volumetric representation of

Meta Research 370 Dec 08, 2022
A unified framework for machine learning with time series

Welcome to sktime A unified framework for machine learning with time series We provide specialized time series algorithms and scikit-learn compatible

The Alan Turing Institute 6k Jan 08, 2023