A Low Complexity Speech Enhancement Framework for Full-Band Audio (48kHz) based on Deep Filtering.

Related tags

Deep LearningDeepFilterNet
Overview

DeepFilterNet

A Low Complexity Speech Enhancement Framework for Full-Band Audio (48kHz) based on Deep Filtering.

  • libDF contains Rust code used for data loading and augmentation.
  • DeepFilterNet contains Python code including a libDF wrapper for data loading, DeepFilterNet training, testing and visualization.
  • models contains DeepFilterNet model weights and config.

Usage

System requirements are cargo and pip (Rust and Python package managers). Usage of a conda or virtualenv recommended. This framework is currently only tested under Linux.

Installation of python dependencies and libDF:

cd path/to/DeepFilterNet/  # cd into repository
# Recommended: Install or activate a python env.
pip install maturin  # Used to compile libDF and load
maturin build --release -m DeepFilterNet/Cargo.toml  # Build python wheel
# Install python wheel. Make sure to specify the correct DeepFilterNet and python version
pip install target/wheels/DeepFilterNet-0.1.0-cp39-cp39-linux_x86_64.whl
# Optional: Install cuda version of pytorch from pytorch.org
pip install -r requirements.txt  # Install remaining dependencies

To enhance noisy audio files using DeepFilterNet run

# usage: enhance.py [-h] [--output-dir OUTPUT_DIR] model_base_dir noisy_audio_files [noisy_audio_files ...]
python DeepFilterNet/df/enhance.py models/DeepFilterNet/ path/to/noisy_audio.wav

License

DeepFilterNet is free and open source! All code in this repository is dual-licensed under either:

at your option. This means you can select the license you prefer!

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Comments
  • when i training with MS-SNSD dataset, and prepared the data correctly.it aways get error: RuntimeError: DF dataloader error: ThreadJoinError(

    when i training with MS-SNSD dataset, and prepared the data correctly.it aways get error: RuntimeError: DF dataloader error: ThreadJoinError("Any { .. }")

    config, and logs as below：

    cat base_dir/config.ini [train] seed = 42 device = model = deepfilternet3 mask_only = False df_only = False jit = False batch_size = 1 batch_size_eval = 0 overfit = False num_workers = 0 max_sample_len_s = 5.0 num_prefetch_batches = 32 global_ds_sampling_f = 1.0 dataloader_snrs = -5,0,5,10,20,40 batch_size_scheduling = max_epochs = 10 validation_criteria = loss validation_criteria_rule = min early_stopping_patience = 5 start_eval = False

    [df] sr = 48000 fft_size = 960 hop_size = 480 nb_erb = 32 nb_df = 96 norm_tau = 1 lsnr_max = 35 lsnr_min = -15 min_nb_erb_freqs = 2 df_order = 5 df_lookahead = 0 pad_mode = input

    [deepfilternet] conv_lookahead = 0 conv_ch = 16 conv_depthwise = True convt_depthwise = True conv_kernel = 1,3 conv_kernel_inp = 3,3 emb_hidden_dim = 256 emb_num_layers = 2 emb_gru_skip = none df_hidden_dim = 256 df_gru_skip = none df_pathway_kernel_size_t = 1 enc_concat = False df_num_layers = 3 df_n_iter = 1 linear_groups = 1 enc_linear_groups = 16 mask_pf = False

    [distortion] p_reverb = 0.2 p_bandwidth_ext = 0.0 p_clipping = 0.0 p_zeroing = 0.0 p_air_absorption = 0.0

    [optim] lr = 0.0005 momentum = 0 weight_decay = 0.05 optimizer = adamw lr_min = 1e-06 lr_warmup = 0.0001 warmup_epochs = 3 lr_cycle_mul = 1.0 lr_cycle_decay = 0.5 lr_cycle_epochs = -1 weight_decay_end = -1

    [maskloss] factor = 0 mask = iam gamma = 0.6 gamma_pred = 0.6 f_under = 2

    [spectralloss] factor_magnitude = 0 factor_complex = 0 factor_under = 1 gamma = 1

    [multiresspecloss] factor = 0 factor_complex = 0 gamma = 1 fft_sizes = 512,1024,2048

    [sdrloss] factor = 0

    [localsnrloss] factor = 0.0005

    logs: python df/train.py dataset_small.cfg data-dir-small/ base_dir_small/ 2022-12-21 15:31:00 | INFO | DF | Running on torch 1.12.0+cu102 2022-12-21 15:31:00 | INFO | DF | Running on host research-bj-green-x4-0 fatal: not a git repository (or any of the parent directories): .git 2022-12-21 15:31:00 | INFO | DF | Loading model settings of base_dir_small 2022-12-21 15:31:00 | INFO | DF | Running on device cuda:0 2022-12-21 15:31:00 | INFO | DF | Initializing model deepfilternet3 2022-12-21 15:31:03 | INFO | DF | Initializing dataloader with data directory data-dir-small/ 2022-12-21 15:31:03 | INFO | DF | Loading HDF5 key cache from .cache_dataset_small.cfg 2022-12-21 15:31:03 | INFO | DF | Start train epoch 0 with batch size 1 2022-12-21 15:31:04 | INFO | DF | [0] [0/9] | loss: 0.28417 | lr: 1.000E-04 thread '' panicked at 'assertion failed: k <= self.len()', /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/slice/mod.rs:3094:9 note: run with RUST_BACKTRACE=1 environment variable to display a backtrace 2022-12-21 15:31:04 | ERROR | DF | An error has been caught in function '', process 'MainProcess' (29909), thread 'MainThread' (140548370358784): Traceback (most recent call last):

    File "df/train.py", line 638, in main() └ <function main at 0x7fd3e3bf40d0>

    File "df/train.py", line 248, in main train_loss = run_epoch( └ <function run_epoch at 0x7fd3e3bfc040>

    File "df/train.py", line 355, in run_epoch for i, batch in enumerate(loader.iter_epoch(split, seed)): │ │ │ │ │ └ 0 │ │ │ │ └ 'train' │ │ │ └ <function PytorchDataLoader.iter_epoch at 0x7fd3e3bf4e50> │ │ └ <libdfdata.torch_dataloader.PytorchDataLoader object at 0x7fd3dc59a5b0> │ └ Batch of size 1: │ SNRs: 40 │ Gains: -6 │
    └ 0

    File "/home/xxx/code/deepfilter/DeepFilterNet-0.4.0/pyDF-data/libdfdata/torch_dataloader.py", line 283, in iter_epoch batch = self._get_batch() │ └ <function PytorchDataLoader._get_batch at 0x7fd3e3bf4d30> └ <libdfdata.torch_dataloader.PytorchDataLoader object at 0x7fd3dc59a5b0>

    File "/home/xxx/code/deepfilter/DeepFilterNet-0.4.0/pyDF-data/libdfdata/torch_dataloader.py", line 232, in _get_batch _, batch = self.data_queue.get() │ │ └ <function PytorchDataLoader._get_worker_queue_dummy.._Queue.get at 0x7fd3da327310> │ └ <libdfdata.torch_dataloader.PytorchDataLoader._get_worker_queue_dummy.._Queue object at 0x7fd3dca65640> └ <libdfdata.torch_dataloader.PytorchDataLoader object at 0x7fd3dc59a5b0>

    File "/home/xxx/code/deepfilter/DeepFilterNet-0.4.0/pyDF-data/libdfdata/torch_dataloader.py", line 214, in get self.loader.cleanup() │ │ └ <method 'cleanup' of 'builtins._FdDataLoader' objects> │ └ <builtins._FdDataLoader object at 0x7fd3dc730100> └ <libdfdata.torch_dataloader.PytorchDataLoader object at 0x7fd3dc59a5b0>

    RuntimeError: DF dataloader error: ThreadJoinError("Any { .. }")

    opened by BWMa 27
  • Question for training

    Question for training

    Hi Rikorose,

    Sorry to bother you again, I try to generate data and train the model according to the training part.

    I generated the training_set.txt (just select 10 files for test.) for speech and made the hdf5.(and so on for noise). Use python df/prepare_data.py --sr 48000 speech training_set.txt TRAIN_SET_SPEECH.hdf5.

    ~/DeepFilterNet/wav/dataset/oblomov_s009036.wav
~/DeepFilterNet/wav/dataset/oblomov_s009040.wav  
~/DeepFilterNet/wav/dataset/oblomov_s009033.wav     
~/DeepFilterNet/wav/dataset/oblomov_s009037.wav    
~/DeepFilterNet/wav/dataset/oblomov_s009041.wav  
~/DeepFilterNet/wav/dataset/oblomov_s009034.wav    
~/DeepFilterNet/wav/dataset/oblomov_s009038.wav     
~/DeepFilterNet/wav/dataset/oblomov_s009042.wav  
~/DeepFilterNet/wav/dataset/oblomov_s009035.wav     
~/DeepFilterNet/wav/dataset/oblomov_s009039.wav

    Generate the dataset.cfg as shown below,

    {
 "train": [
    [
      "~/DeepFilterNet/DeepFilterNet-github/DeepFilterNet/hdf5/TRAIN_SET_SPEECH.hdf5",
      1.0
    ],
    [
      "~/DeepFilterNet/DeepFilterNet-github/DeepFilterNet/hdf5/TRAIN_SET_NOISE.hdf5",
      1.0
    ]
  ],
  "valid": [
    [
      "~/DeepFilterNet/DeepFilterNet-github/DeepFilterNet/hdf5/TRAIN_SET_SPEECH.hdf5",
      0.2
    ],
    [
      "~/DeepFilterNet/DeepFilterNet-github/DeepFilterNet/hdf5/TRAIN_SET_NOISE.hdf5",
      0.2
    ]
  ],
  "test": [
    [
      "~/DeepFilterNet/DeepFilterNet-github/DeepFilterNet/hdf5/TRAIN_SET_SPEECH.hdf5",
      0.2
    ],
    [
      "~/DeepFilterNet/DeepFilterNet-github/DeepFilterNet/hdf5/TRAIN_SET_NOISE.hdf5",
      0.2
    ]
  ]
}

    Encounter some error as shown in the figure below, 擷取

    In addition, I have some questions:

    1. In the command python df/train.py dataset.cfg ~/wav_folder/ ./base_dir/, -Is data_dir a wav folder or an hdf5 folder? (I think is hdf5 folder) -Can base_dir/ not exist? (But we need to give config.ini, so here I enter pertrained_model/ and delete .ckpt)
    2. I found that the log says dataloader len:0, is this a problem?
    3. I remove the all 'df.' of each file while import (ex. from df.config import ... -> from config import ...), otherwise it will cause an import error.

    Thanks,

    opened by aaronhsueh0506 25
  • Reproducing results

    Reproducing results

    Hi,

    I had tried to re-train the deepfilternet model using the DNS-3 challenge dataset mentioned in your work.

    I don't have the additional 10k IR. However, the other dataset remains the same.

    On VCTK test set, using the config.ini in the pre-trained model as my training config, my "best model" on validation gives PESQ score of 2.60. It is much lower than 2.81 from the pre-trained model.

    In config.ini, Adamw is used, while in the paper Adam as optimizer is mentioned.

    Do you think any other factors would result in such a performance drop?

    Could you clarify on the 3 s sample for training? Suppose the DNS-3 sample has 10 s in a sample, do I need to split it into 3 s segments so as to utilize the entire train clip? Or just use the first 3 seconds of the clip? Alternatively, is random 3 s generated on-the-fly while training?

    In the hdf5 setup, does the speech/noise/rir need to have sample number of samples? Or is the noise and RIR sampled randomly from a list? For example, if the speech list has 1000 samples, noise list is 100 samples and rir list is 100 samples, is it okay? or should it be 1000 speech, 1000 noise, 1000 rir? Is it needed to make the duration of speech and noise samples to be the same?

    How about the reverberation parameter p_reverb = 0.05? The data augmentation is performed by default or any other config is needed? conv_lookahead = 2 in config.ini. But the paper mentions "look-ahead of l = 1 frame for both DF as well as in the DNN convolutions".

    stale 
    opened by rohithmars 15
  • RuntimeError: DF dataloader error: ThreadJoinError(

    RuntimeError: DF dataloader error: ThreadJoinError("Any { .. }") when attempting to train DeepFilterNet

    I am trying to train DeepFilterNet but I am running into the following error when training: RuntimeError: DF dataloader error: ThreadJoinError("Any { .. }").

    What I did

    I installed DeepFilterNet via PyPI using:

    pip3 install torch torchvision torchaudio
pip install deepfilternet[train]

    Then I generated the following HDF5 dataset files:

    TEST_SET_NOISE.hdf5
TEST_SET_SPEECH.hdf5
TRAIN_SET_NOISE.hdf5
TRAIN_SET_SPEECH.hdf5
VALID_SET_NOISE.hdf5
VALID_SET_SPEECH.hdf5

    and placed them into the same directory .../data_folder.

    I also created a config.ini file with the following contents:

    config.ini 

    [deepfilternet]
emb_hidden_dim = 256
df_hidden_dim = 256
df_num_layers = 3
conv_ch = 16
conv_lookahead = 0
conv_depthwise = True
convt_depthwise = True
conv_kernel = 1,3
conv_kernel_inp = 3,3
emb_num_layers = 2
emb_gru_skip = none
df_gru_skip = none
df_pathway_kernel_size_t = 1
enc_concat = False
df_n_iter = 1
linear_groups = 1
enc_linear_groups = 16
mask_pf = False
gru_type = grouped
gru_groups = 1
group_shuffle = True
dfop_method = real_unfold
df_output_layer = linear

[df]
fft_size = 960
nb_erb = 32
nb_df = 96
sr = 48000
hop_size = 480
norm_tau = 1
lsnr_max = 35
lsnr_min = -15
min_nb_erb_freqs = 2
df_order = 5
df_lookahead = 0
pad_mode = input

[train]
model = deepfilternet2
batch_size = 16
batch_size_eval = 16
num_workers = 4
overfit = false
lr = 0.001
max_epochs = 30
seed = 42
device = cuda:1
mask_only = False
df_only = False
jit = False
max_sample_len_s = 5.0
num_prefetch_batches = 32
global_ds_sampling_f = 1.0
dataloader_snrs = -5,0,5,10,20,40
batch_size_scheduling = 
validation_criteria = loss
validation_criteria_rule = min
early_stopping_patience = 5
start_eval = False

[distortion]
p_reverb = 0.0
p_bandwidth_ext = 0.0
p_clipping = 0.0
p_air_absorption = 0.0

[optim]
lr = 0.0005
momentum = 0
weight_decay = 0.05
optimizer = adamw
lr_min = 1e-06
lr_warmup = 0.0001
warmup_epochs = 3
lr_cycle_mul = 1.0
lr_cycle_decay = 0.5
lr_cycle_epochs = -1
weight_decay_end = -1

[maskloss]
factor = 0
mask = iam
gamma = 0.6
gamma_pred = 0.6
f_under = 2

[spectralloss]
factor_magnitude = 0
factor_complex = 0
factor_under = 1
gamma = 1

[multiresspecloss]
factor = 0
factor_complex = 0
gamma = 1
fft_sizes = 512,1024,2048

[sdrloss]
factor = 0

[localsnrloss]
factor = 0.0005

    and put it in a directory .../log_folder.

    I also created a dataset.cfg file with the following contents:

    dataset.cfg 

    {
  "train": [
    [
      "TRAIN_SET_SPEECH.hdf5",
      1.0
    ],
    [
      "TRAIN_SET_NOISE.hdf5",
      1.0
    ]
  ],
  "valid": [
    [
      "VALID_SET_SPEECH.hdf5",
      1.0
    ],
    [
      "VALID_SET_NOISE.hdf5",
      1.0
    ]
  ],
  "test": [
    [
      "TEST_SET_SPEECH.hdf5",
      1.0
    ],
    [
      "TEST_SET_NOISE.hdf5",
      1.0
    ]
  ]
}

    Lastly, I ran the command to train the model:

    python df/train.py path/to/dataset.cfg .../data_folder .../log_folder

    and this is the output that I get:

    2022-11-22 22:10:07 | INFO     | DF | Running on torch 1.13.0+cu117
2022-11-22 22:10:07 | INFO     | DF | Running on host workstation3
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
2022-11-22 22:10:07 | INFO     | DF | Loading model settings of log_folder
2022-11-22 22:10:07 | INFO     | DF | Running on device cuda:1
2022-11-22 22:10:07 | INFO     | DF | Initializing model `deepfilternet2`
2022-11-22 22:10:08 | DEPRECATED | DF | Use of `linear` for `df_ouput_layer` is marked as deprecated.
2022-11-22 22:10:08 | INFO     | DF | Initializing dataloader with data directory /home/tester/bokleong/dl_project_dfn/data_folder/
2022-11-22 22:10:08 | INFO     | DF | Loading HDF5 key cache from /home/tester/bokleong/dl_project_dfn/.cache_dataset.cfg
2022-11-22 22:10:08 | INFO     | DF | Start train epoch 0 with batch size 32
thread 'DataLoader Worker 1' panicked at 'called `Option::unwrap()` on a `None` value', libDF/src/dataset.rs:1134:69
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'DataLoader Worker 0' panicked at 'called `Option::unwrap()` on a `None` value', libDF/src/dataset.rs:1134:69
thread 'DataLoader Worker 2' panicked at 'called `Option::unwrap()` on a `None` value', libDF/src/dataset.rs:1134:69
thread 'DataLoader Worker 3' panicked at 'called `Option::unwrap()` on a `None` value', libDF/src/dataset.rs:1134:69
Exception in thread PinMemoryLoop:
Traceback (most recent call last):
  File "/home/tester/miniconda3/envs/user5/lib/python3.9/threading.py", line 980, in _bootstrap_inner
    self.run()
  File "/home/tester/miniconda3/envs/user5/lib/python3.9/threading.py", line 917, in run
    self._target(*self._args, **self._kwargs)
  File "/home/tester/miniconda3/envs/user5/lib/python3.9/site-packages/torch/utils/data/_utils/pin_memory.py", line 49, in _pin_memory_loop
    do_one_step()
  File "/home/tester/miniconda3/envs/user5/lib/python3.9/site-packages/torch/utils/data/_utils/pin_memory.py", line 26, in do_one_step
    r = in_queue.get(timeout=MP_STATUS_CHECK_INTERVAL)
  File "/home/tester/miniconda3/envs/user5/lib/python3.9/site-packages/libdfdata/torch_dataloader.py", line 206, in get
    self.loader.cleanup()
RuntimeError: DF dataloader error: ThreadJoinError("Any { .. }")

    Appreciate any help on this, thank you!

    opened by AnonymousEliforp 13
  • Unable to run deepFIlter form CLI

    Unable to run deepFIlter form CLI

    Hi guys, I have followed the installation instruction using pip and now I'm stuck with SLURM and I don't know how to fix it

    I'm executing deepfilternet from colab pro+ account.

    Here is the issue.

    Traceback (most recent call last): File "/usr/local/bin/deepFilter", line 5, in from df.enhance import run File "/usr/local/lib/python3.7/dist-packages/df/enhance.py", line 18, in from df.logger import init_logger, warn_once File "/usr/local/lib/python3.7/dist-packages/df/logger.py", line 49 if (jobid := os.getenv("SLURM_JOB_ID")) is not None: ^ SyntaxError: invalid syntax

    This is which I get while trying to execute this :-> !deepFilter /content/test_audio_053830.wav --output-dir /content

    Anyone got into this kind of issue??

    DO let me know the solution / how can I run this.

    opened by Anand195 13
  • Converter to .onnx model

    Converter to .onnx model

    Hi, I've been trying to use export.py function to convert retrained model to single onnx file, but it seems that there are version mismatch of torch and onnx operation. After some modifications I've almost made it work, but now I'm stuck with

    Traceback (most recent call last):
  File "/data/code_jb/deepfilter2_git/DeepFilterNet/df/export.py", line 327, in <module>
    main(args)
  File "/data/code_jb/deepfilter2_git/DeepFilterNet/df/export.py", line 292, in main
    export(
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context                                                                                                                        
    return func(*args, **kwargs)
  File "/data/code_jb/deepfilter2_git/DeepFilterNet/df/export.py", line 160, in export
    export_impl(
  File "/data/code_jb/deepfilter2_git/DeepFilterNet/df/export.py", line 100, in export_impl
    torch.onnx.export(
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/onnx/__init__.py", line 350, in export
    return utils.export(
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/onnx/utils.py", line 163, in export
    _export(
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/onnx/utils.py", line 1074, in _export
    graph, params_dict, torch_out = _model_to_graph(
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/onnx/utils.py", line 731, in _model_to_graph
    graph = _optimize_graph(
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/onnx/utils.py", line 308, in _optimize_graph
    graph = _C._jit_pass_onnx(graph, operator_export_type)
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/onnx/__init__.py", line 416, in _run_symbolic_function                                                                                                                      
    return utils._run_symbolic_function(*args, **kwargs)
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/onnx/utils.py", line 1421, in _run_symbolic_function                                                                                                                        
    raise symbolic_registry.UnsupportedOperatorError(
torch.onnx.symbolic_registry.UnsupportedOperatorError: Exporting the operator ::view_as_complex to ONNX opset version 14 is not supported. Please feel free to request support or submit a pull request on PyTorch GitHub.

    Am I doing something wrong or the latest Deepfilternet2 modifications was not tested with this converter? Or maybe I should use specific versions of onnxruntime/torch?

    opened by JBloodless 12
  • Noise of typing not working well

    Noise of typing not working well

    Hi Rikorose,

    Thanks for working on version 2 of Deepfilternet. Now I can do the real-time inference process with buffer size=1, which is the same as the full signal effect. The point is that the state of the RNN needs to be inherited.

    Now I'm having a trouble with typing/keyboard noise not working well. But I only use spectral loss with c=0.3 in Deepfilternet2 now, will multi-resolution loss improve in this case? or maybe c=0.6 in preious work is better?

    Thanks, Aaron

    opened by aaronhsueh0506 12
  • Implement a real-time loop for DeepFilterNet

    Implement a real-time loop for DeepFilterNet

    Hi Hendrik, Just curious as didn't see any benchmarks but could it run chunking audio as it goes or does it need the overall file to analyse? How does it compare to https://github.com/breizhn/DTLN?

    Thanks Stuart

    enhancement pinned 
    opened by StuartIanNaylor 10
  • All Cargo Tests pass, but train.py returns error

    All Cargo Tests pass, but train.py returns error

    I have been struggling with the same error for a while now...

    I am using a conda environment in linux on a HPC solution. I previously had problems with Rust detecting multiple versions of HDF5 and (I think) I fixed it by defining HDF5_VERSION being the same as library and header (version 1.10.6) and HDF5_DIR as conda environment root. Following the instructions on https://github.com/aldanor/hdf5-rust, I also do

    $ conda env config vars set RUSTFLAGS="-C link-args=-Wl,-rpath,$HDF5_DIR/lib"

    And add the directory to path: $ conda develop /zhome/a7/0/155527/Desktop/s204161/DEEPL_PLS/DeepFilterNet-main/DeepFilterNet/

    Doing this, I manage to succesfully pass the cargo tests (It no longer complains about HDF5 version used by rust not being the same as the one in h5py):

    (DEEPL_PLS) cargo test Finished test [optimized + debuginfo] target(s) in 2.06s Running unittests src/lib.rs (target/debug/deps/libdfdata-beeffcf7e03d4848)

    running 0 tests

    test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src/lib.rs (target/debug/deps/libdf-9cd65e9fa81eadda)

    running 0 tests

    test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src/lib.rs (target/debug/deps/deep_filter_ladspa-c16148d79da4c5bf)

    running 0 tests

    test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src/lib.rs (target/debug/deps/df-f01c984070e59647)

    running 30 tests test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_06 - should panic ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_05 - should panic ... ok test tests::test_erb_inout ... ok test reexport_dataset_modules::dataset::tests::test_mix_audio_signal ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_07 - should panic ... ok test transforms::tests::test_find_max_abs ... ok test reexport_dataset_modules::augmentations::tests::test_rand_resample ... ok test transforms::tests::test_stft_istft_delay ... ok test reexport_dataset_modules::augmentations::tests::test_low_pass ... ok test reexport_dataset_modules::augmentations::tests::test_clipping ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_10 ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_read_pcm ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_04 - should panic ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_02 ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_01 ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_03 ... ok test reexport_dataset_modules::augmentations::tests::test_gen_noise ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_08 ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_09 ... ok test transforms::tests::test_ext_bandwidth_spectral ... ok test reexport_dataset_modules::augmentations::tests::test_reverb ... ok test reexport_dataset_modules::dataset::tests::test_fft_dataset ... ok test reexport_dataset_modules::augmentations::tests::test_filters ... ok test reexport_dataset_modules::augmentations::tests::test_compose ... ok test reexport_dataset_modules::dataset::tests::test_td_dataset ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_read_flac ... ok test transforms::tests::test_estimate_bandwidth ... ok test reexport_dataset_modules::augmentations::tests::test_air_absorption ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_read_vorbis ... ok test reexport_dataset_modules::dataloader::tests::test_data_loader ... ok

    test result: ok. 30 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 6.37s

    Doc-tests df

    running 0 tests

    test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

    However when I cd into the repo and run for training it gives the error

    python DeepFilterNet/df/train.py /work3/s204161/config.cfg /work3/s204161/formatted_data/ /zhome/a7/0/155527/Desktop/s204161/DEEPL_PLS/DeepFilterNet/ --debug 2022-11-21 05:42:10 | INFO | df.logger:init_logger:44 | Running on torch 1.12.0 2022-11-21 05:42:10 | INFO | df.logger:init_logger:45 | Running on host n-62-27-19 fatal: not a git repository (or any parent up to mount point /) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). 2022-11-21 05:42:10 | INFO | df.logger:init_logger:67 | Loading model settings of DeepFilterNet 2022-11-21 05:42:10 | INFO | main:main:94 | Running on device cpu 2022-11-21 05:42:10 | INFO | df.model:init_model:21 | Initializing model deepfilternet3 2022-11-21 05:42:10 | INFO | libdfdata.torch_dataloader:init:99 | Initializing dataloader with data directory /work3/s204161/formatted_data/ 2022-11-21 05:42:10 | ERROR | main::633 | An error has been caught in function '', process 'MainProcess' (16810), thread 'MainThread' (139678983296832): Traceback (most recent call last):

    File "/zhome/a7/0/155527/Desktop/s204161/DEEPL_PLS/DeepFilterNet-main/DeepFilterNet/df/train.py", line 633, in main() └ <function main at 0x7f08f2c37b50>

    File "/zhome/a7/0/155527/Desktop/s204161/DEEPL_PLS/DeepFilterNet-main/DeepFilterNet/df/train.py", line 139, in main dataloader = DataLoader( └ <class 'libdfdata.torch_dataloader.PytorchDataLoader'>

    File "/zhome/a7/0/155527/ENTER/envs/DEEPL_PLS/lib/python3.10/site-packages/libdfdata/torch_dataloader.py", line 101, in init self.loader = _FdDataLoader( │ └ <class 'builtins._FdDataLoader'> └ <libdfdata.torch_dataloader.PytorchDataLoader object at 0x7f08f1dc6b60>

    RuntimeError: DF dataset error: Hdf5ErrorDetail { source: H5Fopen(): unable to open file: bad superblock version number, msg: "Error during File::open of dataset /work3/s204161/formatted_data/VALID_SET_SPEECH.hdf5" }

    I have no idea what is left to do. I do not have any prior rust experience. I've attached conda_info.txt should it be relevant and when I run the script:

    import h5py hdfFile = h5py.File('/work3/s204161/formatted_data/TRAIN_SET_SPEECH.hdf5', 'r') print(h5py.version.info) for name in hdfFile: print(hdfFile[name]) print(hdfFile.keys())

    Summary of the h5py configuration

    h5py 3.7.0 HDF5 1.10.6 Python 3.10.8 (main, Nov 4 2022, 13:48:29) [GCC 11.2.0] sys.platform linux sys.maxsize 9223372036854775807 numpy 1.21.5 cython (built with) 0.29.30 numpy (built against) 1.21.5 HDF5 (built against) 1.10.6 <HDF5 group "/speech" (1314 members)> <KeysViewHDF5 ['speech']>

    :(

    conda_info.txt

    opened by Student204161 9
  • About whitenoise performance

    About whitenoise performance

    Thanks for your awesome work! I have installed deepfilternet through pip and test some samples. The overall noise reduction effect is great, but in whitenoise scene, there are more residues in the speech, which leads to a poor subjective feelings. Have you noticed this phenomenon? I will attach the samples below. samples.zip

    stale 
    opened by sqummy 9
  • What is up with the canned deepFilter?

    What is up with the canned deepFilter?

    (DeepFilterNet) C:\Users\sdr\Downloads\Playground\DeepFilterNet>deepFilter test.wav Traceback (most recent call last): File "C:\Users\sdr\anaconda3\envs\DeepFilterNet\lib\runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Users\sdr\anaconda3\envs\DeepFilterNet\lib\runpy.py", line 86, in run_code exec(code, run_globals) File "C:\Users\sdr\anaconda3\envs\DeepFilterNet\Scripts\deepFilter.exe_main.py", line 7, in TypeError: main() missing 1 required positional argument: 'args'

    (DeepFilterNet) C:\Users\sdr\Downloads\Playground\DeepFilterNet>

    stale 
    opened by Joeweav 9
  • build(deps): bump actions/stale from 6 to 7

    build(deps): bump actions/stale from 6 to 7

    Bumps actions/stale from 6 to 7.

    Release notes

    Sourced from actions/stale's releases.

    v7.0.0

    ⚠️ This version contains breaking changes ⚠️

    What's Changed

    Breaking Changes

    • In this release we prevent this action from managing the stale label on items included in exempt-issue-labels and exempt-pr-labels
    • We decided that this is outside of the scope of this action, and to be left up to the maintainer

    New Contributors

    Full Changelog: https://github.com/actions/stale/compare/v6...v7.0.0

    v6.0.1

    Update @​actions/core to 1.10.0 #839

    Full Changelog: https://github.com/actions/stale/compare/v6.0.0...v6.0.1

    Changelog

    Sourced from actions/stale's changelog.

    Changelog

    [7.0.0]

    :warning: Breaking change :warning:

    [6.0.1]

    Update @​actions/core to v1.10.0 (#839)

    [6.0.0]

    :warning: Breaking change :warning:

    Issues/PRs default close-issue-reason is now not_planned(#789)

    [5.1.0]

    Don't process stale issues right after they're marked stale [Add close-issue-reason option]#764#772 Various dependabot/dependency updates

    4.1.0 (2021-07-14)

    Features

    4.0.0 (2021-07-14)

    Features

    Bug Fixes

    • dry-run: forbid mutations in dry-run (#500) (f1017f3), closes #499
    • logs: coloured logs (#465) (5fbbfba)
    • operations: fail fast the current batch to respect the operations limit (#474) (5f6f311), closes #466
    • label comparison: make label comparison case insensitive #517, closes #516
    • filtering comments by actor could have strange behavior: "stale" comments are now detected based on if the message is the stale message not who made the comment(#519), fixes #441, #509, #518

    Breaking Changes

    ... (truncated)

    Commits
    • 6f05e42 draft release for v7.0.0 (#888)
    • eed91cb Update how stale handles exempt items (#874)
    • 10dc265 Merge pull request #880 from akv-platform/update-stale-repo
    • 9c1eb3f Update .md files and allign build-test.yml with the current test.yml
    • bc357bd Update .github/workflows/release-new-action-version.yml
    • 690ede5 Update .github/ISSUE_TEMPLATE/bug_report.md
    • afbcabf Merge branch 'main' into update-stale-repo
    • e364411 Update name of codeql.yml file
    • 627cef3 fix print outputs step (#859)
    • 975308f Merge pull request #876 from jongwooo/chore/use-cache-in-check-dist
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies github_actions 
    opened by dependabot[bot] 0
  • When I retrain the model by models/DeepFilterNet2/config.ini, the result is strange~

    When I retrain the model by models/DeepFilterNet2/config.ini, the result is strange~

    Thanks for your awesome work! I'm trying to recurrence the training through models/DeepFilterNet2/config.ini, but when I use DeepFilterNet/df/enhance.py to test the retrained model, output wavs is nearly to silence. This is my retraining step:Thanks for your awesome work! I'm trying to recurrence the training through models/DeepFilterNet2/config.ini, but when I use DeepFilterNet/df/enhance.py to test the retrained model, output wavs is nearly to silence. This is my retraining step: (1) copy the config.ini of DeepFilterNet2

        cd DeepFilterNet
    mkdir base_dir
    cp ../models/DeepFilterNet2/config.ini ./base_dir

    (2) preparing data take VocalSet_48kHz_mono_000_NA_NA.tar.bz2 as clean data take noise_fullband/datasets_fullband.noise_fullband.audioset_000.tar.bz2 as noise data

        find ../scripts/out/datasets_fullband/clean_fullband/VocalSet_48kHz_mono -iname "*.wav" > training_set_speech.txt
    python df/scripts/prepare_data.py --sr 48000 speech training_set_speech.txt ./data_dir/TRAIN_SET_SPEECH.hdf5
    find ../scripts/out/datasets_fullband/noise_fullband -iname "*.wav" > training_set_noise.txt
    python df/scripts/prepare_data.py --sr 48000 noise training_set_noise.txt ./data_dir/TRAIN_SET_NOISE.hdf5

    this is my dataset.cfg： { "train": [ [ "TRAIN_SET_SPEECH.hdf5", 1.0 ], [ "TRAIN_SET_NOISE.hdf5", 1.0 ], [ "TRAIN_SET_RIR.hdf5", 0.0 ] ], "valid": [ [ "TRAIN_SET_SPEECH.hdf5", 0.2 ], [ "TRAIN_SET_NOISE.hdf5", 0.2 ], [ "TRAIN_SET_RIR.hdf5", 0.0 ] ], "test": [ [ "TRAIN_SET_SPEECH.hdf5", 0.2 ], [ "TRAIN_SET_NOISE.hdf5", 0.2 ], [ "TRAIN_SET_RIR.hdf5", 0.0 ] ] }

    (3) start retraining python df/train.py dataset.cfg ./data_dir/ ./base_dir/

    (4) test retrained model

    python DeepFilterNet/df/enhance.py -m DeepFilterNet/base_dir2 --output-dir DeepFilterNet/test_dir_out/ DeepFilterNet/test_dir_48k/interview_48k.wav

    but the output of the reatrained model is just like this retraing_model_results

    I’m not sure which step is wrong or missed, have you ever met the same problem?

    opened by Yancongs 6
  • Multithread ?

    Multithread ?

    Hi and thanks for a amazing work ! I tested the ladspa filter on a Allwinner H3 (ARM Cortex-A7 Quad-Core 1.296 GHz) , it was easy to install; great work The filter seems to be single-threaded, a single core seems not be enough to process. Would it be possible to involve the other cores ?

    2022-12-22T16:33:37.579Z | WARN |  deep_filter_ladspa | DF 7b7326a9e80d | Underrun detected (RTF: 2.39). Processing too slow!
overrun!!! (at least 277.761 ms long)
2022-12-22T16:33:38.903Z | WARN |  deep_filter_ladspa | DF 3bfb6693797b | Underrun detected (RTF: 2.48). Processing too slow!
overrun!!! (at least 312.537 ms long)
2022-12-22T16:33:40.179Z | WARN |  deep_filter_ladspa | DF 9f61b96d4beb | Underrun detected (RTF: 2.36). Processing too slow!
overrun!!! (at least 266.241 ms long)
2022-12-22T16:33:41.476Z | WARN |  deep_filter_ladspa | DF 54a1822b409f | Underrun detected (RTF: 2.41). Processing too slow!
overrun!!! (at least 284.917 ms long)
    wontfix 
    opened by jenskastensson 2
  • build(deps): update roots requirement from 0.0.7 to 0.0.8 in /libDF

    build(deps): update roots requirement from 0.0.7 to 0.0.8 in /libDF

    Updates the requirements on roots to permit the latest version.

    Release notes

    Sourced from roots's releases.

    December 2022

    Remove debug output, fix quartic equations, allow mutable closures.

    Changelog

    Sourced from roots's changelog.

    0.0.8 - 2022-12-21

    • Remove debug print - thanks to J-F-Liu, Ralith
    • Fixed discriminant check in quartic equations - thanks to jingnanshi
    • Allowed mutable closures for secant etc. - thanks to vron

    0.0.7 - 2021-06-17

    • SearchError becomes public - thanks to JP-Ellis
    • Unnormalized cubic equations are solved using the general formula rather than trigonometrically - thanks to Logicalshift

    0.0.6 - 2019-12-22

    • Fixed cubic equations with very small a3 - thanks to Andrew Hunter
    • Improved quartic equations with multiple roots (for f64; f32 is still a problem) - thanks to Tim Lueke
    • Removed warnings of rustc 1.40.0
    • Switched benchmarks from Bencher to Criterion

    0.0.5 - 2019-01-20

    • Trait Error implemented for SearchError - thanks to phillyfan1138
    • Find roots of higher-degree polynomials using eigenvalues - thanks to stiv-yakovenko
    • Find roots of higher-degree polynomials using Sturm's theorem recursively (experimental)
    • Inverse quadratic approximation

    0.0.4 - 2017-09-05

    • Reduced the performance overhead by using generics - thanks to aepsil0n
    • Handle special cases of quadratic equations - thanks to stiv-yakovenko

    0.0.3 - 2017-03-28

    • New version of the compiler
    • Benchmarks
    • Improved the speed of the Brent-Dekker method
    • Reduced the convergency boilerplate

    0.0.2 - 2015-06-08

    • Fight against the compiler

    [0.0.1] - 2015-03-24

    • Initial version
    Commits

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies rust 
    opened by dependabot[bot] 0
  • Run tract model in half float

    Run tract model in half float

    running

    cargo run -p deep_filter --features=tract,bin,wav-utils,transforms --bin deep-filter -- --half-floats assets/noisy_snr0.wav -o out -vv -D

    results in the following stack trace:

       Compiling deep_filter v0.4.1-pre (/home/hendrik/projects/DeepFilterNet/libDF)
    Finished dev [optimized + debuginfo] target(s) in 20.20s
     Running `target/debug/deep-filter --half-floats assets/noisy_snr0.wav -o out -vv -D`
[2022-12-13T10:09:59Z INFO  df::tract] Init encoder with delay: 0
[2022-12-13T10:09:59Z INFO  tract_linalg::x86_64_fma] mmm_f32, sigmoid_f32, tanh_f32: x86_64/fma activated
[2022-12-13T10:09:59Z INFO  tract_linalg::x86_64_fma] mmm_i8_i8 and mmm_i8_i32: x86_64/avx2 activated
[2022-12-13T10:09:59Z INFO  df::tract] Init ERB decoder with delay: 0
[2022-12-13T10:09:59Z INFO  df::tract] Init DF decoder with delay: 0
[2022-12-13T10:09:59Z INFO  df::tract] Running with model type deepfilternet3 lookahead 2
Error: Evaluating #3 "/erb_conv0/1/Conv.pad" Pad

Caused by:
    Tensor datum type error: tensor is F32, accessed as F16

Stack backtrace:
   0: tract_data::tensor::Tensor::check_for_access
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-data-0.18.5/src/tensor.rs:748:13
   1: tract_data::tensor::Tensor::to_scalar
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-data-0.18.5/src/tensor.rs:848:9
   2: tract_core::ops::array::pad::Pad::eval_t
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-core-0.18.5/src/ops/array/pad.rs:34:38
      <tract_core::ops::array::pad::Pad as tract_core::ops::EvalOp>::eval
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-core-0.18.5/src/ops/array/pad.rs:109:18
   3: tract_core::plan::eval
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-core-0.18.5/src/plan.rs:491:17
   4: core::ops::function::FnMut::call_mut
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/ops/function.rs:164:5
      tract_core::plan::SimpleState<F,O,M,P>::exec_plan_with_eval
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-core-0.18.5/src/plan.rs:264:26
   5: tract_core::plan::SimpleState<F,O,M,P>::run_plan_with_eval
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-core-0.18.5/src/plan.rs:199:9
      tract_core::plan::SimpleState<F,O,M,P>::run
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-core-0.18.5/src/plan.rs:177:9
   6: df::tract::DfTract::process
             at ./libDF/src/tract.rs:401:17
   7: deep_filter::main
             at ./libDF/src/bin/enhance_wav.rs:139:13
   8: core::ops::function::FnOnce::call_once
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/ops/function.rs:248:5
      std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/sys_common/backtrace.rs:122:18
   9: std::rt::lang_start::{{closure}}
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/rt.rs:166:18
  10: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/ops/function.rs:283:13
      std::panicking::try::do_call
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:492:40
      std::panicking::try
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:456:19
      std::panic::catch_unwind
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panic.rs:137:14
      std::rt::lang_start_internal::{{closure}}
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/rt.rs:148:48
      std::panicking::try::do_call
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:492:40
      std::panicking::try
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:456:19
      std::panic::catch_unwind
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panic.rs:137:14
      std::rt::lang_start_internal
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/rt.rs:148:20
  11: main
  12: __libc_start_call_main
  13: __libc_start_main_alias_2
  14: _start
    opened by Rikorose 2
Releases(v0.4.0)
Owner
Hendrik Schröter
Hendrik Schröter
GitHub Repository
This is the official code for the paper "Ad2Attack: Adaptive Adversarial Attack for Real-Time UAV Tracking".

Ad^2Attack：Adaptive Adversarial Attack on Real-Time UAV Tracking Demo video 📹 Our video on bilibili demonstrates the test results of Ad^2Attack on se

Intelligent Vision for Robotics in Complex Environment 10 Nov 07, 2022
Creating multimodal multitask models

Fusion Brain Challenge The English version of the document can be found here. Обновления 01.11 Мы выкладываем пример данных, аналогичных private test

Sber AI 43 Nov 28, 2022
Rainbow DQN implementation that outperforms the paper's results on 40% of games using 20x less data 🌈

Rainbow 🌈 An implementation of Rainbow DQN which outperforms the paper's (Hessel et al. 2017) results on 40% of tested games while using 20x less dat

Dominik Schmidt 31 Dec 21, 2022
Pytorch implementation of "Neural Wireframe Renderer: Learning Wireframe to Image Translations"

Neural Wireframe Renderer: Learning Wireframe to Image Translations Pytorch implementation of ideas from the paper Neural Wireframe Renderer: Learning

Yuan Xue 7 Nov 14, 2022
An implementation of MobileFormer

MobileFormer An implementation of MobileFormer proposed by Yinpeng Chen, Xiyang Dai et al. Including [1] Mobile-Former proposed in:

slwang9353 62 Dec 28, 2022
StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversion

StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversion Yinghao Aaron Li, Ali Zare, Nima Mesgarani We pres

Aaron (Yinghao) Li 282 Jan 01, 2023
AirPose: Multi-View Fusion Network for Aerial 3D Human Pose and Shape Estimation

AirPose AirPose: Multi-View Fusion Network for Aerial 3D Human Pose and Shape Estimation Check the teaser video This repository contains the code of A

Robot Perception Group 41 Dec 05, 2022
Use .csv files to record, play and evaluate motion capture data.

Purpose These scripts allow you to record mocap data to, and play from .csv files. This approach facilitates parsing of body movement data in statisti

21 Dec 12, 2022
Official Pytorch and JAX implementation of "Efficient-VDVAE: Less is more"

The Official Pytorch and JAX implementation of "Efficient-VDVAE: Less is more" Arxiv preprint Louay Hazami   ·   Rayhane Mama   ·   Ragavan Thurairatn

Rayhane Mama 144 Dec 23, 2022
Code for "Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification", ECCV 2020 Spotlight

Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification Implementation of "Learning From Multiple Experts: Se

27 Nov 05, 2022
Offical implementation for "Trash or Treasure? An Interactive Dual-Stream Strategy for Single Image Reflection Separation".

Trash or Treasure? An Interactive Dual-Stream Strategy for Single Image Reflection Separation (NeurIPS 2021) by Qiming Hu, Xiaojie Guo. Dependencies P

Qiming Hu 31 Dec 20, 2022
Job-Recommend-Competition - Vectorwise Interpretable Attentions for Multimodal Tabular Data

SiD - Simple Deep Model Vectorwise Interpretable Attentions for Multimodal Tabul

Jungwoo Park 40 Dec 22, 2022
Code for DeepXML: A Deep Extreme Multi-Label Learning Framework Applied to Short Text Documents

DeepXML Code for DeepXML: A Deep Extreme Multi-Label Learning Framework Applied to Short Text Documents Architectures and algorithms DeepXML supports

Extreme Classification 49 Nov 06, 2022
Implementation of the master's thesis "Temporal copying and local hallucination for video inpainting".

Temporal copying and local hallucination for video inpainting This repository contains the implementation of my master's thesis "Temporal copying and

David Álvarez de la Torre 1 Dec 02, 2022
All of the figures and notebooks for my deep learning book, for free!

"Deep Learning - A Visual Approach" by Andrew Glassner This is the official repo for my book from No Starch Press. Ordering the book My book is called

Andrew Glassner 227 Jan 04, 2023
NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsampling

NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsampling For Official repo of NU-Wave: A Diffusion Probabilistic Model for Neural Audio Up

Rishikesh (ऋषिकेश) 38 Oct 11, 2022
Rank1 Conversation Emotion Detection Task

Rank1-Conversation_Emotion_Detection_Task accuracy macro-f1 recall 0.826 0.7544 0.719 基于预训练模型和时序预测模型的对话情感探测任务 1 摘要 针对对话情感探测任务，本文将其分为文本分类和时间序列预测两个子任务，分

Yuchen Han 2 Nov 28, 2021
A port of muP to JAX/Haiku

MUP for Haiku This is a (very preliminary) port of Yang and Hu et al.'s μP repo to Haiku and JAX. It's not feature complete, and I'm very open to sugg

18 Dec 30, 2022
Pytorch implementation of the paper: "SAPNet: Segmentation-Aware Progressive Network for Perceptual Contrastive Image Deraining"

SAPNet This repository contains the official Pytorch implementation of the paper: "SAPNet: Segmentation-Aware Progressive Network for Perceptual Contr

11 Oct 17, 2022
Use unsupervised and supervised learning to predict stocks

AIAlpha: Multilayer neural network architecture for stock return prediction This project is meant to be an advanced implementation of stacked neural n

Vivek Palaniappan 1.5k Jan 06, 2023