A Low Complexity Speech Enhancement Framework for Full-Band Audio (48kHz) based on Deep Filtering.

Related tags

Deep LearningDeepFilterNet
Overview

DeepFilterNet

A Low Complexity Speech Enhancement Framework for Full-Band Audio (48kHz) based on Deep Filtering.

  • libDF contains Rust code used for data loading and augmentation.
  • DeepFilterNet contains Python code including a libDF wrapper for data loading, DeepFilterNet training, testing and visualization.
  • models contains DeepFilterNet model weights and config.

Usage

System requirements are cargo and pip (Rust and Python package managers). Usage of a conda or virtualenv recommended. This framework is currently only tested under Linux.

Installation of python dependencies and libDF:

cd path/to/DeepFilterNet/  # cd into repository
# Recommended: Install or activate a python env.
pip install maturin  # Used to compile libDF and load
maturin build --release -m DeepFilterNet/Cargo.toml  # Build python wheel
# Install python wheel. Make sure to specify the correct DeepFilterNet and python version
pip install target/wheels/DeepFilterNet-0.1.0-cp39-cp39-linux_x86_64.whl
# Optional: Install cuda version of pytorch from pytorch.org
pip install -r requirements.txt  # Install remaining dependencies

To enhance noisy audio files using DeepFilterNet run

# usage: enhance.py [-h] [--output-dir OUTPUT_DIR] model_base_dir noisy_audio_files [noisy_audio_files ...]
python DeepFilterNet/df/enhance.py models/DeepFilterNet/ path/to/noisy_audio.wav

License

DeepFilterNet is free and open source! All code in this repository is dual-licensed under either:

at your option. This means you can select the license you prefer!

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Comments
  • when i training with MS-SNSD dataset, and prepared the data correctly.it aways get error: RuntimeError: DF dataloader error: ThreadJoinError(

    when i training with MS-SNSD dataset, and prepared the data correctly.it aways get error: RuntimeError: DF dataloader error: ThreadJoinError("Any { .. }")

    config, and logs as below：

    cat base_dir/config.ini [train] seed = 42 device = model = deepfilternet3 mask_only = False df_only = False jit = False batch_size = 1 batch_size_eval = 0 overfit = False num_workers = 0 max_sample_len_s = 5.0 num_prefetch_batches = 32 global_ds_sampling_f = 1.0 dataloader_snrs = -5,0,5,10,20,40 batch_size_scheduling = max_epochs = 10 validation_criteria = loss validation_criteria_rule = min early_stopping_patience = 5 start_eval = False

    [df] sr = 48000 fft_size = 960 hop_size = 480 nb_erb = 32 nb_df = 96 norm_tau = 1 lsnr_max = 35 lsnr_min = -15 min_nb_erb_freqs = 2 df_order = 5 df_lookahead = 0 pad_mode = input

    [deepfilternet] conv_lookahead = 0 conv_ch = 16 conv_depthwise = True convt_depthwise = True conv_kernel = 1,3 conv_kernel_inp = 3,3 emb_hidden_dim = 256 emb_num_layers = 2 emb_gru_skip = none df_hidden_dim = 256 df_gru_skip = none df_pathway_kernel_size_t = 1 enc_concat = False df_num_layers = 3 df_n_iter = 1 linear_groups = 1 enc_linear_groups = 16 mask_pf = False

    [distortion] p_reverb = 0.2 p_bandwidth_ext = 0.0 p_clipping = 0.0 p_zeroing = 0.0 p_air_absorption = 0.0

    [optim] lr = 0.0005 momentum = 0 weight_decay = 0.05 optimizer = adamw lr_min = 1e-06 lr_warmup = 0.0001 warmup_epochs = 3 lr_cycle_mul = 1.0 lr_cycle_decay = 0.5 lr_cycle_epochs = -1 weight_decay_end = -1

    [maskloss] factor = 0 mask = iam gamma = 0.6 gamma_pred = 0.6 f_under = 2

    [spectralloss] factor_magnitude = 0 factor_complex = 0 factor_under = 1 gamma = 1

    [multiresspecloss] factor = 0 factor_complex = 0 gamma = 1 fft_sizes = 512,1024,2048

    [sdrloss] factor = 0

    [localsnrloss] factor = 0.0005

    logs: python df/train.py dataset_small.cfg data-dir-small/ base_dir_small/ 2022-12-21 15:31:00 | INFO | DF | Running on torch 1.12.0+cu102 2022-12-21 15:31:00 | INFO | DF | Running on host research-bj-green-x4-0 fatal: not a git repository (or any of the parent directories): .git 2022-12-21 15:31:00 | INFO | DF | Loading model settings of base_dir_small 2022-12-21 15:31:00 | INFO | DF | Running on device cuda:0 2022-12-21 15:31:00 | INFO | DF | Initializing model deepfilternet3 2022-12-21 15:31:03 | INFO | DF | Initializing dataloader with data directory data-dir-small/ 2022-12-21 15:31:03 | INFO | DF | Loading HDF5 key cache from .cache_dataset_small.cfg 2022-12-21 15:31:03 | INFO | DF | Start train epoch 0 with batch size 1 2022-12-21 15:31:04 | INFO | DF | [0] [0/9] | loss: 0.28417 | lr: 1.000E-04 thread '' panicked at 'assertion failed: k <= self.len()', /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/slice/mod.rs:3094:9 note: run with RUST_BACKTRACE=1 environment variable to display a backtrace 2022-12-21 15:31:04 | ERROR | DF | An error has been caught in function '', process 'MainProcess' (29909), thread 'MainThread' (140548370358784): Traceback (most recent call last):

    File "df/train.py", line 638, in main() └ <function main at 0x7fd3e3bf40d0>

    File "df/train.py", line 248, in main train_loss = run_epoch( └ <function run_epoch at 0x7fd3e3bfc040>

    File "df/train.py", line 355, in run_epoch for i, batch in enumerate(loader.iter_epoch(split, seed)): │ │ │ │ │ └ 0 │ │ │ │ └ 'train' │ │ │ └ <function PytorchDataLoader.iter_epoch at 0x7fd3e3bf4e50> │ │ └ <libdfdata.torch_dataloader.PytorchDataLoader object at 0x7fd3dc59a5b0> │ └ Batch of size 1: │ SNRs: 40 │ Gains: -6 │
    └ 0

    File "/home/xxx/code/deepfilter/DeepFilterNet-0.4.0/pyDF-data/libdfdata/torch_dataloader.py", line 283, in iter_epoch batch = self._get_batch() │ └ <function PytorchDataLoader._get_batch at 0x7fd3e3bf4d30> └ <libdfdata.torch_dataloader.PytorchDataLoader object at 0x7fd3dc59a5b0>

    File "/home/xxx/code/deepfilter/DeepFilterNet-0.4.0/pyDF-data/libdfdata/torch_dataloader.py", line 232, in _get_batch _, batch = self.data_queue.get() │ │ └ <function PytorchDataLoader._get_worker_queue_dummy.._Queue.get at 0x7fd3da327310> │ └ <libdfdata.torch_dataloader.PytorchDataLoader._get_worker_queue_dummy.._Queue object at 0x7fd3dca65640> └ <libdfdata.torch_dataloader.PytorchDataLoader object at 0x7fd3dc59a5b0>

    File "/home/xxx/code/deepfilter/DeepFilterNet-0.4.0/pyDF-data/libdfdata/torch_dataloader.py", line 214, in get self.loader.cleanup() │ │ └ <method 'cleanup' of 'builtins._FdDataLoader' objects> │ └ <builtins._FdDataLoader object at 0x7fd3dc730100> └ <libdfdata.torch_dataloader.PytorchDataLoader object at 0x7fd3dc59a5b0>

    RuntimeError: DF dataloader error: ThreadJoinError("Any { .. }")

    opened by BWMa 27
  • Question for training

    Question for training

    Hi Rikorose,

    Sorry to bother you again, I try to generate data and train the model according to the training part.

    I generated the training_set.txt (just select 10 files for test.) for speech and made the hdf5.(and so on for noise). Use python df/prepare_data.py --sr 48000 speech training_set.txt TRAIN_SET_SPEECH.hdf5.

    ~/DeepFilterNet/wav/dataset/oblomov_s009036.wav
~/DeepFilterNet/wav/dataset/oblomov_s009040.wav  
~/DeepFilterNet/wav/dataset/oblomov_s009033.wav     
~/DeepFilterNet/wav/dataset/oblomov_s009037.wav    
~/DeepFilterNet/wav/dataset/oblomov_s009041.wav  
~/DeepFilterNet/wav/dataset/oblomov_s009034.wav    
~/DeepFilterNet/wav/dataset/oblomov_s009038.wav     
~/DeepFilterNet/wav/dataset/oblomov_s009042.wav  
~/DeepFilterNet/wav/dataset/oblomov_s009035.wav     
~/DeepFilterNet/wav/dataset/oblomov_s009039.wav

    Generate the dataset.cfg as shown below,

    {
 "train": [
    [
      "~/DeepFilterNet/DeepFilterNet-github/DeepFilterNet/hdf5/TRAIN_SET_SPEECH.hdf5",
      1.0
    ],
    [
      "~/DeepFilterNet/DeepFilterNet-github/DeepFilterNet/hdf5/TRAIN_SET_NOISE.hdf5",
      1.0
    ]
  ],
  "valid": [
    [
      "~/DeepFilterNet/DeepFilterNet-github/DeepFilterNet/hdf5/TRAIN_SET_SPEECH.hdf5",
      0.2
    ],
    [
      "~/DeepFilterNet/DeepFilterNet-github/DeepFilterNet/hdf5/TRAIN_SET_NOISE.hdf5",
      0.2
    ]
  ],
  "test": [
    [
      "~/DeepFilterNet/DeepFilterNet-github/DeepFilterNet/hdf5/TRAIN_SET_SPEECH.hdf5",
      0.2
    ],
    [
      "~/DeepFilterNet/DeepFilterNet-github/DeepFilterNet/hdf5/TRAIN_SET_NOISE.hdf5",
      0.2
    ]
  ]
}

    Encounter some error as shown in the figure below, 擷取

    In addition, I have some questions:

    1. In the command python df/train.py dataset.cfg ~/wav_folder/ ./base_dir/, -Is data_dir a wav folder or an hdf5 folder? (I think is hdf5 folder) -Can base_dir/ not exist? (But we need to give config.ini, so here I enter pertrained_model/ and delete .ckpt)
    2. I found that the log says dataloader len:0, is this a problem?
    3. I remove the all 'df.' of each file while import (ex. from df.config import ... -> from config import ...), otherwise it will cause an import error.

    Thanks,

    opened by aaronhsueh0506 25
  • Reproducing results

    Reproducing results

    Hi,

    I had tried to re-train the deepfilternet model using the DNS-3 challenge dataset mentioned in your work.

    I don't have the additional 10k IR. However, the other dataset remains the same.

    On VCTK test set, using the config.ini in the pre-trained model as my training config, my "best model" on validation gives PESQ score of 2.60. It is much lower than 2.81 from the pre-trained model.

    In config.ini, Adamw is used, while in the paper Adam as optimizer is mentioned.

    Do you think any other factors would result in such a performance drop?

    Could you clarify on the 3 s sample for training? Suppose the DNS-3 sample has 10 s in a sample, do I need to split it into 3 s segments so as to utilize the entire train clip? Or just use the first 3 seconds of the clip? Alternatively, is random 3 s generated on-the-fly while training?

    In the hdf5 setup, does the speech/noise/rir need to have sample number of samples? Or is the noise and RIR sampled randomly from a list? For example, if the speech list has 1000 samples, noise list is 100 samples and rir list is 100 samples, is it okay? or should it be 1000 speech, 1000 noise, 1000 rir? Is it needed to make the duration of speech and noise samples to be the same?

    How about the reverberation parameter p_reverb = 0.05? The data augmentation is performed by default or any other config is needed? conv_lookahead = 2 in config.ini. But the paper mentions "look-ahead of l = 1 frame for both DF as well as in the DNN convolutions".

    stale 
    opened by rohithmars 15
  • RuntimeError: DF dataloader error: ThreadJoinError(

    RuntimeError: DF dataloader error: ThreadJoinError("Any { .. }") when attempting to train DeepFilterNet

    I am trying to train DeepFilterNet but I am running into the following error when training: RuntimeError: DF dataloader error: ThreadJoinError("Any { .. }").

    What I did

    I installed DeepFilterNet via PyPI using:

    pip3 install torch torchvision torchaudio
pip install deepfilternet[train]

    Then I generated the following HDF5 dataset files:

    TEST_SET_NOISE.hdf5
TEST_SET_SPEECH.hdf5
TRAIN_SET_NOISE.hdf5
TRAIN_SET_SPEECH.hdf5
VALID_SET_NOISE.hdf5
VALID_SET_SPEECH.hdf5

    and placed them into the same directory .../data_folder.

    I also created a config.ini file with the following contents:

    config.ini 

    [deepfilternet]
emb_hidden_dim = 256
df_hidden_dim = 256
df_num_layers = 3
conv_ch = 16
conv_lookahead = 0
conv_depthwise = True
convt_depthwise = True
conv_kernel = 1,3
conv_kernel_inp = 3,3
emb_num_layers = 2
emb_gru_skip = none
df_gru_skip = none
df_pathway_kernel_size_t = 1
enc_concat = False
df_n_iter = 1
linear_groups = 1
enc_linear_groups = 16
mask_pf = False
gru_type = grouped
gru_groups = 1
group_shuffle = True
dfop_method = real_unfold
df_output_layer = linear

[df]
fft_size = 960
nb_erb = 32
nb_df = 96
sr = 48000
hop_size = 480
norm_tau = 1
lsnr_max = 35
lsnr_min = -15
min_nb_erb_freqs = 2
df_order = 5
df_lookahead = 0
pad_mode = input

[train]
model = deepfilternet2
batch_size = 16
batch_size_eval = 16
num_workers = 4
overfit = false
lr = 0.001
max_epochs = 30
seed = 42
device = cuda:1
mask_only = False
df_only = False
jit = False
max_sample_len_s = 5.0
num_prefetch_batches = 32
global_ds_sampling_f = 1.0
dataloader_snrs = -5,0,5,10,20,40
batch_size_scheduling = 
validation_criteria = loss
validation_criteria_rule = min
early_stopping_patience = 5
start_eval = False

[distortion]
p_reverb = 0.0
p_bandwidth_ext = 0.0
p_clipping = 0.0
p_air_absorption = 0.0

[optim]
lr = 0.0005
momentum = 0
weight_decay = 0.05
optimizer = adamw
lr_min = 1e-06
lr_warmup = 0.0001
warmup_epochs = 3
lr_cycle_mul = 1.0
lr_cycle_decay = 0.5
lr_cycle_epochs = -1
weight_decay_end = -1

[maskloss]
factor = 0
mask = iam
gamma = 0.6
gamma_pred = 0.6
f_under = 2

[spectralloss]
factor_magnitude = 0
factor_complex = 0
factor_under = 1
gamma = 1

[multiresspecloss]
factor = 0
factor_complex = 0
gamma = 1
fft_sizes = 512,1024,2048

[sdrloss]
factor = 0

[localsnrloss]
factor = 0.0005

    and put it in a directory .../log_folder.

    I also created a dataset.cfg file with the following contents:

    dataset.cfg 

    {
  "train": [
    [
      "TRAIN_SET_SPEECH.hdf5",
      1.0
    ],
    [
      "TRAIN_SET_NOISE.hdf5",
      1.0
    ]
  ],
  "valid": [
    [
      "VALID_SET_SPEECH.hdf5",
      1.0
    ],
    [
      "VALID_SET_NOISE.hdf5",
      1.0
    ]
  ],
  "test": [
    [
      "TEST_SET_SPEECH.hdf5",
      1.0
    ],
    [
      "TEST_SET_NOISE.hdf5",
      1.0
    ]
  ]
}

    Lastly, I ran the command to train the model:

    python df/train.py path/to/dataset.cfg .../data_folder .../log_folder

    and this is the output that I get:

    2022-11-22 22:10:07 | INFO     | DF | Running on torch 1.13.0+cu117
2022-11-22 22:10:07 | INFO     | DF | Running on host workstation3
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
2022-11-22 22:10:07 | INFO     | DF | Loading model settings of log_folder
2022-11-22 22:10:07 | INFO     | DF | Running on device cuda:1
2022-11-22 22:10:07 | INFO     | DF | Initializing model `deepfilternet2`
2022-11-22 22:10:08 | DEPRECATED | DF | Use of `linear` for `df_ouput_layer` is marked as deprecated.
2022-11-22 22:10:08 | INFO     | DF | Initializing dataloader with data directory /home/tester/bokleong/dl_project_dfn/data_folder/
2022-11-22 22:10:08 | INFO     | DF | Loading HDF5 key cache from /home/tester/bokleong/dl_project_dfn/.cache_dataset.cfg
2022-11-22 22:10:08 | INFO     | DF | Start train epoch 0 with batch size 32
thread 'DataLoader Worker 1' panicked at 'called `Option::unwrap()` on a `None` value', libDF/src/dataset.rs:1134:69
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'DataLoader Worker 0' panicked at 'called `Option::unwrap()` on a `None` value', libDF/src/dataset.rs:1134:69
thread 'DataLoader Worker 2' panicked at 'called `Option::unwrap()` on a `None` value', libDF/src/dataset.rs:1134:69
thread 'DataLoader Worker 3' panicked at 'called `Option::unwrap()` on a `None` value', libDF/src/dataset.rs:1134:69
Exception in thread PinMemoryLoop:
Traceback (most recent call last):
  File "/home/tester/miniconda3/envs/user5/lib/python3.9/threading.py", line 980, in _bootstrap_inner
    self.run()
  File "/home/tester/miniconda3/envs/user5/lib/python3.9/threading.py", line 917, in run
    self._target(*self._args, **self._kwargs)
  File "/home/tester/miniconda3/envs/user5/lib/python3.9/site-packages/torch/utils/data/_utils/pin_memory.py", line 49, in _pin_memory_loop
    do_one_step()
  File "/home/tester/miniconda3/envs/user5/lib/python3.9/site-packages/torch/utils/data/_utils/pin_memory.py", line 26, in do_one_step
    r = in_queue.get(timeout=MP_STATUS_CHECK_INTERVAL)
  File "/home/tester/miniconda3/envs/user5/lib/python3.9/site-packages/libdfdata/torch_dataloader.py", line 206, in get
    self.loader.cleanup()
RuntimeError: DF dataloader error: ThreadJoinError("Any { .. }")

    Appreciate any help on this, thank you!

    opened by AnonymousEliforp 13
  • Unable to run deepFIlter form CLI

    Unable to run deepFIlter form CLI

    Hi guys, I have followed the installation instruction using pip and now I'm stuck with SLURM and I don't know how to fix it

    I'm executing deepfilternet from colab pro+ account.

    Here is the issue.

    Traceback (most recent call last): File "/usr/local/bin/deepFilter", line 5, in from df.enhance import run File "/usr/local/lib/python3.7/dist-packages/df/enhance.py", line 18, in from df.logger import init_logger, warn_once File "/usr/local/lib/python3.7/dist-packages/df/logger.py", line 49 if (jobid := os.getenv("SLURM_JOB_ID")) is not None: ^ SyntaxError: invalid syntax

    This is which I get while trying to execute this :-> !deepFilter /content/test_audio_053830.wav --output-dir /content

    Anyone got into this kind of issue??

    DO let me know the solution / how can I run this.

    opened by Anand195 13
  • Converter to .onnx model

    Converter to .onnx model

    Hi, I've been trying to use export.py function to convert retrained model to single onnx file, but it seems that there are version mismatch of torch and onnx operation. After some modifications I've almost made it work, but now I'm stuck with

    Traceback (most recent call last):
  File "/data/code_jb/deepfilter2_git/DeepFilterNet/df/export.py", line 327, in <module>
    main(args)
  File "/data/code_jb/deepfilter2_git/DeepFilterNet/df/export.py", line 292, in main
    export(
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context                                                                                                                        
    return func(*args, **kwargs)
  File "/data/code_jb/deepfilter2_git/DeepFilterNet/df/export.py", line 160, in export
    export_impl(
  File "/data/code_jb/deepfilter2_git/DeepFilterNet/df/export.py", line 100, in export_impl
    torch.onnx.export(
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/onnx/__init__.py", line 350, in export
    return utils.export(
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/onnx/utils.py", line 163, in export
    _export(
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/onnx/utils.py", line 1074, in _export
    graph, params_dict, torch_out = _model_to_graph(
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/onnx/utils.py", line 731, in _model_to_graph
    graph = _optimize_graph(
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/onnx/utils.py", line 308, in _optimize_graph
    graph = _C._jit_pass_onnx(graph, operator_export_type)
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/onnx/__init__.py", line 416, in _run_symbolic_function                                                                                                                      
    return utils._run_symbolic_function(*args, **kwargs)
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/onnx/utils.py", line 1421, in _run_symbolic_function                                                                                                                        
    raise symbolic_registry.UnsupportedOperatorError(
torch.onnx.symbolic_registry.UnsupportedOperatorError: Exporting the operator ::view_as_complex to ONNX opset version 14 is not supported. Please feel free to request support or submit a pull request on PyTorch GitHub.

    Am I doing something wrong or the latest Deepfilternet2 modifications was not tested with this converter? Or maybe I should use specific versions of onnxruntime/torch?

    opened by JBloodless 12
  • Noise of typing not working well

    Noise of typing not working well

    Hi Rikorose,

    Thanks for working on version 2 of Deepfilternet. Now I can do the real-time inference process with buffer size=1, which is the same as the full signal effect. The point is that the state of the RNN needs to be inherited.

    Now I'm having a trouble with typing/keyboard noise not working well. But I only use spectral loss with c=0.3 in Deepfilternet2 now, will multi-resolution loss improve in this case? or maybe c=0.6 in preious work is better?

    Thanks, Aaron

    opened by aaronhsueh0506 12
  • Implement a real-time loop for DeepFilterNet

    Implement a real-time loop for DeepFilterNet

    Hi Hendrik, Just curious as didn't see any benchmarks but could it run chunking audio as it goes or does it need the overall file to analyse? How does it compare to https://github.com/breizhn/DTLN?

    Thanks Stuart

    enhancement pinned 
    opened by StuartIanNaylor 10
  • All Cargo Tests pass, but train.py returns error

    All Cargo Tests pass, but train.py returns error

    I have been struggling with the same error for a while now...

    I am using a conda environment in linux on a HPC solution. I previously had problems with Rust detecting multiple versions of HDF5 and (I think) I fixed it by defining HDF5_VERSION being the same as library and header (version 1.10.6) and HDF5_DIR as conda environment root. Following the instructions on https://github.com/aldanor/hdf5-rust, I also do

    $ conda env config vars set RUSTFLAGS="-C link-args=-Wl,-rpath,$HDF5_DIR/lib"

    And add the directory to path: $ conda develop /zhome/a7/0/155527/Desktop/s204161/DEEPL_PLS/DeepFilterNet-main/DeepFilterNet/

    Doing this, I manage to succesfully pass the cargo tests (It no longer complains about HDF5 version used by rust not being the same as the one in h5py):

    (DEEPL_PLS) cargo test Finished test [optimized + debuginfo] target(s) in 2.06s Running unittests src/lib.rs (target/debug/deps/libdfdata-beeffcf7e03d4848)

    running 0 tests

    test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src/lib.rs (target/debug/deps/libdf-9cd65e9fa81eadda)

    running 0 tests

    test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src/lib.rs (target/debug/deps/deep_filter_ladspa-c16148d79da4c5bf)

    running 0 tests

    test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src/lib.rs (target/debug/deps/df-f01c984070e59647)

    running 30 tests test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_06 - should panic ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_05 - should panic ... ok test tests::test_erb_inout ... ok test reexport_dataset_modules::dataset::tests::test_mix_audio_signal ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_07 - should panic ... ok test transforms::tests::test_find_max_abs ... ok test reexport_dataset_modules::augmentations::tests::test_rand_resample ... ok test transforms::tests::test_stft_istft_delay ... ok test reexport_dataset_modules::augmentations::tests::test_low_pass ... ok test reexport_dataset_modules::augmentations::tests::test_clipping ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_10 ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_read_pcm ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_04 - should panic ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_02 ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_01 ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_03 ... ok test reexport_dataset_modules::augmentations::tests::test_gen_noise ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_08 ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_09 ... ok test transforms::tests::test_ext_bandwidth_spectral ... ok test reexport_dataset_modules::augmentations::tests::test_reverb ... ok test reexport_dataset_modules::dataset::tests::test_fft_dataset ... ok test reexport_dataset_modules::augmentations::tests::test_filters ... ok test reexport_dataset_modules::augmentations::tests::test_compose ... ok test reexport_dataset_modules::dataset::tests::test_td_dataset ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_read_flac ... ok test transforms::tests::test_estimate_bandwidth ... ok test reexport_dataset_modules::augmentations::tests::test_air_absorption ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_read_vorbis ... ok test reexport_dataset_modules::dataloader::tests::test_data_loader ... ok

    test result: ok. 30 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 6.37s

    Doc-tests df

    running 0 tests

    test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

    However when I cd into the repo and run for training it gives the error

    python DeepFilterNet/df/train.py /work3/s204161/config.cfg /work3/s204161/formatted_data/ /zhome/a7/0/155527/Desktop/s204161/DEEPL_PLS/DeepFilterNet/ --debug 2022-11-21 05:42:10 | INFO | df.logger:init_logger:44 | Running on torch 1.12.0 2022-11-21 05:42:10 | INFO | df.logger:init_logger:45 | Running on host n-62-27-19 fatal: not a git repository (or any parent up to mount point /) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). 2022-11-21 05:42:10 | INFO | df.logger:init_logger:67 | Loading model settings of DeepFilterNet 2022-11-21 05:42:10 | INFO | main:main:94 | Running on device cpu 2022-11-21 05:42:10 | INFO | df.model:init_model:21 | Initializing model deepfilternet3 2022-11-21 05:42:10 | INFO | libdfdata.torch_dataloader:init:99 | Initializing dataloader with data directory /work3/s204161/formatted_data/ 2022-11-21 05:42:10 | ERROR | main::633 | An error has been caught in function '', process 'MainProcess' (16810), thread 'MainThread' (139678983296832): Traceback (most recent call last):

    File "/zhome/a7/0/155527/Desktop/s204161/DEEPL_PLS/DeepFilterNet-main/DeepFilterNet/df/train.py", line 633, in main() └ <function main at 0x7f08f2c37b50>

    File "/zhome/a7/0/155527/Desktop/s204161/DEEPL_PLS/DeepFilterNet-main/DeepFilterNet/df/train.py", line 139, in main dataloader = DataLoader( └ <class 'libdfdata.torch_dataloader.PytorchDataLoader'>

    File "/zhome/a7/0/155527/ENTER/envs/DEEPL_PLS/lib/python3.10/site-packages/libdfdata/torch_dataloader.py", line 101, in init self.loader = _FdDataLoader( │ └ <class 'builtins._FdDataLoader'> └ <libdfdata.torch_dataloader.PytorchDataLoader object at 0x7f08f1dc6b60>

    RuntimeError: DF dataset error: Hdf5ErrorDetail { source: H5Fopen(): unable to open file: bad superblock version number, msg: "Error during File::open of dataset /work3/s204161/formatted_data/VALID_SET_SPEECH.hdf5" }

    I have no idea what is left to do. I do not have any prior rust experience. I've attached conda_info.txt should it be relevant and when I run the script:

    import h5py hdfFile = h5py.File('/work3/s204161/formatted_data/TRAIN_SET_SPEECH.hdf5', 'r') print(h5py.version.info) for name in hdfFile: print(hdfFile[name]) print(hdfFile.keys())

    Summary of the h5py configuration

    h5py 3.7.0 HDF5 1.10.6 Python 3.10.8 (main, Nov 4 2022, 13:48:29) [GCC 11.2.0] sys.platform linux sys.maxsize 9223372036854775807 numpy 1.21.5 cython (built with) 0.29.30 numpy (built against) 1.21.5 HDF5 (built against) 1.10.6 <HDF5 group "/speech" (1314 members)> <KeysViewHDF5 ['speech']>

    :(

    conda_info.txt

    opened by Student204161 9
  • About whitenoise performance

    About whitenoise performance

    Thanks for your awesome work! I have installed deepfilternet through pip and test some samples. The overall noise reduction effect is great, but in whitenoise scene, there are more residues in the speech, which leads to a poor subjective feelings. Have you noticed this phenomenon? I will attach the samples below. samples.zip

    stale 
    opened by sqummy 9
  • What is up with the canned deepFilter?

    What is up with the canned deepFilter?

    (DeepFilterNet) C:\Users\sdr\Downloads\Playground\DeepFilterNet>deepFilter test.wav Traceback (most recent call last): File "C:\Users\sdr\anaconda3\envs\DeepFilterNet\lib\runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Users\sdr\anaconda3\envs\DeepFilterNet\lib\runpy.py", line 86, in run_code exec(code, run_globals) File "C:\Users\sdr\anaconda3\envs\DeepFilterNet\Scripts\deepFilter.exe_main.py", line 7, in TypeError: main() missing 1 required positional argument: 'args'

    (DeepFilterNet) C:\Users\sdr\Downloads\Playground\DeepFilterNet>

    stale 
    opened by Joeweav 9
  • build(deps): bump actions/stale from 6 to 7

    build(deps): bump actions/stale from 6 to 7

    Bumps actions/stale from 6 to 7.

    Release notes

    Sourced from actions/stale's releases.

    v7.0.0

    ⚠️ This version contains breaking changes ⚠️

    What's Changed

    Breaking Changes

    • In this release we prevent this action from managing the stale label on items included in exempt-issue-labels and exempt-pr-labels
    • We decided that this is outside of the scope of this action, and to be left up to the maintainer

    New Contributors

    Full Changelog: https://github.com/actions/stale/compare/v6...v7.0.0

    v6.0.1

    Update @​actions/core to 1.10.0 #839

    Full Changelog: https://github.com/actions/stale/compare/v6.0.0...v6.0.1

    Changelog

    Sourced from actions/stale's changelog.

    Changelog

    [7.0.0]

    :warning: Breaking change :warning:

    [6.0.1]

    Update @​actions/core to v1.10.0 (#839)

    [6.0.0]

    :warning: Breaking change :warning:

    Issues/PRs default close-issue-reason is now not_planned(#789)

    [5.1.0]

    Don't process stale issues right after they're marked stale [Add close-issue-reason option]#764#772 Various dependabot/dependency updates

    4.1.0 (2021-07-14)

    Features

    4.0.0 (2021-07-14)

    Features

    Bug Fixes

    • dry-run: forbid mutations in dry-run (#500) (f1017f3), closes #499
    • logs: coloured logs (#465) (5fbbfba)
    • operations: fail fast the current batch to respect the operations limit (#474) (5f6f311), closes #466
    • label comparison: make label comparison case insensitive #517, closes #516
    • filtering comments by actor could have strange behavior: "stale" comments are now detected based on if the message is the stale message not who made the comment(#519), fixes #441, #509, #518

    Breaking Changes

    ... (truncated)

    Commits
    • 6f05e42 draft release for v7.0.0 (#888)
    • eed91cb Update how stale handles exempt items (#874)
    • 10dc265 Merge pull request #880 from akv-platform/update-stale-repo
    • 9c1eb3f Update .md files and allign build-test.yml with the current test.yml
    • bc357bd Update .github/workflows/release-new-action-version.yml
    • 690ede5 Update .github/ISSUE_TEMPLATE/bug_report.md
    • afbcabf Merge branch 'main' into update-stale-repo
    • e364411 Update name of codeql.yml file
    • 627cef3 fix print outputs step (#859)
    • 975308f Merge pull request #876 from jongwooo/chore/use-cache-in-check-dist
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies github_actions 
    opened by dependabot[bot] 0
  • When I retrain the model by models/DeepFilterNet2/config.ini, the result is strange~

    When I retrain the model by models/DeepFilterNet2/config.ini, the result is strange~

    Thanks for your awesome work! I'm trying to recurrence the training through models/DeepFilterNet2/config.ini, but when I use DeepFilterNet/df/enhance.py to test the retrained model, output wavs is nearly to silence. This is my retraining step:Thanks for your awesome work! I'm trying to recurrence the training through models/DeepFilterNet2/config.ini, but when I use DeepFilterNet/df/enhance.py to test the retrained model, output wavs is nearly to silence. This is my retraining step: (1) copy the config.ini of DeepFilterNet2

        cd DeepFilterNet
    mkdir base_dir
    cp ../models/DeepFilterNet2/config.ini ./base_dir

    (2) preparing data take VocalSet_48kHz_mono_000_NA_NA.tar.bz2 as clean data take noise_fullband/datasets_fullband.noise_fullband.audioset_000.tar.bz2 as noise data

        find ../scripts/out/datasets_fullband/clean_fullband/VocalSet_48kHz_mono -iname "*.wav" > training_set_speech.txt
    python df/scripts/prepare_data.py --sr 48000 speech training_set_speech.txt ./data_dir/TRAIN_SET_SPEECH.hdf5
    find ../scripts/out/datasets_fullband/noise_fullband -iname "*.wav" > training_set_noise.txt
    python df/scripts/prepare_data.py --sr 48000 noise training_set_noise.txt ./data_dir/TRAIN_SET_NOISE.hdf5

    this is my dataset.cfg： { "train": [ [ "TRAIN_SET_SPEECH.hdf5", 1.0 ], [ "TRAIN_SET_NOISE.hdf5", 1.0 ], [ "TRAIN_SET_RIR.hdf5", 0.0 ] ], "valid": [ [ "TRAIN_SET_SPEECH.hdf5", 0.2 ], [ "TRAIN_SET_NOISE.hdf5", 0.2 ], [ "TRAIN_SET_RIR.hdf5", 0.0 ] ], "test": [ [ "TRAIN_SET_SPEECH.hdf5", 0.2 ], [ "TRAIN_SET_NOISE.hdf5", 0.2 ], [ "TRAIN_SET_RIR.hdf5", 0.0 ] ] }

    (3) start retraining python df/train.py dataset.cfg ./data_dir/ ./base_dir/

    (4) test retrained model

    python DeepFilterNet/df/enhance.py -m DeepFilterNet/base_dir2 --output-dir DeepFilterNet/test_dir_out/ DeepFilterNet/test_dir_48k/interview_48k.wav

    but the output of the reatrained model is just like this retraing_model_results

    I’m not sure which step is wrong or missed, have you ever met the same problem?

    opened by Yancongs 6
  • Multithread ?

    Multithread ?

    Hi and thanks for a amazing work ! I tested the ladspa filter on a Allwinner H3 (ARM Cortex-A7 Quad-Core 1.296 GHz) , it was easy to install; great work The filter seems to be single-threaded, a single core seems not be enough to process. Would it be possible to involve the other cores ?

    2022-12-22T16:33:37.579Z | WARN |  deep_filter_ladspa | DF 7b7326a9e80d | Underrun detected (RTF: 2.39). Processing too slow!
overrun!!! (at least 277.761 ms long)
2022-12-22T16:33:38.903Z | WARN |  deep_filter_ladspa | DF 3bfb6693797b | Underrun detected (RTF: 2.48). Processing too slow!
overrun!!! (at least 312.537 ms long)
2022-12-22T16:33:40.179Z | WARN |  deep_filter_ladspa | DF 9f61b96d4beb | Underrun detected (RTF: 2.36). Processing too slow!
overrun!!! (at least 266.241 ms long)
2022-12-22T16:33:41.476Z | WARN |  deep_filter_ladspa | DF 54a1822b409f | Underrun detected (RTF: 2.41). Processing too slow!
overrun!!! (at least 284.917 ms long)
    wontfix 
    opened by jenskastensson 2
  • build(deps): update roots requirement from 0.0.7 to 0.0.8 in /libDF

    build(deps): update roots requirement from 0.0.7 to 0.0.8 in /libDF

    Updates the requirements on roots to permit the latest version.

    Release notes

    Sourced from roots's releases.

    December 2022

    Remove debug output, fix quartic equations, allow mutable closures.

    Changelog

    Sourced from roots's changelog.

    0.0.8 - 2022-12-21

    • Remove debug print - thanks to J-F-Liu, Ralith
    • Fixed discriminant check in quartic equations - thanks to jingnanshi
    • Allowed mutable closures for secant etc. - thanks to vron

    0.0.7 - 2021-06-17

    • SearchError becomes public - thanks to JP-Ellis
    • Unnormalized cubic equations are solved using the general formula rather than trigonometrically - thanks to Logicalshift

    0.0.6 - 2019-12-22

    • Fixed cubic equations with very small a3 - thanks to Andrew Hunter
    • Improved quartic equations with multiple roots (for f64; f32 is still a problem) - thanks to Tim Lueke
    • Removed warnings of rustc 1.40.0
    • Switched benchmarks from Bencher to Criterion

    0.0.5 - 2019-01-20

    • Trait Error implemented for SearchError - thanks to phillyfan1138
    • Find roots of higher-degree polynomials using eigenvalues - thanks to stiv-yakovenko
    • Find roots of higher-degree polynomials using Sturm's theorem recursively (experimental)
    • Inverse quadratic approximation

    0.0.4 - 2017-09-05

    • Reduced the performance overhead by using generics - thanks to aepsil0n
    • Handle special cases of quadratic equations - thanks to stiv-yakovenko

    0.0.3 - 2017-03-28

    • New version of the compiler
    • Benchmarks
    • Improved the speed of the Brent-Dekker method
    • Reduced the convergency boilerplate

    0.0.2 - 2015-06-08

    • Fight against the compiler

    [0.0.1] - 2015-03-24

    • Initial version
    Commits

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies rust 
    opened by dependabot[bot] 0
  • Run tract model in half float

    Run tract model in half float

    running

    cargo run -p deep_filter --features=tract,bin,wav-utils,transforms --bin deep-filter -- --half-floats assets/noisy_snr0.wav -o out -vv -D

    results in the following stack trace:

       Compiling deep_filter v0.4.1-pre (/home/hendrik/projects/DeepFilterNet/libDF)
    Finished dev [optimized + debuginfo] target(s) in 20.20s
     Running `target/debug/deep-filter --half-floats assets/noisy_snr0.wav -o out -vv -D`
[2022-12-13T10:09:59Z INFO  df::tract] Init encoder with delay: 0
[2022-12-13T10:09:59Z INFO  tract_linalg::x86_64_fma] mmm_f32, sigmoid_f32, tanh_f32: x86_64/fma activated
[2022-12-13T10:09:59Z INFO  tract_linalg::x86_64_fma] mmm_i8_i8 and mmm_i8_i32: x86_64/avx2 activated
[2022-12-13T10:09:59Z INFO  df::tract] Init ERB decoder with delay: 0
[2022-12-13T10:09:59Z INFO  df::tract] Init DF decoder with delay: 0
[2022-12-13T10:09:59Z INFO  df::tract] Running with model type deepfilternet3 lookahead 2
Error: Evaluating #3 "/erb_conv0/1/Conv.pad" Pad

Caused by:
    Tensor datum type error: tensor is F32, accessed as F16

Stack backtrace:
   0: tract_data::tensor::Tensor::check_for_access
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-data-0.18.5/src/tensor.rs:748:13
   1: tract_data::tensor::Tensor::to_scalar
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-data-0.18.5/src/tensor.rs:848:9
   2: tract_core::ops::array::pad::Pad::eval_t
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-core-0.18.5/src/ops/array/pad.rs:34:38
      <tract_core::ops::array::pad::Pad as tract_core::ops::EvalOp>::eval
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-core-0.18.5/src/ops/array/pad.rs:109:18
   3: tract_core::plan::eval
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-core-0.18.5/src/plan.rs:491:17
   4: core::ops::function::FnMut::call_mut
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/ops/function.rs:164:5
      tract_core::plan::SimpleState<F,O,M,P>::exec_plan_with_eval
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-core-0.18.5/src/plan.rs:264:26
   5: tract_core::plan::SimpleState<F,O,M,P>::run_plan_with_eval
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-core-0.18.5/src/plan.rs:199:9
      tract_core::plan::SimpleState<F,O,M,P>::run
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-core-0.18.5/src/plan.rs:177:9
   6: df::tract::DfTract::process
             at ./libDF/src/tract.rs:401:17
   7: deep_filter::main
             at ./libDF/src/bin/enhance_wav.rs:139:13
   8: core::ops::function::FnOnce::call_once
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/ops/function.rs:248:5
      std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/sys_common/backtrace.rs:122:18
   9: std::rt::lang_start::{{closure}}
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/rt.rs:166:18
  10: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/ops/function.rs:283:13
      std::panicking::try::do_call
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:492:40
      std::panicking::try
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:456:19
      std::panic::catch_unwind
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panic.rs:137:14
      std::rt::lang_start_internal::{{closure}}
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/rt.rs:148:48
      std::panicking::try::do_call
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:492:40
      std::panicking::try
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:456:19
      std::panic::catch_unwind
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panic.rs:137:14
      std::rt::lang_start_internal
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/rt.rs:148:20
  11: main
  12: __libc_start_call_main
  13: __libc_start_main_alias_2
  14: _start
    opened by Rikorose 2
Releases(v0.4.0)
Owner
Hendrik Schröter
Hendrik Schröter
GitHub Repository
Tutorials and implementations for "Self-normalizing networks"

Self-Normalizing Networks Tutorials and implementations for "Self-normalizing networks"(SNNs) as suggested by Klambauer et al. (arXiv pre-print). Vers

Institute of Bioinformatics, Johannes Kepler University Linz 1.6k Jan 07, 2023
Awesome Monocular 3D detection

Awesome Monocular 3D detection Paper list of 3D detetction, keep updating! Contents Paper List 2022 2021 2020 2019 2018 2017 2016 KITTI Results Paper

Zhikang Zou 184 Jan 04, 2023
A curated list of awesome deep long-tailed learning resources.

A curated list of awesome deep long-tailed learning resources.

vanint 210 Dec 25, 2022
Controlling a game using mediapipe hand tracking

These scripts use the Google mediapipe hand tracking solution in combination with a webcam in order to send game instructions to a racing game. It features 2 methods of control

3 May 17, 2022
Submodular Subset Selection for Active Domain Adaptation (ICCV 2021)

S3VAADA: Submodular Subset Selection for Virtual Adversarial Active Domain Adaptation ICCV 2021 Harsh Rangwani, Arihant Jain*, Sumukh K Aithal*, R. Ve

Video Analytics Lab -- IISc 13 Dec 28, 2022
Unpaired Caricature Generation with Multiple Exaggerations

CariMe-pytorch The official pytorch implementation of the paper "CariMe: Unpaired Caricature Generation with Multiple Exaggerations" CariMe: Unpaired

Gu Zheng 37 Dec 30, 2022
Unsupervised phone and word segmentation using dynamic programming on self-supervised VQ features.

Unsupervised Phone and Word Segmentation using Vector-Quantized Neural Networks Overview Unsupervised phone and word segmentation on speech data is pe

Herman Kamper 13 Dec 11, 2022
[CVPR 2021] "Multimodal Motion Prediction with Stacked Transformers": official code implementation and project page.

mmTransformer Introduction This repo is official implementation for mmTransformer in pytorch. Currently, the core code of mmTransformer is implemented

DeciForce: Crossroads of Machine Perception and Autonomy 232 Dec 31, 2022
Semi-SDP Semi-supervised parser for semantic dependency parsing.

Semi-SDP Semi-supervised parser for semantic dependency parsing. This repo contains the code used for the semi-supervised semantic dependency parser i

12 Sep 17, 2021
Character-Input - Create a program that asks the user to enter their name and their age

Character-Input Create a program that asks the user to enter their name and thei

PyLaboratory 0 Feb 06, 2022
TensorFlow implementation of Elastic Weight Consolidation

Elastic weight consolidation Introduction A TensorFlow implementation of elastic weight consolidation as presented in Overcoming catastrophic forgetti

James Stokes 67 Oct 11, 2022
Solving reinforcement learning tasks which require language and vision

Multimodal Reinforcement Learning JAX implementations of the following multimodal reinforcement learning approaches. Dual-coding Episodic Memory from

Henry Prior 31 Feb 26, 2022
这是一个mobilenet-yolov4-lite的库，把yolov4主干网络修改成了mobilenet，修改了Panet的卷积组成，使参数量大幅度缩小。

YOLOV4：You Only Look Once目标检测模型-修改mobilenet系列主干网络-在Keras当中的实现 2021年2月8日更新： 加入letterbox_image的选项，关闭letterbox_image后网络的map一般可以得到提升。

Bubbliiiing 65 Dec 01, 2022
CNNs for Sentence Classification in PyTorch

Introduction This is the implementation of Kim's Convolutional Neural Networks for Sentence Classification paper in PyTorch. Kim's implementation of t

Shawn Ng 956 Dec 19, 2022
Repository for XLM-T, a framework for evaluating multilingual language models on Twitter data

This is the XLM-T repository, which includes data, code and pre-trained multilingual language models for Twitter. XLM-T - A Multilingual Language Mode

Cardiff NLP 112 Dec 27, 2022
MNE: Magnetoencephalography (MEG) and Electroencephalography (EEG) in Python

MNE-Python MNE-Python software is an open-source Python package for exploring, visualizing, and analyzing human neurophysiological data such as MEG, E

MNE tools for MEG and EEG data analysis 2.1k Dec 28, 2022
Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models

Molecular Sets (MOSES): A benchmarking platform for molecular generation models Deep generative models are rapidly becoming popular for the discovery

MOSES 656 Dec 29, 2022
4st place solution for the PBVS 2022 Multi-modal Aerial View Object Classification Challenge - Track 1 (SAR) at PBVS2022

A Two-Stage Shake-Shake Network for Long-tailed Recognition of SAR Aerial View Objects 4st place solution for the PBVS 2022 Multi-modal Aerial View Ob

LinpengPan 5 Nov 09, 2022
Bagua is a flexible and performant distributed training algorithm development framework.

Bagua is a flexible and performant distributed training algorithm development framework.

786 Dec 17, 2022
Jupyter notebooks for using & learning Keras

deep-learning-with-keras-notebooks 這個github的repository主要是個人在學習Keras的一些記錄及練習。希望在學習過程中發現到一些好的資訊與範例也可以對想要學習使用 Keras來解決問題的同好，或是對深度學習有興趣的在學學生可以有一些方便理解與上手範例

ErhWen Kuo 2.1k Dec 27, 2022