Repo for my Tensorflow/Keras CV experiments. Mostly revolving around the Danbooru20xx dataset

Last update: Dec 27, 2022

Related tags

Deep Learning SW-CV-ModelZoo

Overview

SW-CV-ModelZoo

Repo for my Tensorflow/Keras CV experiments. Mostly revolving around the Danbooru20xx dataset

Framework: TF/Keras 2.7

Training SQLite DB built using fire-egg's tools: https://github.com/fire-eggs/Danbooru2019

Currently training on Danbooru2021, 512px SFW subset (sans the rating:q images that had been included in the 2022-01-21 release of the dataset)

Reference:

Anonymous, The Danbooru Community, & Gwern Branwen; “Danbooru2021: A Large-Scale Crowdsourced and Tagged Anime Illustration Dataset”, 2022-01-21. Web. Accessed 2022-01-28 https://www.gwern.net/Danbooru2021

Journal

06/02/2022: great news crew! TRC allowed me to use a bunch of TPUs!

To make better use of this amount of compute I had to overhaul a number of components, so a bunch of things are likely to have fallen to bitrot in the process. I can only guarantee NFNet can work pretty much as before with the right arguments.
NFResNet changes should have left it retrocompatible with the previous version.
ResNet has been streamlined to be mostly in line with the Bag-of-Tricks paper (arXiv:1812.01187) with the exception of the stem. It is not compatible with the previous version of the code.

The training labels have been included in the 2021_0000_0899 folder for convenience.
The list of files used for training is going to be uploaded as a GitHub Release.

Now for some numbers:
compared to my previous best run, the one that resulted in NFNetL1V1-100-0.57141:

I'm using 1.86x the amount of images: 2.8M vs 1.5M
I'm training bigger models: 61M vs 45M params
... in less time: 232 vs 700 hours of processor time
don't get me started on actual wall clock time
with a few amenities thrown in: ECA for channel attention, SiLU activation

And it's all thanks to the folks at TRC, so shout out to them!

I currently have a few runs in progress across a couple of dimensions:

effect of model size with NFNet L0/L1/L2, with SiLU and ECA for all three of them
effect of activation function with NFNet L0, with SiLU/HSwish/ReLU, no ECA

Once the experiments are over, the plan is to select the network definitions that lay on the Pareto curve between throughput and F1 score and release the trained weights.

One last thing.
I'd like to call your attention to the tools/cleanlab_stuff.py script.
It reads two files: one with the binarized labels from the database, the other with the predicted probabilities.
It then uses the cleanlab package to estimate whether if an image in a set could be missing a given label. At the end it stores its conclusions in a json file.
This file could, potentially, be used in some tool to assist human intervention to add the missing tags.

You might also like...

Human head pose estimation using Keras over TensorFlow.

RealHePoNet: a robust single-stage ConvNet for head pose estimation in the wild.

71 Jan 5, 2023

Graph Neural Networks with Keras and Tensorflow 2.

Welcome to Spektral Spektral is a Python library for graph deep learning, based on the Keras API and TensorFlow 2. The main goal of this project is to

2.2k Jan 8, 2023

QKeras: a quantization deep learning library for Tensorflow Keras

QKeras github.com/google/qkeras QKeras 0.8 highlights: Automatic quantization using QKeras; Stochastic behavior (including stochastic rouding) is disa

437 Jan 3, 2023

Hyperparameter Optimization for TensorFlow, Keras and PyTorch

Hyperparameter Optimization for Keras Talos • Key Features • Examples • Install • Support • Docs • Issues • License • Download Talos radically changes

1.6k Dec 15, 2022

MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.

MMdnn MMdnn is a comprehensive and cross-framework tool to convert, visualize and diagnose deep learning (DL) models. The "MM" stands for model manage

5.7k Jan 9, 2023

Deep GPs built on top of TensorFlow/Keras and GPflow

GPflux Documentation | Tutorials | API reference | Slack What does GPflux do? GPflux is a toolbox dedicated to Deep Gaussian processes (DGP), the hier

107 Nov 2, 2022

tf2onnx - Convert TensorFlow, Keras and Tflite models to ONNX.

tf2onnx converts TensorFlow (tf-1.x or tf-2.x), tf.keras and tflite models to ONNX via command line or python api.

1.8k Jan 8, 2023

Build tensorflow keras model pipelines in a single line of code. Created by Ram Seshadri. Collaborators welcome. Permission granted upon request.

deep_autoviml Build keras pipelines and models in a single line of code! Table of Contents Motivation How it works Technology Install Usage API Image

102 Dec 17, 2022

Advanced Deep Learning with TensorFlow 2 and Keras (Updated for 2nd Edition)

1.5k Jan 3, 2023

Releases(models_db2021_5500_2022_10_21)

models_db2021_5500_2022_10_21(Oct 21, 2022)

ConvNext B, ViT B16
Trained on Danbooru2021 512px SFW subset, modulos 0000-0899
top 5500 tags (2021_0000_0899_5500/selected_tags.csv)
alpha to white
padding to make the image square is white
channel order is BGR, input is 0...255, scaled to -1...1 within the model

| run_name | definition_name | params_human | image_size | thres | F1 | |:---------------------------------|:------------------|:---------------|-------------:|--------:|-------:| | ConvNextBV1_09_25_2022_05h13m55s | B | 93.2M | 448 | 0.3673 | 0.6941 | | ViTB16_09_25_2022_04h53m38s | B16 | 90.5M | 448 | 0.3663 | 0.6918 |
Source code(tar.gz)
Source code(zip)
ConvNextBV1_09_25_2022_05h13m55s.7z(322.58 MB)
ViTB16_09_25_2022_04h53m38s.7z(312.96 MB)
convnexts_db2021_2022_03_22(Mar 22, 2022)

ConvNext, T/S/B
Trained on Danbooru2021 512px SFW subset, modulos 0000-0899
alpha to white
padding to make the image square is white
channel order is BGR, input is 0...255, scaled to -1...1 within the model

| run_name | definition_name | params_human | image_size | thres | F1 | |:---------------------------------|:------------------|:---------------|-------------:|--------:|-------:| | ConvNextBV1_03_10_2022_21h41m23s | B | 90.01M | 448 | 0.3372 | 0.6892 | | ConvNextSV1_03_11_2022_17h49m56s | S | 51.28M | 384 | 0.3301 | 0.6798 | | ConvNextTV1_03_05_2022_15h56m42s | T | 29.65M | 320 | 0.3259 | 0.6595 |
Source code(tar.gz)
Source code(zip)
ConvNextBV1_03_10_2022_21h41m23s.7z(311.29 MB)
ConvNextSV1_03_11_2022_17h49m56s.7z(177.36 MB)
ConvNextTV1_03_05_2022_15h56m42s.7z(102.96 MB)
nfnets_db2021_2022_03_04(Mar 4, 2022)

NFNet, L0/L1/L2 (based on timm Lx model definitions) Trained on Danbooru2021 512px SFW subset, modulos 0000-0899 alpha to white padding to make the image square is white channel order is BGR, input is 0...255, scaled to -1...1 within the model

| run_name | definition_name | params_human | image_size | thres | F1 | |:---------------------------------|:------------------|:---------------|-------------:|--------:|-------:| | NFNetL2V1_02_20_2022_10h27m08s | L2 | 60.96M | 448 | 0.3231 | 0.6785 | | NFNetL1V1_02_17_2022_20h18m38s | L1 | 45.65M | 384 | 0.3259 | 0.6691 | | NFNetL0V1_02_10_2022_17h50m14s | L0 | 27.32M | 320 | 0.3190 | 0.6509 |
Source code(tar.gz)
Source code(zip)
NFNetL0V1_02_10_2022_17h50m14s.7z(94.98 MB)
NFNetL1V1_02_17_2022_20h18m38s.7z(157.97 MB)
NFNetL2V1_02_20_2022_10h27m08s.7z(210.49 MB)
nfnet_tpu_training(Feb 6, 2022)

Attaching the lists of files currently in use for training and validation, along with the selected labels.

Training and validation files have been selected from the 512px SFW subset.
Source code(tar.gz)
Source code(zip)
2021_0000_0899.7z(1.32 MB)
NFNetL1V1-100-0.57141(Dec 31, 2021)
NFNet, L1 (based on timm Lx model definitions), 100 epochs, F1 @ 0.4 at the end of the 100th epoch was 0.57141

Trained on Danbooru2020 512px SFW subset, modulos 0000-0599

320px per side

alpha to white

padding to make the image square is white

channel order is BGR, scaled to 0-1

mixup alpha = 0.2 during epochs 76-100

analyze_metrics on Danbooru2020 original set, modulos 0984-0999: {'thres': 0.3485, 'F1': 0.6133, 'F2': 0.6133, 'MCC': 0.6094, 'A': 0.9923, 'R': 0.6133, 'P': 0.6133}

analyze_metrics on image IDs 4970000-5000000: {'thres': 0.3148, 'F1': 0.5942, 'F2': 0.5941, 'MCC': 0.5892, 'A': 0.9901, 'R': 0.5940, 'P': 0.5943}

Source code(tar.gz)
Source code(zip)
NFNetL1V1-100-0.57141.7z(158.09 MB)

Owner

GitHub Repository

Python code for the paper How to scale hyperparameters for quickshift image segmentation

How to scale hyperparameters for quickshift image segmentation Python code for the paper How to scale hyperparameters for quickshift image segmentatio

0 Jan 25, 2022

Collect some papers about transformer with vision. Awesome Transformer with Computer Vision (CV)

Awesome Visual-Transformer Collect some Transformer with Computer-Vision (CV) papers. If you find some overlooked papers, please open issues or pull r

2.8k Jan 08, 2023

Re-implememtation of MAE (Masked Autoencoders Are Scalable Vision Learners) using PyTorch.

mae-repo PyTorch re-implememtation of "masked autoencoders are scalable vision learners". In this repo, it heavily borrows codes from codebase https:/

1 Dec 14, 2021

A-ESRGAN aims to provide better super-resolution images by using multi-scale attention U-net discriminators.

A-ESRGAN: Training Real-World Blind Super-Resolution with Attention-based U-net Discriminators The authors are hidden for the purpose of double blind

77 Dec 16, 2022

Sub-tomogram-Detection - Deep learning based model for Cyro ET Sub-tomogram-Detection

Deep learning based model for Cyro ET Sub-tomogram-Detection High degree of stru

2 Feb 04, 2022

BigbrotherBENL - Face recognition on the Big Brother episodes in Belgium and the Netherlands.

BigbrotherBENL - Face recognition on the Big Brother episodes in Belgium and the Netherlands. Keeping statistics of whom are most visible and recognisable in the series and wether or not it has an im

2 Jan 04, 2022

A curated list of references for MLOps

9.3k Jan 07, 2023

This is the pytorch implementation for the paper: Generalizable Mixed-Precision Quantization via Attribution Rank Preservation, which is accepted to ICCV2021.

GMPQ: Generalizable Mixed-Precision Quantization via Attribution Rank Preservation This is the pytorch implementation for the paper: Generalizable Mix

18 Sep 02, 2022

Code and project page for ICCV 2021 paper "DisUnknown: Distilling Unknown Factors for Disentanglement Learning"

DisUnknown: Distilling Unknown Factors for Disentanglement Learning See introduction on our project page Requirements PyTorch = 1.8.0 torch.linalg.ei

24 May 16, 2022

Applying curriculum to meta-learning for few shot classification

Curriculum Meta-Learning for Few-shot Classification We propose an adaptation of the curriculum training framework, applicable to state-of-the-art met

3 Oct 25, 2022

Efficient Two-Step Networks for Temporal Action Segmentation (Neurocomputing 2021)

Efficient Two-Step Networks for Temporal Action Segmentation This repository provides a PyTorch implementation of the paper Efficient Two-Step Network

8 Apr 16, 2022

An official reimplementation of the method described in the INTERSPEECH 2021 paper - Speech Resynthesis from Discrete Disentangled Self-Supervised Representations.

Speech Resynthesis from Discrete Disentangled Self-Supervised Representations Implementation of the method described in the Speech Resynthesis from Di

253 Jan 06, 2023

A Pytree Module system for Deep Learning in JAX

Treex A Pytree-based Module system for Deep Learning in JAX Intuitive: Modules are simple Python objects that respect Object-Oriented semantics and sh

216 Dec 20, 2022

Tweesent-back - Tweesent backend uses fastAPI as the web framework

TweeSent Backend Tweesent backend. This repo uses fastAPI as the web framework.

0 Mar 26, 2022

The most simple and minimalistic navigation dashboard.

Navigation This project follows a goal to have simple and lightweight dashboard with different links. I use it to have my own self-hosted service dash

23 Dec 23, 2022

PyBullet CartPole and Quadrotor environments—with CasADi symbolic a priori dynamics—for learning-based control and reinforcement learning

safe-control-gym Physics-based CartPole and Quadrotor Gym environments (using PyBullet) with symbolic a priori dynamics (using CasADi) for learning-ba

300 Dec 28, 2022

Repo for my Tensorflow/Keras CV experiments. Mostly revolving around the Danbooru20xx dataset

Related tags

Overview

SW-CV-ModelZoo

Reference:

Journal

You might also like...

Human head pose estimation using Keras over TensorFlow.

Graph Neural Networks with Keras and Tensorflow 2.

QKeras: a quantization deep learning library for Tensorflow Keras

Hyperparameter Optimization for TensorFlow, Keras and PyTorch

MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.

Deep GPs built on top of TensorFlow/Keras and GPflow

tf2onnx - Convert TensorFlow, Keras and Tflite models to ONNX.

Build tensorflow keras model pipelines in a single line of code. Created by Ram Seshadri. Collaborators welcome. Permission granted upon request.

Advanced Deep Learning with TensorFlow 2 and Keras (Updated for 2nd Edition)

Releases(models_db2021_5500_2022_10_21)

models_db2021_5500_2022_10_21(Oct 21, 2022)

convnexts_db2021_2022_03_22(Mar 22, 2022)

nfnets_db2021_2022_03_04(Mar 4, 2022)

nfnet_tpu_training(Feb 6, 2022)

NFNetL1V1-100-0.57141(Dec 31, 2021)

Owner

Python code for the paper How to scale hyperparameters for quickshift image segmentation

Collect some papers about transformer with vision. Awesome Transformer with Computer Vision (CV)

Re-implememtation of MAE (Masked Autoencoders Are Scalable Vision Learners) using PyTorch.

A-ESRGAN aims to provide better super-resolution images by using multi-scale attention U-net discriminators.

Sub-tomogram-Detection - Deep learning based model for Cyro ET Sub-tomogram-Detection

BigbrotherBENL - Face recognition on the Big Brother episodes in Belgium and the Netherlands.

A curated list of references for MLOps

This is the pytorch implementation for the paper: Generalizable Mixed-Precision Quantization via Attribution Rank Preservation, which is accepted to ICCV2021.

Code and project page for ICCV 2021 paper "DisUnknown: Distilling Unknown Factors for Disentanglement Learning"

Applying curriculum to meta-learning for few shot classification

Efficient Two-Step Networks for Temporal Action Segmentation (Neurocomputing 2021)

An official reimplementation of the method described in the INTERSPEECH 2021 paper - Speech Resynthesis from Discrete Disentangled Self-Supervised Representations.

A Pytree Module system for Deep Learning in JAX

Tweesent-back - Tweesent backend uses fastAPI as the web framework

The most simple and minimalistic navigation dashboard.

PyBullet CartPole and Quadrotor environments—with CasADi symbolic a priori dynamics—for learning-based control and reinforcement learning

HyperDict - Self linked dictionary in Python

[IEEE Transactions on Computational Imaging] Self-Gated Memory Recurrent Network for Efficient Scalable HDR Deghosting

Freecodecamp Scientific Computing with Python Certification; Solution for Challenge 2: Time Calculator

All materials of Cassandra Event, Udyam'22