Garbage classification using structure data.

Overview

垃圾分类模型使用说明

1.包含以下数据文件

文件 描述
data/MaterialMapping.csv 物体以及其归类的信息
data/TestRecords 光谱原始测试数据 CSV 文件
data/TestRecordDesc.zip CSV 文件描述文件
data/Boundaries.csv 物体轮廓信息

2.包含以下模型文件

文件夹 描述
output/Category/ 包含预测大类别的分类模型
output/Material/ 包含预测大类别(4类)的分类模型
output/Backgroud/ 包含预测小类别(50类)的分类模型

3.环境配置

  进入garbage路径,在anaconda命令行运行pip install -r requirements.txt

4.数据预处理

  在anaconda命令行运行python data_preprocess.py,即可在data文件夹中生成AllEmbracingDataset.csv。若将来更新数据,按照和原来相同的格式和路径保存在data文件夹中,即可用data_preprocess.py生成更新后的数据集

  • 运行数据预处理Python脚本,将上述数据的信息集合到一个数据文件中
python code/data_preprocess.py -data_dir D:/datasets/garbage \
                        -test \
                        -groupbyObjID

运行脚本生成的数据文件 datasets/AllEmbracingDataset.csv 数据集

5.模型训练Python脚本

python code/train_gbdt_lr.py -data_dir D:/datasets/garbage/ \
                    -use_groupbyID True \
                    -output_dir output/ \
                    -skip_data_preprocess

其他 Python脚本说明:

  • feature_engineering.py 特征工程代码
  • ref.py 数据处理和模型推理所需的配置文件
  • utils.py 数据处理所需的一些函数
  • gbdt_feature.py 用gbdt模型生成特征

6.模型推理Python脚本

python code/predict_gbdt_lr.py -data_dir D:/datasets/garbage/ \
                    -use_groupbyID True \
                    -output_dir output/ \
                    -skip_data_preprocess \
                    -save_dir output/ 

  注1:只要同一个ObjID的多条数据的预测结果有一个不是背景零,最终预测结果就不是背景零。

  注2:预测出的Material只会是在训练数据中出现过的唯一标记号。这次数据中不同的唯一标记号共有148个,具体可参见output/log/log.txt中的LabelEncoder.classes

  • 预测结果文件(predictions.csv)说明:对每个物体(即每个ObjID,通常对应多条测试记录)给出多个预测结果汇总后的预测结果。
# 域名 意义
1 ObjID 被测物体唯一标记。同一物体会对应多条测试记录
2 Category 物体分类,从训练数据中获取
3 Material 物体对应的唯一标识号,从训练数据中获取
4 pred_Category 模型所预测出的物体分类
5 pred_Material 模型所预测出的物体唯一标识号
6 pred_background 模型预测的背景和物体 (背景标记为 0,物体标记为 1)
7 pred_Category_final 模型所预测出的物体分类
8 pred_Material_final 模型所预测出的物体材料分类

7. 模型精度

  对于Category、Material和Background三种场景的预测,我们均使用GBDT+LR模型。尝试过SVM、XGBoost、LightGBM和GBDT+LR模型,对比之下,GBDT+LR模型表现最好。   在测试集上的Accuracy如下:

场景 Accuracy
Category 0.7583130575831306
Material 0.6042173560421735
Background 0.996044825313118
Owner
wenqi
Learning is all you need!
wenqi
Nightmare-Writeup - Writeup for the Nightmare CTF Challenge from 2022 DiceCTF

Nightmare: One Byte to ROP // Alternate Solution TLDR: One byte write, no leak.

1 Feb 17, 2022
Unofficial implementation of Google "CutPaste: Self-Supervised Learning for Anomaly Detection and Localization" in PyTorch

CutPaste CutPaste: image from paper Unofficial implementation of Google's "CutPaste: Self-Supervised Learning for Anomaly Detection and Localization"

Lilit Yolyan 59 Nov 27, 2022
TraSw for FairMOT - A Single-Target Attack example (Attack ID: 19; Screener ID: 24):

TraSw for FairMOT A Single-Target Attack example (Attack ID: 19; Screener ID: 24): Fig.1 Original Fig.2 Attacked By perturbing only two frames in this

Derry Lin 21 Dec 21, 2022
Image Matching Evaluation

Image Matching Evaluation (IME) IME provides to test any feature matching algorithm on datasets containing ground-truth homographies. Also, one can re

32 Nov 17, 2022
Meta Learning Backpropagation And Improving It (VSML)

Meta Learning Backpropagation And Improving It (VSML) This is research code for the NeurIPS 2021 publication Kirsch & Schmidhuber 2021. Many concepts

Louis Kirsch 22 Dec 21, 2022
PROJECT - Az Residential Real Estate Analysis

AZ RESIDENTIAL REAL ESTATE ANALYSIS -Decided on libraries to import. Includes pa

2 Jul 05, 2022
Cooperative Driving Dataset: a dataset for multi-agent driving scenarios

Cooperative Driving Dataset (CODD) The Cooperative Driving dataset is a synthetic dataset generated using CARLA that contains lidar data from multiple

Eduardo Henrique Arnold 124 Dec 28, 2022
Joint Discriminative and Generative Learning for Person Re-identification. CVPR'19 (Oral)

Joint Discriminative and Generative Learning for Person Re-identification [Project] [Paper] [YouTube] [Bilibili] [Poster] [Supp] Joint Discriminative

NVIDIA Research Projects 1.2k Dec 30, 2022
Rotation-Only Bundle Adjustment

ROBA: Rotation-Only Bundle Adjustment Paper, Video, Poster, Presentation, Supplementary Material In this repository, we provide the implementation of

Seong 51 Nov 29, 2022
traiNNer is an open source image and video restoration (super-resolution, denoising, deblurring and others) and image to image translation toolbox based on PyTorch.

traiNNer traiNNer is an open source image and video restoration (super-resolution, denoising, deblurring and others) and image to image translation to

202 Jan 04, 2023
A computer vision pipeline to identify the "icons" in Christian paintings

Christian-Iconography A computer vision pipeline to identify the "icons" in Christian paintings. A bit about iconography. Iconography is related to id

Rishab Mudliar 3 Jul 30, 2022
OCR Post Correction for Endangered Language Texts

📌 Coming soon: an update to the software including features from our paper on semi-supervised OCR post-correction, to be published in the Transaction

Shruti Rijhwani 96 Dec 31, 2022
Companion code for the paper "Meta-Learning the Search Distribution of Black-Box Random Search Based Adversarial Attacks" by Yatsura et al.

META-RS This is the companion code for the paper "Meta-Learning the Search Distribution of Black-Box Random Search Based Adversarial Attacks" by Yatsu

Bosch Research 7 Dec 09, 2022
CSKG is a commonsense knowledge graph that combines seven popular sources into a consolidated representation

CSKG: The CommonSense Knowledge Graph CSKG is a commonsense knowledge graph that combines seven popular sources into a consolidated representation: AT

USC ISI I2 85 Dec 12, 2022
PyTorch framework for Deep Learning research and development.

Accelerated DL & RL PyTorch framework for Deep Learning research and development. It was developed with a focus on reproducibility, fast experimentati

Catalyst-Team 29 Jul 13, 2022
Official repository for Few-shot Image Generation via Cross-domain Correspondence (CVPR '21)

Few-shot Image Generation via Cross-domain Correspondence Utkarsh Ojha, Yijun Li, Jingwan Lu, Alexei A. Efros, Yong Jae Lee, Eli Shechtman, Richard Zh

Utkarsh Ojha 251 Dec 11, 2022
DeepGNN is a framework for training machine learning models on large scale graph data.

DeepGNN Overview DeepGNN is a framework for training machine learning models on large scale graph data. DeepGNN contains all the necessary features in

Microsoft 45 Jan 01, 2023
Keqing Chatbot With Python

KeqingChatbot A public running instance can be found on telegram as @keqingchat_bot. Requirements Python 3.8 or higher. A bot token. Local Deploy git

Rikka-Chan 2 Jan 16, 2022
A Bayesian cognition approach for belief updating of correlation judgement through uncertainty visualizations

Overview Code and supplemental materials for Karduni et al., 2020 IEEE Vis. "A Bayesian cognition approach for belief updating of correlation judgemen

Ryan Wesslen 1 Feb 08, 2022
The official GitHub repository for the Argoverse 2 dataset.

Argoverse 2 API Official GitHub repository for the Argoverse 2 family of datasets. If you have any questions or run into any problems with either the

Argo AI 156 Dec 23, 2022