Efficient and intelligent interactive segmentation annotation software

Overview

EISeg

Python 3.6 PaddlePaddle 2.2 License Downloads

简体中文 | English

最新动向

  • 交互式分割论文EdgeFlow被ICCV 2021 Workshop接收。
  • 支持静态图预测,全面提升交互速度;新增遥感、医疗标注垂类方向,上线宫格标注,最新EISeg 0.4.0推出。

介绍

EISeg(Efficient Interactive Segmentation)是以RITMEdgeFlow算法为基础,基于飞桨开发的一个高效智能的交互式分割标注软件。涵盖了通用、人像、遥感、医疗等不同方向的高质量交互式分割模型,方便开发者快速实现语义及实例标签的标注,降低标注成本。 另外,将EISeg获取到的标注应用到PaddleSeg提供的其他分割模型进行训练,便可得到定制化场景的高精度模型,打通分割任务从数据标注到模型训练及预测的全流程。

4a9ed-a91y1

模型准备

在使用EIseg前,请先下载模型参数。EISeg 0.4.0版本开放了在COCO+LVIS、大规模人像数据、mapping_challenge及LiTS(Liver Tumor Segmentation Challenge)上训练的四个垂类方向模型,满足通用场景、人像场景、建筑物标注及医疗影像肝脏的标注需求。其中模型结构对应EISeg交互工具中的网络选择模块,用户需要根据自己的场景需求选择不同的网络结构和加载参数。

模型类型 适用场景 模型结构 模型下载地址
高精度模型 适用于通用场景的图像标注。 HRNet18_OCR64 static_hrnet18_ocr64_cocolvis
轻量化模型 适用于通用场景的图像标注。 HRNet18s_OCR48 static_hrnet18s_ocr48_cocolvis
高精度模型 适用于人像标注场景。 HRNet18_OCR64 static_hrnet18_ocr64_human
轻量化模型 适用于人像标注场景。 HRNet18s_OCR48 static_hrnet18s_ocr48_human
高精度模型 适用于通用图像标注场景。 EdgeFlow static_edgeflow_cocolvis
轻量化模型 适用于遥感建筑物标注场景。 HRNet18s_OCR48 static_hrnet18_ocr48_rsbuilding_instance
轻量化模型 适用于医疗肝脏标注场景。 HRNet18s_OCR48 static_hrnet18s_ocr48_lits

NOTE: 将下载的模型结构*.pdmodel及相应的模型参数*.pdiparams需要放到同一个目录下,加载模型时只需选择*.pdiparams结尾的模型参数位置即可, *.pdmodel会自动加载。在使用EdgeFlow模型时,请将使用掩膜关闭,其他模型使用时请勾选使用掩膜

安装使用

EISeg提供多种安装方式,其中使用pip运行代码方式可兼容Windows,Mac OS和Linux。为了避免环境冲突,推荐在conda创建的虚拟环境中安装。

版本要求:

  • PaddlePaddle >= 2.2.0

PaddlePaddle安装请参考官网

克隆到本地

通过git将PaddleSeg克隆到本地:

git clone https://github.com/PaddlePaddle/PaddleSeg.git

安装好所需环境后,进入EISeg,可通过直接运行eiseg打开EISeg:

cd PaddleSeg\EISeg
python -m eiseg

或进入eiseg,运行exe.py打开EISeg:

cd PaddleSeg\EISeg\eiseg
python exe.py

PIP

pip安装方式如下:

pip install eiseg

pip会自动安装依赖。安装完成后命令行输入:

eiseg

即可运行软件。

Windows exe

EISeg使用QPT进行打包。可以从这里下载最新EISeg。解压后双击启动程序.exe即可运行程序。程序第一次运行会初始化安装所需要的包,请稍等片刻。

使用

打开软件后,在对项目进行标注前,需要进行如下设置:

  1. 模型参数加载

    根据标注场景,选择合适的网络模型及参数进行加载。目前在EISeg0.4.0中,已经将动态图预测转为静态图预测,全面提升单次点击的预测速度。选择合适的模型及参数下载解压后,模型结构*.pdmodel及相应的模型参数*.pdiparams需要放到同一个目录下,加载模型时只需选择*.pdiparams结尾的模型参数位置即可。静态图模型初始化时间稍长,请耐心等待模型加载完成后进行下一步操作。正确加载的模型参数会记录在近期模型参数中,可以方便切换,并且下次打开软件时自动加载退出时的模型参数。

  2. 图像加载

    打开图像/图像文件夹。当看到主界面图像正确加载,数据列表正确出现图像路径即可。

  3. 标签添加/加载

    添加/加载标签。可以通过添加标签新建标签,标签分为4列,分别对应像素值、说明、颜色和删除。新建好的标签可以通过保存标签列表保存为txt文件,其他合作者可以通过加载标签列表将标签导入。通过加载方式导入的标签,重启软件后会自动加载。

  4. 自动保存设置

    在使用中可以将自动保存设置上,设定好文件夹即可,这样在使用时切换图像会自动将完成标注的图像进行保存。

当设置完成后即可开始进行标注,默认情况下常用的按键/快捷键如下,如需修改可按E弹出快捷键修改。

部分按键/快捷键 功能
鼠标左键 增加正样本点
鼠标右键 增加负样本点
鼠标中键 平移图像
Ctrl+鼠标中键(滚轮) 缩放图像
S 切换上一张图
F 切换下一张图
Space(空格) 完成标注/切换状态
Ctrl+Z 撤销
Ctrl+Shift+Z 清除
Ctrl+Y 重做
Ctrl+A 打开图像
Shift+A 打开文件夹
E 打开快捷键表
Backspace(退格) 删除多边形
鼠标双击(点) 删除点
鼠标双击(边) 添加点

特色功能使用说明

  • 多边形

    • 交互完成后使用Space(空格)完成交互标注,此时出现多边形边界;
    • 当需要在多边形内部继续进行交互,则使用空格切换为交互模式,此时多边形无法选中和更改。
    • 多边形可以删除,使用鼠标左边可以对锚点进行拖动,鼠标左键双击锚点可以删除锚点,双击两点之间的边则可在此边添加一个锚点。
    • 打开保留最大连通块后,所有的点击只会在图像中保留面积最大的区域,其余小区域将不会显示和保存。
  • 保存格式

    • 打开保存JSON保存COCO保存后,多边形会被记录,加载时会自动加载。
    • 若不设置保存路径,默认保存至当前图像文件夹下的label文件夹中。
    • 如果有图像之间名称相同但后缀名不同,可以打开标签和图像使用相同扩展名
    • 还可设置灰度保存、伪彩色保存和抠图保存,见工具栏中7-9号工具。
  • 生成mask

    • 标签按住第二列可以进行拖动,最后生成mask时会根据标签列表从上往下进行覆盖。
  • 界面模块

    • 可在显示中选择需要显示的界面模块,正常退出时将会记录界面模块的状态和位置,下次打开自动加载。
  • 垂类分割

    EISeg目前已添加对遥感图像和医学影像分割的支持,使用相关功能需要安装额外依赖。

版本更新

  • 2021.11.16 0.4.0:【1】将动态图预测转换成静态图预测,单次点击速度提升十倍;【2】新增遥感图像标注功能,支持多光谱数据通道的选择;【3】支持大尺幅数据的切片(多宫格)处理;【4】新增医疗图像标注功能,支持读取dicom的数据格式,支持选择窗宽和窗位。
  • 2021.09.16 0.3.0:【1】初步完成多边形编辑功能,支持对交互标注的结果进行编辑;【2】支持中/英界面;【3】支持保存为灰度/伪彩色标签和COCO格式;【4】界面拖动更加灵活;【5】标签栏可拖动,生成mask的覆盖顺序由上往下覆盖。
  • 2021.07.07 0.2.0:新增contrib:EISeg,可实现人像和通用图像的快速交互式标注。

贡献者

感谢Yuying Hao, Lin Han, Yizhou Chen, Yiakwy, GT, Zhiliang Yu 等开发者及RITM 算法支持。

学术引用

如果我们的项目在学术上帮助到你,请考虑以下引用:

@article{hao2021edgeflow,
  title={EdgeFlow: Achieving Practical Interactive Segmentation with Edge-Guided Flow},
  author={Hao, Yuying and Liu, Yi and Wu, Zewu and Han, Lin and Chen, Yizhou and Chen, Guowei and Chu, Lutao and Tang, Shiyu and Yu, Zhiliang and Chen, Zeyu and others},
  journal={arXiv preprint arXiv:2109.09406},
  year={2021}
}
Comments
  • fix(add bbox) : the output does not give correct bbox info

    fix(add bbox) : the output does not give correct bbox info

    1. the output for coco dataset type does not generate correct output info

    before (note bbox is [0, 0, 0, 0]): before fix

    fixed: fix

    1. also generates correct bbox annotation QGraphic widget which is bettern than default "boundingRect":

    add bbox

    opened by yiakwy 10
  • 模型输入point的组织形式

    模型输入point的组织形式

    请问作者,对于模型前向传播时,输入的point参数的各个维度代表的含义可以稍微解释下吗?

    比如,我点了一个点,输入的point的形状是:(2, 2, 3) , 我现在知道了 最后一个维度中包含的是 (y, x, index), 也知道 第二个维度的值是 正样本点的数量的2倍,但是我没看懂为啥第二个维度的值要是2,另外一部分存的是啥啊?(看结果,y值没变,x值变化) 第一维,嘿嘿也没看懂? 为啥是2,这里的含义是什么呢?

    感谢您的精彩工作!

    solved 
    opened by zhijiejia 9
  • release/0.4.0 当前存在的部分BUG

    release/0.4.0 当前存在的部分BUG

    [SELF]

    • [x] 切换模型参数后,必须切换图像才能换过来
    • [x] 多边形有孔洞可能越界
    • [x] 偶见打开后界面混乱,同时加载最近模型也错乱(目前猜测是调试崩溃后打乱了setting的保存)
    • [x] #67
    • [x] #66
    • [x] 单独打开多张图片,快捷键S和F切换的第一次有问题
    • [x] 宫格完成后mask和polygon有偏移
    • [x] 宫格模式下某些shp保存无法在arcgis中显示(投影问题需要解决)
    • [x] 宫格模式多标签问题(全部导出为一个标签了)
    • [ ] #70
    • [ ] #68
    • [x] 增加显示图像的地理信息
    • [x] 打开图像报错(存在.xxx.jpg类似格式的文件)
    • [x] GPU/TensorRT加载问题(框架版本问题)
    • [x] 保留最大联通块推理,生成的多边形仍包含所有块
    • [x] 多次关闭和选择图像后,地理信息被清除
    • [x] #71
    • [x] 导出shp多边形未闭合
    • [x] 导出tif波段数不对(为原始图像波段数)
    • [x] 【讨论】导出图像名的大小写与原始图像名不一致(Linux大小写敏感会不会在训练时匹配不上文件 @linhandev)
    • [x] jpg等医疗图像调整窗宽等闪退
    • [x] 【完善】英语翻译支持
    • [x] 【完善】pip打包
    • [ ] 【完善】exe打包
    • [x] 【完善】md文档

    [QA]

    • [x] #74
    • [x] #75
    • [x] #76
    • [x] #77
    • [x] #78
    • [x] #79
    • [x] #80
    • [x] #81
    bug solved 
    opened by geoyee 6
  • 直接打开图像报错

    直接打开图像报错

    直接打开一张没有标签的图像报错:

      File "e:\PdCVSIG\github\EISeg\eiseg\app.py", line 1092, in loadLabel
        imgId = self.coco.imgNameToId.get(osp.basename(imgPath), None)
    AttributeError: 'NoneType' object has no attribute 'imgNameToId'
    
    bug 
    opened by geoyee 6
  • 服务器Linux下无法选择文件夹

    服务器Linux下无法选择文件夹

    点加载网络参数的时候,会弹出一个窗口让我选择模型参数,但是我点哪个文件夹都没反应,这个是出窗口的时候的信息。 image目前看起来可能是QT的问题,有下列参考:

    bug good first issue solved 
    opened by geoyee 5
  • 加载模型后闪退

    加载模型后闪退

    D:\anaconda3\lib\site-packages\win32\lib\pywintypes.py:2: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses import imp, sys, os qt.qpa.fonts: Unable to enumerate family ' "Droid Sans Mono Dotted for Powerline" ' qt.qpa.fonts: Unable to enumerate family ' "Droid Sans Mono Slashed for Powerline" ' qt.qpa.fonts: Unable to enumerate family ' "Roboto Mono Medium for Powerline" ' qt.qpa.fonts: Unable to enumerate family ' "Ubuntu Mono derivative Powerline" ' OMP: Error #15: Initializing libiomp5md.dll, but found libiomp5md.dll already initialized. OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/. QObject::~QObject: Timers cannot be stopped from another thread

    bug 
    opened by YQisme 4
  • Add sliding bar to adjust window width and window center

    Add sliding bar to adjust window width and window center

    Add sliding bar to adjust window width and window center.When reading medical images, the window width and window center are adaptive according to the modes

    opened by richarddddd198 3
  • Train module problem: ritm_train.py import error.

    Train module problem: ritm_train.py import error.

    THX for sharing! Import error was found in ritm_train.py from model.model import ( get_hrnet_model, DistMapsHRNetModel, get_deeplab_model, get_shufflenet_model, ) these modules were not exist in model.py, even in this project. so where can i find these modules, please?

    opened by mengmeng716 3
  • plugin\remotesensing\raster.py  line123 遥感图像显示为黑色

    plugin\remotesensing\raster.py line123 遥感图像显示为黑色

    bug描述 请大致描述出错的现象,在什么情况下或操作过程中遇到,在上述条件下是否总是出现等。 float32型tif(0~1)会将数据全部替换为0,显示图像为黑色。

    解决方法 rgb.append(np.uint16(self.src_data.read(b))) --> rgb.append(self.src_data.read(b))

    bug solved 
    opened by yangweiguang213 2
  • 宫格功能时,AttributeError: 'NoneType' object has no attribute 'getGrid'报错。

    宫格功能时,AttributeError: 'NoneType' object has no attribute 'getGrid'报错。

    bug描述 在使用宫格功能时,在点击”保存每个宫格的标签“后,自动弹出文件夹,点击相应文件夹(有要求吗?)后,再次点击下一个要标注的宫格时,程序自动退出。且出现如下报错: AttributeError: 'NoneType' object has no attribute 'getGrid'

    截屏 image

    运行环境(请尽量填写,这可以帮助我们定位问题):

    • 系统: Windows
    • 安装方式:pip
    • 软件版本:2.3(最新)
    bug 
    opened by Yanghanwa 0
  • 启动时protobuf 报错

    启动时protobuf 报错

    bug描述 启动时protobuf 报错, 是否考虑升级protobuf 或者在pip包中指定protobuf版本

    截屏

    # eiseg                                                                                                                                                                                                                                   
    Traceback (most recent call last):
      File "/Users/tachao/miniconda3/bin/eiseg", line 5, in <module>
        from eiseg.run import main
      File "/Users/tachao/miniconda3/lib/python3.9/site-packages/eiseg/run.py", line 25, in <module>
        from app import APP_EISeg  # 导入带槽的界面
      File "/Users/tachao/miniconda3/lib/python3.9/site-packages/eiseg/app.py", line 34, in <module>
        from controller import InteractiveController
      File "/Users/tachao/miniconda3/lib/python3.9/site-packages/eiseg/controller.py", line 23, in <module>
        import paddle
      File "/Users/tachao/miniconda3/lib/python3.9/site-packages/paddle/__init__.py", line 25, in <module>
        from .framework import monkey_patch_variable
      File "/Users/tachao/miniconda3/lib/python3.9/site-packages/paddle/framework/__init__.py", line 17, in <module>
        from . import random  # noqa: F401
      File "/Users/tachao/miniconda3/lib/python3.9/site-packages/paddle/framework/random.py", line 16, in <module>
        import paddle.fluid as fluid
      File "/Users/tachao/miniconda3/lib/python3.9/site-packages/paddle/fluid/__init__.py", line 36, in <module>
        from . import framework
      File "/Users/tachao/miniconda3/lib/python3.9/site-packages/paddle/fluid/framework.py", line 35, in <module>
        from .proto import framework_pb2
      File "/Users/tachao/miniconda3/lib/python3.9/site-packages/paddle/fluid/proto/framework_pb2.py", line 33, in <module>
        _descriptor.EnumValueDescriptor(
      File "/Users/tachao/miniconda3/lib/python3.9/site-packages/google/protobuf/descriptor.py", line 755, in __new__
        _message.Message._CheckCalledFromGeneratedFile()
    TypeError: Descriptors cannot not be created directly.
    If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
    If you cannot immediately regenerate your protos, some other possible workarounds are:
     1. Downgrade the protobuf package to 3.20.x or lower.
     2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
    
    More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
    
    

    运行环境(请尽量填写,这可以帮助我们定位问题):

    • 系统: [Mac os/Linux]
    • 安装方式:[pip]
    • 软件版本:[0.5.0]
    bug 
    opened by TaChao 1
  • Hope to add the function of converting json format labels to coco format labels in one click

    Hope to add the function of converting json format labels to coco format labels in one click

    Currently, if we have labeled a batch of images in JSON format, we can only re-label the image if we want to save the label to coco format. This is not very convenient for users, i hope the official can add this function, or can also release the script to achieve such a function. I saw that this software script only semantic tags into instance tags, so few scripts seem to be inconsistent with such a powerful software.

    opened by Leon-Brant 0
Releases(v1.0.5)
Yas CRNN model training - Yet Another Genshin Impact Scanner

Yas-Train Yet Another Genshin Impact Scanner 又一个原神圣遗物导出器 介绍 该仓库为 Yas 的模型训练程序 相关资料 MobileNetV3 CRNN 使用 假设你会设置基本的pytorch环境。 生成数据集 python main.py gen 训练

wormtql 18 Jan 08, 2023
A simple and lightweight genetic algorithm for optimization of any machine learning model

geneticml This package contains a simple and lightweight genetic algorithm for optimization of any machine learning model. Installation Use pip to ins

Allan Barcelos 8 Aug 10, 2022
Leveraging Instance-, Image- and Dataset-Level Information for Weakly Supervised Instance Segmentation

Leveraging Instance-, Image- and Dataset-Level Information for Weakly Supervised Instance Segmentation This paper has been accepted and early accessed

Yun Liu 39 Sep 20, 2022
ONNX Runtime Web demo is an interactive demo portal showing real use cases running ONNX Runtime Web in VueJS.

ONNX Runtime Web demo is an interactive demo portal showing real use cases running ONNX Runtime Web in VueJS. It currently supports four examples for you to quickly experience the power of ONNX Runti

Microsoft 58 Dec 18, 2022
Grad2Task: Improved Few-shot Text Classification Using Gradients for Task Representation

Grad2Task: Improved Few-shot Text Classification Using Gradients for Task Representation Prerequisites This repo is built upon a local copy of transfo

Jixuan Wang 10 Sep 28, 2022
Image Segmentation and Object Detection in Pytorch

Image Segmentation and Object Detection in Pytorch Pytorch-Segmentation-Detection is a library for image segmentation and object detection with report

Daniil Pakhomov 732 Dec 10, 2022
This is a pytorch implementation of the NeurIPS paper GAN Memory with No Forgetting.

GAN Memory for Lifelong learning This is a pytorch implementation of the NeurIPS paper GAN Memory with No Forgetting. Please consider citing our paper

Miaoyun Zhao 43 Dec 27, 2022
PyTorch Lightning + Hydra. A feature-rich template for rapid, scalable and reproducible ML experimentation with best practices. ⚡🔥⚡

Lightning-Hydra-Template A clean and scalable template to kickstart your deep learning project 🚀 ⚡ 🔥 Click on Use this template to initialize new re

Łukasz Zalewski 2.1k Jan 09, 2023
Code for "Finding Regions of Heterogeneity in Decision-Making via Expected Conditional Covariance" at NeurIPS 2021

Finding Regions of Heterogeneity in Decision-Making via Expected Conditional Covariance Justin Lim, Christina X Ji, Michael Oberst, Saul Blecker, Leor

Sontag Lab 3 Feb 03, 2022
Link prediction using Multiple Order Local Information (MOLI)

Understanding the network formation pattern for better link prediction Authors: [e

Wu Lab 0 Oct 18, 2021
[ECCV 2020] Gradient-Induced Co-Saliency Detection

Gradient-Induced Co-Saliency Detection Zhao Zhang*, Wenda Jin*, Jun Xu, Ming-Ming Cheng ⭐ Project Home » The official repo of the ECCV 2020 paper Grad

Zhao Zhang 35 Nov 25, 2022
Phylogeny Partners

Phylogeny-Partners Two states models Instalation You may need to install the cython, networkx, numpy, scipy package: pip install cython, networkx, num

1 Sep 19, 2022
This program automatically runs Python code copied in clipboard

CopyRun This program runs Python code which is copied in clipboard WARNING!! USE AT YOUR OWN RISK! NO GUARANTIES IF ANYTHING GETS BROKEN. DO NOT COPY

vertinski 4 Sep 10, 2021
[NeurIPS 2020] Blind Video Temporal Consistency via Deep Video Prior

pytorch-deep-video-prior (DVP) Official PyTorch implementation for NeurIPS 2020 paper: Blind Video Temporal Consistency via Deep Video Prior TensorFlo

Yazhou XING 90 Oct 19, 2022
Parsing, analyzing, and comparing source code across many languages

Semantic semantic is a Haskell library and command line tool for parsing, analyzing, and comparing source code. In a hurry? Check out our documentatio

GitHub 8.6k Dec 28, 2022
Official repository of "BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment"

BasicVSR_PlusPlus (CVPR 2022) [Paper] [Project Page] [Code] This is the official repository for BasicVSR++. Please feel free to raise issue related to

Kelvin C.K. Chan 227 Jan 01, 2023
YoHa - A practical hand tracking engine.

YoHa - A practical hand tracking engine.

2k Jan 06, 2023
[ICRA2021] Reconstructing Interactive 3D Scene by Panoptic Mapping and CAD Model Alignment

Interactive Scene Reconstruction Project Page | Paper This repository contains the implementation of our ICRA2021 paper Reconstructing Interactive 3D

97 Dec 28, 2022
计算机视觉中用到的注意力模块和其他即插即用模块PyTorch Implementation Collection of Attention Module and Plug&Play Module

PyTorch实现多种计算机视觉中网络设计中用到的Attention机制,还收集了一些即插即用模块。由于能力有限精力有限,可能很多模块并没有包括进来,有任何的建议或者改进,可以提交issue或者进行PR。

PJDong 599 Dec 23, 2022
Learning to Reconstruct 3D Manhattan Wireframes from a Single Image

Learning to Reconstruct 3D Manhattan Wireframes From a Single Image This repository contains the PyTorch implementation of the paper: Yichao Zhou, Hao

Yichao Zhou 50 Dec 27, 2022