Jittor is a high-performance deep learning framework based on JIT compiling and meta-operators.

Related tags

Deep Learningjittor
Overview

Jittor: a Just-in-time(JIT) deep learning framework

Jittor Logo

Quickstart | Install | Tutorial | Chinese

Jittor is a high-performance deep learning framework based on JIT compiling and meta-operators. The whole framework and meta-operators are compiled just-in-time. A powerful op compiler and tuner are integrated into Jittor. It allowed us to generate high-performance code with specialized for your model. Jittor also contains a wealth of high-performance model libraries, including: image recognition, detection, segmentation, generation, differentiable rendering, geometric learning, reinforcement learning, etc. .

The front-end language is Python. Module Design and Dynamic Graph Execution is used in the front-end, which is the most popular design for deeplearning framework interface. The back-end is implemented by high performance language, such as CUDA,C++.

Related Links:

The following example shows how to model a two-layer neural network step by step and train from scratch In a few lines of Python code.

import jittor as jt
from jittor import Module
from jittor import nn
import numpy as np

class Model(Module):
    def __init__(self):
        self.layer1 = nn.Linear(1, 10)
        self.relu = nn.Relu() 
        self.layer2 = nn.Linear(10, 1)
    def execute (self,x) :
        x = self.layer1(x)
        x = self.relu(x)
        x = self.layer2(x)
        return x

def get_data(n): # generate random data for training test.
    for i in range(n):
        x = np.random.rand(batch_size, 1)
        y = x*x
        yield jt.float32(x), jt.float32(y)


learning_rate = 0.1
batch_size = 50
n = 1000

model = Model()
optim = nn.SGD(model.parameters(), learning_rate)

for i,(x,y) in enumerate(get_data(n)):
    pred_y = model(x)
    dy = pred_y - y
    loss = dy * dy
    loss_mean = loss.mean()
    optim.step(loss_mean)
    print(f"step {i}, loss = {loss_mean.data.sum()}")

Contents

Quickstart

We provide some jupyter notebooks to help you quick start with Jittor.

Install

Jittor environment requirements:

  • System: Linux(e.g. Ubuntu/CentOS/Arch), macOS, or Windows, enviroment requirements of Linux and Mac are list below:

  • Python version >= 3.7

  • CPU compiler (require at least one of the following)

    • g++ (>=5.4.0)
    • clang (>=8.0)
  • GPU compiler (optional)

    • nvcc (>=10.0 for g++ or >=10.2 for clang)
  • GPU library: cudnn-dev (recommend tar file installation, reference link)

Windows requirements atr:

Note#1: macOS users have to install additional dependencies, see macOS install.

Jittor offers three ways to install: pip, docker, or manual.

Pip install

sudo apt install python3.7-dev libomp-dev
python3.7 -m pip install jittor
# or install from github(latest version)
# python3.7 -m pip install git+https://github.com/Jittor/jittor.git
python3.7 -m jittor.test.test_example

macOS install

Please first install additional dependencies with homebrew.

brew install [email protected] onednn libomp

Then you can install jittor through pip and run the example.

python3.7 -m pip install jittor
python3.7 -m jittor.test.test_example

Currently jittor only supports CPU in macOS.

Windows install

# check your python version(>=3.8)
python --version
python -m pip install jittor
# if conda is used
conda install pywin32

In Windows, jittor will automatically detect and install CUDA, please make sure your NVIDIA driver support CUDA 10.2 or above, or you can manually let jittor install CUDA for you:

python -m jittor_utils.install_cuda

Docker Install

We provide a Docker installation method to save you from configuring the environment. The Docker installation method is as follows:

# CPU only(Linux)
docker run -it --network host jittor/jittor
# CPU and CUDA(Linux)
docker run -it --network host --gpus all jittor/jittor-cuda
# CPU only(Mac and Windows)
docker run -it -p 8888:8888 jittor/jittor

manual install

We will show how to install Jittor in Ubuntu 16.04 step by step, Other Linux distributions may have similar commands.

Step 1: Choose your back-end compiler

# g++
sudo apt install g++ build-essential libomp-dev

# OR clang++-8
wget -O - https://raw.githubusercontent.com/Jittor/jittor/master/script/install_llvm.sh > /tmp/llvm.sh
bash /tmp/llvm.sh 8

Step 2: Install Python and python-dev

Jittor need python version >= 3.7.

sudo apt install python3.7 python3.7-dev

Step 3: Run Jittor

The whole framework is compiled Just-in-time. Let's install jittor via pip

git clone https://github.com/Jittor/jittor.git
sudo pip3.7 install ./jittor
export cc_path="clang++-8"
# if other compiler is used, change cc_path
# export cc_path="g++"
# export cc_path="icc"

# run a simple test
python3.7 -m jittor.test.test_example

if the test is passed, your Jittor is ready.

Optional Step 4: Enable CUDA

Using CUDA in Jittor is very simple, Just setup environment value nvcc_path

# replace this var with your nvcc location 
export nvcc_path="/usr/local/cuda/bin/nvcc" 
# run a simple cuda test
python3.7 -m jittor.test.test_cuda 

if the test is passed, your can use Jittor with CUDA by setting use_cuda flag.

import jittor as jt
jt.flags.use_cuda = 1

Optional Step 5: Test Resnet18 training

To check the integrity of Jittor, you can run Resnet18 training test. Note: 6G GPU RAM is requires in this test.

python3.7 -m jittor.test.test_resnet

if those tests are failed, please report bugs for us, and feel free to contribute ^_^

Tutorial

In the tutorial section, we will briefly explain the basic concept of Jittor.

To train your model with Jittor, there are only three main concepts you need to know:

  • Var: basic data type of jittor
  • Operations: Jittor'op is simular with numpy

Var

First, let's get started with Var. Var is the basic data type of jittor. Computation process in Jittor is asynchronous for optimization. If you want to access the data, Var.data can be used for synchronous data accessing.

import jittor as jt
a = jt.float32([1,2,3])
print (a)
print (a.data)
# Output: float32[3,]
# Output: [ 1. 2. 3.]

And we can give the variable a name.

a.name('a')
print(a.name())
# Output: a

Operations

Jittor'op is simular with numpy. Let's try some operations. We create Var a and b via operation jt.float32, and add them. Printing those variables shows they have the same shape and dtype.

import jittor as jt
a = jt.float32([1,2,3])
b = jt.float32([4,5,6])
c = a*b
print(a,b,c)
print(type(a), type(b), type(c))
# Output: float32[3,] float32[3,] float32[3,]
# Output: <class 'jittor_core.Var'> <class 'jittor_core.Var'> <class 'jittor_core.Var'>

Beside that, All the operators we used jt.xxx(Var, ...) have alias Var.xxx(...). For example:

c.max() # alias of jt.max(c)
c.add(a) # alias of jt.add(c, a)
c.min(keepdims=True) # alias of jt.min(c, keepdims=True)

if you want to know all the operation which Jittor supports. try help(jt.ops). All the operation you found in jt.ops.xxx, can be used via alias jt.xxx.

help(jt.ops)
# Output:
#   abs(x: core.Var) -> core.Var
#   add(x: core.Var, y: core.Var) -> core.Var
#   array(data: array) -> core.Var
#   binary(x: core.Var, y: core.Var, op: str) -> core.Var
#   ......

More

If you want to know more about Jittor, please check out the notebooks below:

Those notebooks can be started in your own computer by python3.7 -m jittor.notebook

Contributing

Jittor is still young. It may contain bugs and issues. Please report them in our bug track system. Contributions are welcome. Besides, if you have any ideas about Jittor, please let us know.

You can help Jittor in the following ways:

  • Citing Jittor in your paper
  • recommend Jittor to your friends
  • Contributing code
  • Contributed tutorials and documentation
  • File an issue
  • Answer jittor related questions
  • Light up the stars
  • Keep an eye on jittor
  • ......

Contact Us

Website: http://cg.cs.tsinghua.edu.cn/jittor/

Email: [email protected]

File an issue: https://github.com/Jittor/jittor/issues

QQ Group: 761222083

The Team

Jittor is currently maintained by the Tsinghua CSCG Group. If you are also interested in Jittor and want to improve it, Please join us!

Citation

@article{hu2020jittor,
  title={Jittor: a novel deep learning framework with meta-operators and unified graph execution},
  author={Hu, Shi-Min and Liang, Dun and Yang, Guo-Ye and Yang, Guo-Wei and Zhou, Wen-Yang},
  journal={Science China Information Sciences},
  volume={63},
  number={222103},
  pages={1--21},
  year={2020}
}

License

Jittor is Apache 2.0 licensed, as found in the LICENSE.txt file.

Comments
  • 在 ArchLinux 系统中 Jittor CUDA 无法使用

    在 ArchLinux 系统中 Jittor CUDA 无法使用

    按照 pip 方式安装后任何方式皆无法运行,下为执行日志,系统为 Arch Linux,Linux 系统内核版本为 Linux 5.13 rc4

    >>> import jittor
    [i 0604 13:52:10.823614 92 compiler.py:857] Jittor(1.2.3.14) src: /home/peter/.local/lib/python3.8/site-packages/jittor
    [i 0604 13:52:10.828678 92 compiler.py:858] g++ at /usr/bin/g++(11.1.0)
    [i 0604 13:52:10.828826 92 compiler.py:859] cache_path: /home/peter/.cache/jittor/default/g++
    [i 0604 13:52:10.839448 92 __init__.py:258] Found nvcc(11.3.58) at /opt/cuda/bin/nvcc.
    [i 0604 13:52:10.963726 92 __init__.py:258] Found gdb(10.2) at /usr/bin/gdb.
    [i 0604 13:52:10.970791 92 __init__.py:258] Found addr2line(2.36.1) at /usr/bin/addr2line.
    [i 0604 13:52:10.999637 92 compiler.py:915] py_include: -I/opt/anaconda/include/python3.8 -I/opt/anaconda/include/python3.8
    [i 0604 13:52:11.027781 92 compiler.py:917] extension_suffix: .cpython-38-x86_64-linux-gnu.so
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/home/peter/.local/lib/python3.8/site-packages/jittor/__init__.py", line 18, in <module>
        from . import compiler
      File "/home/peter/.local/lib/python3.8/site-packages/jittor/compiler.py", line 929, in <module>
        check_cache_compile()
      File "/home/peter/.local/lib/python3.8/site-packages/jittor/compiler.py", line 800, in check_cache_compile
        assert jit_utils.cc
    AssertionError
    
    opened by MrPeterJin 28
  • can not install llvm

    can not install llvm

    I followed the install instruction to install llvm on Ubuntu 16.04 but failed with

    Ign:27 http://ppa.launchpad.net/mc3man/trusty-media/ubuntu xenial/main i386 Packages
    Ign:28 http://ppa.launchpad.net/mc3man/trusty-media/ubuntu xenial/main all Packages
    Ign:29 http://ppa.launchpad.net/mc3man/trusty-media/ubuntu xenial/main Translation-en_US
    Ign:30 http://ppa.launchpad.net/mc3man/trusty-media/ubuntu xenial/main Translation-en
    Ign:31 http://ppa.launchpad.net/mc3man/trusty-media/ubuntu xenial/main amd64 DEP-11 Metadata
    Ign:32 http://ppa.launchpad.net/mc3man/trusty-media/ubuntu xenial/main DEP-11 64x64 Icons
    Ign:33 http://ppa.launchpad.net/jonathonf/python-3.6/ubuntu xenial/main amd64 Packages
    Ign:34 http://ppa.launchpad.net/jonathonf/python-3.6/ubuntu xenial/main i386 Packages
    Ign:21 http://ppa.launchpad.net/jonathonf/python-3.6/ubuntu xenial/main all Packages
    Ign:22 http://ppa.launchpad.net/jonathonf/python-3.6/ubuntu xenial/main Translation-en_US
    Ign:35 http://ppa.launchpad.net/jonathonf/python-3.6/ubuntu xenial/main Translation-en
    Ign:24 http://ppa.launchpad.net/jonathonf/python-3.6/ubuntu xenial/main amd64 DEP-11 Metadata
    Ign:25 http://ppa.launchpad.net/jonathonf/python-3.6/ubuntu xenial/main DEP-11 64x64 Icons
    Err:26 http://ppa.launchpad.net/mc3man/trusty-media/ubuntu xenial/main amd64 Packages
      404  Not Found [IP: 2001:67c:1560:8008::15 80]
    Ign:27 http://ppa.launchpad.net/mc3man/trusty-media/ubuntu xenial/main i386 Packages
    Ign:28 http://ppa.launchpad.net/mc3man/trusty-media/ubuntu xenial/main all Packages
    Ign:29 http://ppa.launchpad.net/mc3man/trusty-media/ubuntu xenial/main Translation-en_US
    Ign:30 http://ppa.launchpad.net/mc3man/trusty-media/ubuntu xenial/main Translation-en
    Ign:31 http://ppa.launchpad.net/mc3man/trusty-media/ubuntu xenial/main amd64 DEP-11 Metadata
    Ign:32 http://ppa.launchpad.net/mc3man/trusty-media/ubuntu xenial/main DEP-11 64x64 Icons
    Err:33 http://ppa.launchpad.net/jonathonf/python-3.6/ubuntu xenial/main amd64 Packages
      403  Forbidden [IP: 2001:67c:1560:8008::15 80]
    Ign:34 http://ppa.launchpad.net/jonathonf/python-3.6/ubuntu xenial/main i386 Packages
    Ign:35 http://ppa.launchpad.net/jonathonf/python-3.6/ubuntu xenial/main Translation-en
    Reading package lists... Done
    W: The repository 'http://ppa.launchpad.net/jonathonf/python-3.6/ubuntu xenial Release' does not have a Release file.
    N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
    N: See apt-secure(8) manpage for repository creation and user configuration details.
    W: The repository 'http://ppa.launchpad.net/mc3man/trusty-media/ubuntu xenial Release' does not have a Release file.
    N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
    N: See apt-secure(8) manpage for repository creation and user configuration details.
    E: Failed to fetch http://ppa.launchpad.net/mc3man/trusty-media/ubuntu/dists/xenial/main/binary-amd64/Packages  404  Not Found [IP: 2001:67c:1560:8008::15 80]
    E: Failed to fetch http://ppa.launchpad.net/jonathonf/python-3.6/ubuntu/dists/xenial/main/binary-amd64/Packages  403  Forbidden [IP: 2001:67c:1560:8008::15 80]
    E: Some index files failed to download. They have been ignored, or old ones used instead.
    

    How to solve this problem?

    opened by hiyyg 11
  • 无网络环境安装jittor:undefined symbol: _ZN6jittor4Node11tflag_count

    无网络环境安装jittor:undefined symbol: _ZN6jittor4Node11tflag_count

    执行顺序 git clone https://github.com/Jittor/jittor.git sudo pip install ./jittor

    python -m jittor.test.test_example

    错误 urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)>

    注释掉以下代码后 version_file = os.path.join(jittor_path, "version") if os.path.isfile(version_file) and not os.path.isdir(os.path.join(jittor_path, "src", "data")): with open(version_file, 'r') as f: version = f.read().strip() # key = f"{version}-{cc_type}-{'cuda' if has_cuda else 'cpu'}.o" key = f"{version}-g++-cpu" os_id = os_release["ID"] os_key = os_type.get(os_id, "ubuntu") if "os_key" in os.environ: os_key = os.environ['os_key'] if platform.machine()=='aarch64': os_key += '-aarch64' LOG.i("OS type:", os_id, " OS key:", os_key) key += '-' + os_key + '.o' # TODO: open the website extra_obj = os.path.join(cache_path, key) url = os.path.join("https://cg.cs.tsinghua.edu.cn/jittor/assets/build/"+key) jit_utils.download(url, extra_obj) files.append(extra_obj)

    另一个错误 ImportError: .cache/jittor/default/g++/jittor_core.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN6jittor4Node11tflag_countE

    opened by FullZing 10
  • 模型训练和预测时间不正常

    模型训练和预测时间不正常

    1.模型(ResNet)代码:

    import jittor as jt
    from jittor import nn
    from jittor import Module
    from jittor import init
    from jittor.contrib import concat, argmax_pool
    import time
    
    __all__ = ["ResNet", "ResNet18", "ResNet34", "ResNet50", "ResNet101", "ResNet152"]
    
    
    class BasicBlock(Module):
        expansion = 1
    
        def __init__(self, inplanes, planes, stride=1, downsample=None):
            self.conv1 = nn.Conv(inplanes, planes, kernel_size=3, stride=stride, padding=1, bias=False)
            self.bn1 = nn.BatchNorm(planes)
            self.relu = nn.Relu()
            self.conv2 = nn.Conv(planes, planes, kernel_size=3, stride=1, padding=1, bias=False)
            self.bn2 = nn.BatchNorm(planes)
            self.downsample = downsample
            self.stride = stride
            self.planes = planes
    
        def execute(self, x):
            residual = x
            out = self.conv1(x)
            out = self.bn1(out)
            out = self.relu(out)
            out = self.conv2(out)
            out = self.bn2(out)
    
            if self.downsample is not None:
                residual = self.downsample(x)
    
            out += residual
            out = self.relu(out)
            return out
    
    
    class Bottleneck(Module):
        expansion = 4
    
        def __init__(self, inplanes, planes, stride=1, downsample=None):
            self.conv1 = nn.Conv(inplanes, planes, kernel_size=1, bias=False)
            self.bn1 = nn.BatchNorm(planes)
            self.conv2 = nn.Conv(planes, planes, kernel_size=3, stride=stride, padding=1, bias=False)
            self.bn2 = nn.BatchNorm(planes)
            self.conv3 = nn.Conv(planes, planes * self.expansion, kernel_size=1, bias=False)
            self.bn3 = nn.BatchNorm(planes * self.expansion)
            self.relu = nn.Relu()
            self.downsample = downsample
            self.stride = stride
    
        def execute(self, x):
            residual = x
    
            out = self.conv1(x)
            out = self.bn1(out)
            out = self.relu(out)
    
            out = self.conv2(out)
            out = self.bn2(out)
            out = self.relu(out)
    
            out = self.conv3(out)
            out = self.bn3(out)
    
            if self.downsample is not None:
                residual = self.downsample(x)
    
            out += residual
            out = self.relu(out)
            return out
    
    
    class ResNet(Module):
        def __init__(self, block, layers, num_classes=1000):
            self.inplanes = 64
            self.conv1 = nn.Conv(3, 64, kernel_size=7, stride=2, padding=3, bias=False)
            self.bn1 = nn.BatchNorm(64)
            self.relu = nn.Relu()
            self.maxpool = nn.Pool(kernel_size=3, stride=2, padding=1)
            self.layer1 = self._make_layer(block, 64, layers[0])
            self.layer2 = self._make_layer(block, 128, layers[1], stride=2)
            self.layer3 = self._make_layer(block, 256, layers[2], stride=2)
            self.layer4 = self._make_layer(block, 512, layers[3], stride=2)
            self.avgpool = nn.Pool(7, stride=1, op="mean")
            self.fc = nn.Linear(512 * block.expansion, num_classes)
    
        def _make_layer(self, block, planes, blocks, stride=1):
            downsample = None
            if stride != 1 or self.inplanes != planes * block.expansion:
                downsample = nn.Sequential(
                    nn.Conv(self.inplanes, planes * block.expansion,
                            kernel_size=1, stride=stride, bias=False),
                    nn.BatchNorm(planes * block.expansion),
                )
    
            layers = []
            layers.append(block(self.inplanes, planes, stride, downsample))
            self.inplanes = planes * block.expansion
            for i in range(1, blocks):
                layers.append(block(self.inplanes, planes))
    
            return nn.Sequential(*layers)
    
        def execute(self, x):
            x = self.conv1(x)
            x = self.bn1(x)
            x = self.relu(x)
            x = self.maxpool(x)
            x = self.layer1(x)
            x = self.layer2(x)
            x = self.layer3(x)
            x = self.layer4(x)
    
            x = self.avgpool(x)
            x = jt.reshape(x, [x.shape[0], -1])
            x = self.fc(x)
    
            return x
    
    
    def ResNet18():
        model = ResNet(BasicBlock, [2, 2, 2, 2])
        return model
    
    
    def ResNet34():
        model = ResNet(BasicBlock, [3, 4, 6, 3])
        return model
    
    
    def ResNet50():
        model = ResNet(Bottleneck, [3, 4, 6, 3])
        return model
    
    
    def ResNet101():
        model = ResNet(Bottleneck, [3, 4, 23, 3])
        return model
    
    
    def ResNet152():
        model = ResNet(Bottleneck, [3, 8, 36, 3])
        return model
    

    2.模型VGG代码

    # ***************************************************************
    # Copyright (c) 2020 Jittor. Authors:
    #     Guoye Yang <[email protected]>
    #     Dun Liang <[email protected]>.
    # All Rights Reserved.
    # This file is subject to the terms and conditions defined in
    # file 'LICENSE.txt', which is part of this source code package.
    # ***************************************************************
    import jittor as jt
    from jittor import nn
    
    
    __all__ = ['VGG', 'VGG11', 'VGG11_bn', 'VGG13', 'VGG13_bn', 'VGG16', 'VGG16_bn',
        'VGG19_bn', 'VGG19']
    
    
    class VGG(nn.Module):
    
        def __init__(self, features, num_classes=1000, init_weights=True):
            super(VGG, self).__init__()
            self.features = features
            self.classifier = nn.Sequential(
                nn.Linear(512 * 7 * 7, 4096),
                nn.ReLU(),
                nn.Dropout(),
                nn.Linear(4096, 4096),
                nn.ReLU(),
                nn.Dropout(),
                nn.Linear(4096, num_classes),
            )
    
        def execute(self, x):
            x = self.features(x)
            x = jt.reshape(x, [x.shape[0],-1])
            x = self.classifier(x)
            return x
    
    def make_layers(cfg, batch_norm=False):
        layers = []
        in_channels = 3
        for v in cfg:
            if v == 'M':
                layers += [nn.Pool(kernel_size=2, stride=2, op="maximum")]
            else:
                conv2d = nn.Conv(in_channels, v, kernel_size=3, padding=1)
                if batch_norm:
                    layers += [conv2d, nn.BatchNorm(v), nn.ReLU()]
                else:
                    layers += [conv2d, nn.ReLU()]
                in_channels = v
        return nn.Sequential(*layers)
    
    
    cfgs = {
        'A': [64, 'M', 128, 'M', 256, 256, 'M', 512, 512, 'M', 512, 512, 'M'],
        'B': [64, 64, 'M', 128, 128, 'M', 256, 256, 'M', 512, 512, 'M', 512, 512, 'M'],
        'D': [64, 64, 'M', 128, 128, 'M', 256, 256, 256, 'M', 512, 512, 512, 'M', 512, 512, 512, 'M'],
        'E': [64, 64, 'M', 128, 128, 'M', 256, 256, 256, 256, 'M', 512, 512, 512, 512, 'M', 512, 512, 512, 512, 'M'],
    }
    
    
    def _vgg(arch, cfg, batch_norm, **kwargs):
        model = VGG(make_layers(cfgs[cfg], batch_norm=batch_norm), **kwargs)
        return model
    
    
    def VGG11(**kwargs):
        r"""VGG 11-layer model (configuration "A") from
        `"Very Deep Convolutional Networks For Large-Scale Image Recognition" <https://arxiv.org/pdf/1409.1556.pdf>`_
        """
        return _vgg('vgg11', 'A', False, **kwargs)
    
    
    def VGG11_bn(**kwargs):
        r"""VGG 11-layer model (configuration "A") with batch normalization
        `"Very Deep Convolutional Networks For Large-Scale Image Recognition" <https://arxiv.org/pdf/1409.1556.pdf>`_
        """
        return _vgg('vgg11_bn', 'A', True, **kwargs)
    
    
    def VGG13(**kwargs):
        r"""VGG 13-layer model (configuration "B")
        `"Very Deep Convolutional Networks For Large-Scale Image Recognition" <https://arxiv.org/pdf/1409.1556.pdf>`_
        """
        return _vgg('vgg13', 'B', False, **kwargs)
    
    
    def VGG13_bn(**kwargs):
        r"""VGG 13-layer model (configuration "B") with batch normalization
        `"Very Deep Convolutional Networks For Large-Scale Image Recognition" <https://arxiv.org/pdf/1409.1556.pdf>`_
        """
        return _vgg('vgg13_bn', 'B', True, **kwargs)
    
    
    def VGG16(**kwargs):
        r"""VGG 16-layer model (configuration "D")
        `"Very Deep Convolutional Networks For Large-Scale Image Recognition" <https://arxiv.org/pdf/1409.1556.pdf>`_
        """
        return _vgg('vgg16', 'D', False, **kwargs)
    
    
    def VGG16_bn(**kwargs):
        r"""VGG 16-layer model (configuration "D") with batch normalization
        `"Very Deep Convolutional Networks For Large-Scale Image Recognition" <https://arxiv.org/pdf/1409.1556.pdf>`_
        """
        return _vgg('vgg16_bn', 'D', True, **kwargs)
    
    
    def VGG19(**kwargs):
        r"""VGG 19-layer model (configuration "E")
        `"Very Deep Convolutional Networks For Large-Scale Image Recognition" <https://arxiv.org/pdf/1409.1556.pdf>`_
        """
        return _vgg('vgg19', 'E', False, **kwargs)
    
    
    def VGG19_bn(**kwargs):
        r"""VGG 19-layer model (configuration 'E') with batch normalization
        `"Very Deep Convolutional Networks For Large-Scale Image Recognition" <https://arxiv.org/pdf/1409.1556.pdf>`_
        """
        return _vgg('vgg19_bn', 'E', True, **kwargs)
    
    

    3.模型测速代码:

    import argparse
    import os
    import re
    import subprocess
    import time
    import models.resnet as res_models
    import models.vgg as vgg_models
    import numpy as np
    from jittor import nn
    import jittor as jt
    jt.flags.use_cuda = 1
    
    
    def parse_args():
        parser = argparse.ArgumentParser()
        parser.add_argument('-a', '--arch', default='ResNet18', type=str)
    
        parser.add_argument('--batch-size', type=int, default=64)  # 128
        parser.add_argument('--learning-rate', type=float, default=0.025)
        parser.add_argument('--momentum', type=float, default=0.9)
        parser.add_argument('--weight-decay', type=float, default=1e-4)
    
        parser.add_argument('--dynamic', action='store_true')
        parser.add_argument('--single', action='store_false')
        parser.add_argument('--compiled', action='store_true')
        parser.add_argument('--step', type=int, default=100)
    
        return parser.parse_args()
    
    
    def main():
        args = parse_args()
        worker(args)
    
    def worker(args):
        if 'ResNet' in args.arch:
            model = res_models.__dict__[args.arch]()
        elif 'VGG' in args.arch:
            model = vgg_models.__dict__[args.arch]()
        else:
            raise NotImplementedError
    
        def train_func():
            model.train()
    
            logits = model(image)
            loss = nn.cross_entropy_loss(logits, label, ignore_index=255)
            optimizer.step(loss)
    
        def eval_func():
            model.eval()
            logits = model(image)
    
    
        image = np.random.random([args.batch_size, 3, 224, 224]).astype(np.float32)
        label = np.random.randint(1000, size=[args.batch_size , 1])
        image = jt.float32(image)
        label = jt.int64(label)
        optimizer = nn.SGD(model.parameters(), args.learning_rate, args.momentum, args.weight_decay)
    
    
        for i in range(10):
            train_func()
    
        train_start_time = time.time()
        for i in range(args.step):
            train_func()
    
        train_time = (time.time() - train_start_time) / args.step
    
        eval_start_time = time.time()
        for i in range(args.step):
            eval_func()
    
        eval_time = (time.time() - eval_start_time) / args.step
        stdout = subprocess.getoutput('nvidia-smi')
        mem = re.findall(r'\|  (.*?)MiB /', stdout)[0].strip()
    
        print('Jittor, Model:{0},Dy:{1},#GPU:{2},batch_size:{3},mem:{4}M, avg_train_time:{5:.3f}ms, avg_eval_time:{6:.3f}ms'. \
            format(args.arch, args.dynamic, 1, args.batch_size, mem, train_time * 1000, eval_time * 1000))
    
    
    if __name__ == "__main__":
        main()
    

    模型测速

    • 测试设置:batch_size=64, GPU:1张1080ti, 测试迭代次数: 100;

      • 测试模型代码由jittor repo给出,
      • 训练和预测代码根据jittor exmaple写出。
    • 测试结果

    |Model | avg training time/iter | avg evaluate time/iter| |------- |--- | --- | ResNet-18 | 107.17ms | 1.26ms VGG-16 | 1.44ms |0.44ms

    • 从测试结果看,提出以下问题: 1.ResNet和VGG都存在inference time非常短,其他深度学习框架约几十毫秒。请问是上述测试代码中,对于jittor的使用存在问题么? 2.VGG-16的每个iter训练时间也非常短,极不正常。 期望得到作者的解答,非常感谢。
    opened by larenzhang 9
  • conda环境中使用pip安装出现无法找到glibcxx错误

    conda环境中使用pip安装出现无法找到glibcxx错误

    错误

    新建了一个conda环境后使用pip安装,测试安装,出现以下错误: image

    暂时解决

    如果使用系统自带的python和pip安装就没有问题,据悉是conda环境中修改了LD_LIBRARY_PATH导致无法找到glibcxx。 由于对自环境有刚需,所以请教解决方法,或加入conda install支持。

    环境

    system: openSUSE Tumbleweed (latest 5.16) compiler: gcc/g++11.2.1 cuda/nvcc: 11.6 with update 1 python: 3.8.12 conda 4.11.0/4.8.3

    opened by sleeplessai 7
  • 多代码并行的时候sync报错

    多代码并行的时候sync报错

    同时运行两个代码的时候第二个代码会报错:

    RuntimeError: Wrong inputs arguments, Please refer to examples(help(jt.sync)).

    Types of your inputs are: self = Var, args = (),

    The function declarations are: void sync(bool device_sync = false)

    Failed reason:[f 0909 17:29:40.576724 60 executor.cc:527] Execute fused operator(2324/4776) failed: [Op(0x5623cd93d830:0:0:1:i1:o1:s0,broadcast_to->0x5623cd93d8e0),Op(0x5623cf951be0:0: 0:1:i1:o1:s0,broadcast_to->0x5623cf951c90),Op(0x5623cf9534b0:0:0:1:i2:o1:s0,binary.multiply->0x5623cf953540),Op(0x5623cfb976e0:0:0:1:i1:o1:s0,reindex_reduce.add->0x5623cf9565c0),]

    Reason: [f 0909 17:29:39.839614 60 helper_cuda.h:126] CUDA error at ~/.cache/jittor/default/g++/jit/cudnn_conv_backward_x_Tx:float32__Ty:float32__Tw:float32__XFORMAT:abcd__WF ORMAT:oihw__YFOR...hash:74f24b7a5fa4fe17_op.cc:167 code=4( CUDNN_STATUS_INTERNAL_ERROR ) cudnnFindConvolutionBackwardDataAlgorithmEx( handle_, cudnnFdesc, w->ptr(), cudnnOdesc, y->ptr(), cudnnConvDesc, cudnnIdesc, x->ptr(), num_algos, &perf_count, perf_results, ws, max_ws_size) [e 0909 17:29:40.891578 60 helper_cuda.h:115] Peek CUDA error at ~/anaconda3/envs/py3/lib/python3.8/site-packages/jittor/src/mem/allocator/cuda_dual_allocator.h:101 code=700 ( cudaErrorIllegalAddress ) _cudaLaunchHostFunc(0, &to_free_allocation, 0) terminate called after throwing an instance of 'std::runtime_error'
    what(): [f 0909 17:29:42.170272 60 helper_cuda.h:126] CUDA error at ~/anaconda3/envs/py3/lib/python3.8/site-packages/jittor/extern/cuda/cudnn/src/cudnn_warper.cc:34 code= 4( CUDNN_STATUS_INTERNAL_ERROR ) cudnnDestroy(cudnn_handle)

    请问下是什么问题?

    opened by sunhm15 6
  • 未通过Jittor测试(test_example)

    未通过Jittor测试(test_example)

    按照标准流程手动安装,输出如下:

    > python -m jittor.test.test_example
    [i 1121 15:17:16.379761 12 __init__.py:257] Found g++(10.2.0) at g++
    [i 1121 15:17:16.420423 12 compiler.py:839] Jittor(1.2.1.1) src: /usr/lib/python3.8/site-packages/jittor
    [i 1121 15:17:16.420647 12 compiler.py:840] cache_path: /home/peter/.cache/jittor/default/g++
    [i 1121 15:17:16.427812 12 compiler.py:791] Found /usr/local/cuda/bin/nvcc(11.1.105) at /opt/cuda/bin/nvcc
    [i 1121 15:17:16.522639 12 __init__.py:249] Found gdb(10.1) at /usr/bin/gdb.
    [i 1121 15:17:16.529103 12 __init__.py:249] Found addr2line(2.35.1) at /usr/bin/addr2line.
    [i 1121 15:17:16.627024 12 compiler.py:889] pybind_include: -I/usr/include/python3.8 -I/usr/lib/python3.8/site-packages/pybind11/include
    [i 1121 15:17:16.677954 12 compiler.py:891] extension_suffix: .cpython-38-x86_64-linux-gnu.so
    g++: error: /usr/lib/python3.8/site-packages/jittor/src/utils/cache_compile.cc: Not a directory
    g++: error: /usr/lib/python3.8/site-packages/jittor/src/utils/log.cc: Not a directory
    g++: error: /usr/lib/python3.8/site-packages/jittor/src/utils/tracer.cc: Not a directory
    g++: error: /usr/lib/python3.8/site-packages/jittor/src/utils/jit_utils.cc: Not a directory
    Traceback (most recent call last):
      File "/usr/lib/python3.8/runpy.py", line 185, in _run_module_as_main
        mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
      File "/usr/lib/python3.8/runpy.py", line 111, in _get_module_details
        __import__(pkg_name)
      File "/usr/lib/python3.8/site-packages/jittor/__init__.py", line 16, in <module>
        from . import compiler
      File "/usr/lib/python3.8/site-packages/jittor/compiler.py", line 901, in <module>
        check_cache_compile()
      File "/usr/lib/python3.8/site-packages/jittor/compiler.py", line 775, in check_cache_compile
        recompile = compile(cc_path, cc_flags+f" {opt_flags} ", files, 'jit_utils_core'+extension_suffix, True)
      File "/usr/lib/python3.8/site-packages/jittor/compiler.py", line 69, in compile
        return do_compile(cmd)
      File "/usr/lib/python3.8/site-packages/jittor/compiler.py", line 47, in do_compile
        run_cmd(cmd)
      File "/usr/lib/python3.8/site-packages/jittor_utils/__init__.py", line 136, in run_cmd
        raise Exception(err_msg)
    Exception: Run cmd failed: g++ /usr/lib/python3.8/site-packages/jittor/src/utils/cache_compile.cc /usr/lib/python3.8/site-packages/jittor/src/utils/log.cc /usr/lib/python3.8/site-packages/jittor/src/utils/tracer.cc /usr/lib/python3.8/site-packages/jittor/src/utils/jit_utils.cc   -Wall -Werror -Wno-unknown-pragmas -std=c++14 -fPIC -march=native  -fdiagnostics-color=always -I/usr/include/python3.8 -I/usr/lib/python3.8/site-packages/pybind11/include -I/usr/lib/python3.8/site-packages/jittor/src   -O2    -lstdc++ -ldl -shared  -o /home/peter/.cache/jittor/default/g++/jit_utils_core.cpython-38-x86_64-linux-gnu.so
    

    系统基本配置:

    OS: Manjaro Linux x86_64 
    Host: 80RQ Lenovo Rescuer-15ISK
    Kernel: 5.9.8-2-MANJARO 
    
    opened by MrPeterJin 6
  • 性能测试及显存占用

    性能测试及显存占用

    import os
    import sys
    import numpy as np
    import jittor as jt
    import jittor.models as jtmodels
    import json
    import time
    jt.flags.use_cuda=1
    a=np.random.rand(5, 3,512,512).astype('float32')
    inputs=jt.array(a)
    res=m(inputs)
    t=time.time()
    r=[]
    for i in range(10):
        a=np.random.rand(5, 3,512,512).astype('float32')
        inputs=jt.array(a)
        res=m(inputs)
        r.append(res.max().numpy().tolist()[0])
    print(time.time()-t)
    

    在v100卡上对比当前版本的计图和pytorch1.4, 同样的程序计图平均在960ms,pytorch在950ms左右,感觉没有起到加速效果 同时计图占用了15516M显存,gc后会减少到6736M,显存占用较大, 请问针对这种推理方式有优化使用的方式么

    opened by feihuidiqiu 6
  • Better compiler path searching with environment varible

    Better compiler path searching with environment varible

    I'm using module for package management on a cluster, the current environment is shown as following:

    Currently Loaded Modulefiles:
      1) compiler/intel/2019.5.281    4) tools/mumps_seq/5.2.1        7) tools/matlab/R2017a
      2) mpi/intel/2019.5.281         5) tools/superlu/5.2.1          8) compiler/cuda/10.1
      3) tools/tmux/1.8               6) tools/suitesparse/5.6.0      9) interpreter/python/3.7-gpu
    

    Note that python 3.7 and nvcc 10.1 are both loaded, and currently c/c++ compiler is icc, so let's start jittor program

    [i 0518 20:13:45.097043 52 compiler.py:851] Jittor(1.2.3.8) src: /home/users/$MYNAME$/.local/lib/python3.7/site-packages/jittor
    [i 0518 20:13:45.097205 52 compiler.py:852] g++ at /software/local/bin/g++
    [i 0518 20:13:45.097302 52 compiler.py:853] cache_path: /home/users/$MYNAME$/.cache/jittor/default/g++
    which: no nvcc in (/usr/local/cuda/bin)
    [i 0518 20:13:45.195274 52 __init__.py:255] Found gdb(7.6.1) at /bin/gdb.
    [i 0518 20:13:45.212988 52 __init__.py:255] Found addr2line(7.3) at /bin/addr2line.
    [i 0518 20:13:45.338448 52 compiler.py:893] py_include: -I/software/python/virtualenv/py3.7-gpu/include/python3.7m -I/software/python/virtualenv/py3.7-gpu/include/python3.7m
    [i 0518 20:13:45.437864 52 compiler.py:895] extension_suffix: .cpython-37m-x86_64-linux-gnu.so
    

    I don't know why jittor is trying to obtain g++, which i haven't loaded (so the environment is not correct for g++) and why it hasn't found nvcc, maybe compilers should be found by using current environment path

    opened by yhl1219 5
  • 关于用jittor读取torch pth文件可能的错误

    关于用jittor读取torch pth文件可能的错误

    最近使用jittor读取torch嵌套pth文件发现这样一个问题:一旦torch的sequential或者嵌套module带有名字,jittor就无法读取(和jittor sequential数字索引有关。)因此用户必须按照数字顺序命名。我自己尝试使用回溯来是的用户可以任意命名。 另外还有一个小bug:打印module时,其中的LeakyReLU层会显示MakeModule,查阅文档似乎ReLU系层都存在这个问题。

    opened by Exusial 5
  • jittor.linalg.einsum: index out of bounds exception if pattern contains space

    jittor.linalg.einsum: index out of bounds exception if pattern contains space

    Minimal Reproduce

    import jittor
    a = jittor.ones((2,2))
    b = jittor.ones((2,2))
    jt.linalg.einsum('ac,     ac -> a', a, b) # <- one space is enough to cause the error
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/zpool-portable/shared/diffusion/diffusion-pytorch/lib/python3.10/site-packages/jittor/linalg.py", line 497, in einsum
        output_shape = [int(x) for x in einsum_outshape(string, args)]
      File "/zpool-portable/shared/diffusion/diffusion-pytorch/lib/python3.10/site-packages/jittor/linalg.py", line 495, in einsum_outshape
        return tuple(shps[(np_cpu.concatenate(inop[:-1])[:,None]==inop[-1]).argmax(0)].astype(np_cpu.int64))
    IndexError: index 4 is out of bounds for axis 0 with size 4
    

    Expected behavior

    The corresponding numpy einsum func has no such issue,

    c = numpy.ones((2,2))
    d = numpy.ones((2,2))
    nump.einsum('a    c   , a c ->  a ', c , d) # <-whatever space has no effect
    

    perhaps, need to strip any space in einsum_outshape or split by space?

    opened by alexfanqi 4
  • 无法开启cuda

    无法开启cuda

    Traceback (most recent call last): File "main.py", line 6, in jt.flags.use_cuda = 1 RuntimeError: Wrong inputs arguments, Please refer to examples(help(jt.Flags.use_cuda)).

    Types of your inputs are: self = Flags, arg = int,

    The function declarations are: void _set_use_cuda(int v) void _set_use_cuda(bool v)

    Failed reason:[f 1219 15:54:51.134013 56 cuda_flags.cc:44] Check failed: value==0 No CUDA found.

    opened by ckaiwen 0
  • CVE-2007-4559 Patch

    CVE-2007-4559 Patch

    Patching CVE-2007-4559

    Hi, we are security researchers from the Advanced Research Center at Trellix. We have began a campaign to patch a widespread bug named CVE-2007-4559. CVE-2007-4559 is a 15 year old bug in the Python tarfile package. By using extract() or extractall() on a tarfile object without sanitizing input, a maliciously crafted .tar file could perform a directory path traversal attack. We found at least one unsantized extractall() in your codebase and are providing a patch for you via pull request. The patch essentially checks to see if all tarfile members will be extracted safely and throws an exception otherwise. We encourage you to use this patch or your own solution to secure against CVE-2007-4559. Further technical information about the vulnerability can be found in this blog.

    If you have further questions you may contact us through this projects lead researcher Kasimir Schulz.

    opened by TrellixVulnTeam 0
  • RuntimeError when calling jt.matmul in cuda

    RuntimeError when calling jt.matmul in cuda

    Describe the bug

    RuntimeError occurs when calling jt.matmul in WSL on an NVIDIA GeForce MX450 laptop.

    Full Log

    [i 1031 23:09:50.689249 80 cuda_flags.cc:32] CUDA enabled.
    ---------------------------------------------------------------------------
    RuntimeError                              Traceback (most recent call last)
    <ipython-input-3-b8f9d3eceb14> in <module>
    ----> 1 jt.flags.use_cuda = 1; jt.matmul(jt.rand((1, 1, 100, 100)), jt.rand(((1, 16, 100, 1)))).shape
    
    ~/anaconda3/envs/gm-jittor/lib/python3.7/site-packages/jittor/nn.py in matmul(a, b)
        122             # a: [..., n, m], b: [..., m, k], c:[..., n, k]
        123             if jt.flags.use_cuda and jt.compile_extern.cublas_ops:
    --> 124                 return jt.compile_extern.cublas_ops.cublas_batched_matmul(a, b, 0, 0)
        125         shape = []
        126         len_c = max(len_a, len_b)
    
    RuntimeError: Wrong inputs arguments, Please refer to examples(help(jt.cublas_batched_matmul)).
    
    Types of your inputs are:
     self   = module,
     args   = (Var, Var, int, int, ),
    
    The function declarations are:
     VarHolder* cublas_batched_matmul(VarHolder* a, VarHolder* b,  bool trans_a,  bool trans_b)
    
    Failed reason:[f 1031 23:09:50.690284 80 cublas_batched_matmul_op.cc:75] Check failed a->shape[i](1) == b->shape[i](16) Something wrong ... Could you please report this issue?
    

    Minimal Reproduce

    # On CPU, jt.matmul() works
    jt.flags.use_cuda = 0; jt.matmul(jt.rand((1, 1, 100, 100)), jt.rand(((1, 16, 100, 1)))).shape
    # On GPU, it doesn`t work
    jt.flags.use_cuda = 1; jt.matmul(jt.rand((1, 1, 100, 100)), jt.rand(((1, 16, 100, 1)))).shape
    

    Possible Solution

    # Add repeat(1, 16, 1, 1) to explicitly specify the shape to broadcast
    jt.flags.use_cuda = 1; jt.matmul(jt.rand((1, 1, 100, 100)).repeat(1, 16, 1, 1), jt.rand(((1, 16, 100, 1)))).shape
    

    However, this solution results in large computational error.

    a = torch.randn(1,1,100,100).numpy()
    b = torch.randn(1,16,100,1).numpy()
    c_torch = torch.matmul(torch.tensor(a), torch.tensor(b)).numpy()
    c_jt = jt.matmul(jt.Var(a).repeat(1, 16, 1, 1), jt.Var(b)).numpy()
    np.testing.assert_almost_equal(c_torch, c_jt) # Passed
    
    jt.flags.use_cuda = 1
    c_jt = jt.matmul(jt.Var(a).repeat(1, 16, 1, 1), jt.Var(b)).numpy()
    np.testing.assert_almost_equal(c_torch, c_jt) # AssertionError
    
    AssertionError:
    Arrays are not almost equal to 7 decimals
    
    Mismatched elements: 1350 / 1600 (84.4%)
    Max absolute difference: 1.1444092e-05
    Max relative difference: 3.5474255e-05
     x: array([[[[  4.539161 ],
             [  5.9756746],
             [ 13.752586 ],...
     y: array([[[[  4.53916  ],
             [  5.9756722],
             [ 13.752583 ],...
    

    Expected behavior

    jt.matmul should work correctly on CUDA.

    opened by Purewhite2019 1
  • Caught segfault at address 0x0

    Caught segfault at address 0x0

    Describe the bug

    Mac import nerf 或运行python tools/run_net.py --config-file ./projects/ngp/configs/ngp_base.py --task test 的时候会崩溃

    Full Log

    [i 1022 00:01:50.683523 00 compiler.py:955] Jittor(1.3.5.21) src: /Users/mei/opt/anaconda3/lib/python3.7/site-packages/jittor [i 1022 00:01:50.734341 00 compiler.py:956] g++ at /usr/bin/g++(12.0.0) [i 1022 00:01:50.734457 00 compiler.py:957] cache_path: /Users/mei/.cache/jittor/jt1.3.5/g++12.0.0/py3.7.6/Darwin-20.6.0-x92/IntelRCoreTMi5x84/default [i 1022 00:01:51.089361 00 init.py:232] using 4 procs for compiling. [i 1022 00:01:51.343270 00 jit_compiler.cc:28] Load cc_path: /usr/bin/g++ clang: error: unsupported option '--showme:version' [i 1022 00:01:52.669768 00 compile_extern.py:517] mpicc not found, distribution disabled. Caught segfault at address 0x0, thread_name: '', flush log... [bt] Execution path: [bt] #1 1 jit_utils_core.cpython-37m-darwin.s 0x00000001058371b8 _ZN6jittor18segfault_sigactionEiP9__siginfoPv + 2216 [bt] #2 2 libsystem_platform.dylib 0x00007fff20821d7d _sigtramp + 29 [bt] #3 3 python 0x0000000104ea0cd1 _PyObject_Malloc + 17 [bt] #4 4 libc++abi.1.0.dylib 0x00000001059eebd8 __cxa_throw + 104 [bt] #5 5 jittor_core.cpython-37m-darwin.so 0x0000000105a0a671 _ZN6jittor15LogFatalVoidifyaaERNS_3LogE + 129 [bt] #6 6 jittor_core.cpython-37m-darwin.so 0x0000000105c1598d _ZN6jittor15setter_use_cudaEi + 253 [bt] #7 7 jittor_core.cpython-37m-darwin.so 0x0000000105c15880 _ZN6jittor12set_use_cudaERKi + 16 [bt] #8 8 jittor_core.cpython-37m-darwin.so 0x0000000105b9eb06 _ZZN6jittor18pyjt_def_jit_flagsEP7_objectEN5$_1158__invokeES1_S1_Pv + 374 [bt] #9 9 python 0x0000000104e9ef4b _PyObject_GenericSetAttrWithDict + 219 [bt] #10 10 python 0x0000000104e9dd40 PyObject_SetAttr + 96 [bt] #11 11 python 0x0000000104f76dfb _PyEval_EvalFrameDefault + 30555 [bt] #12 12 python 0x0000000104f6e46e _PyEval_EvalCodeWithName + 414 [bt] #13 13 python 0x0000000104f68e9b builtin_exec + 347 [bt] #14 14 python 0x0000000104e3ec0f _PyMethodDef_RawFastCallDict + 255 [bt] #15 15 python 0x0000000104e4031d PyCFunction_Call + 61 Segfault, exit

    Minimal Reproduce

    Reproduce this error with a file or several lines of code. If it is not possible, leave it blank.

    Expected behavior

    A clear and concise description of what you expected to happen.

    If you are submitting an issue for the first time, please refer to our guideline

    opened by MING410 1
  • Inconsistent behaviour in 'nn.CrossEntropyLoss()' for different ways to pass 'weight'

    Inconsistent behaviour in 'nn.CrossEntropyLoss()' for different ways to pass 'weight'

    Describe the bug

    在使用nn.CrossEntropyLoss()时,指定weight参数为jitter.ones(num_classes, dtype='float32')和置为None的运行结果不同。且指定包括全一在内的值后会带来梯度爆炸。

    Full Log

    这是指定了weight具体值的训练曲线。 accuracy loss

    这是不指定weight值的训练曲线。 accuracy loss

    Minimal Reproduce

    数据集是CIFAR-10,简单的Conv+Linear,类似于Jittor教程的MNIST分类网络。

    Expected behavior

    希望两者训练行为至少一致,最好是都不会发生梯度爆炸。

    opened by Ailon-Island 1
Releases(1.3.4.1)
COCO Style Dataset Generator GUI

A simple GUI-based COCO-style JSON Polygon masks' annotation tool to facilitate quick and efficient crowd-sourced generation of annotation masks and bounding boxes. Optionally, one could choose to us

Hans Krupakar 142 Dec 09, 2022
Alignment Attention Fusion framework for Few-Shot Object Detection

AAF framework Framework generalities This repository contains the code of the AAF framework proposed in this paper. The main idea behind this work is

Pierre Le Jeune 20 Dec 16, 2022
Implementation of "Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency"

Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency (ICCV2021) Paper Link: https://arxiv.org/abs/2107.11355 This implementation bui

32 Nov 17, 2022
My Body is a Cage: the Role of Morphology in Graph-Based Incompatible Control

My Body is a Cage: the Role of Morphology in Graph-Based Incompatible Control

yobi byte 29 Oct 09, 2022
Detection of drones using their thermal signatures from thermal camera through YOLO-V3 based CNN with modifications to encapsulate drone motion

Drone Detection using Thermal Signature This repository highlights the work for night-time drone detection using a using an Optris PI Lightweight ther

Chong Yu Quan 6 Dec 31, 2022
GANfolk: Using AI to create portraits of fictional people to sell as NFTs

GANfolk are AI-generated renderings of fictional people. Each image in the collection was created by a pair of Generative Adversarial Networks (GANs) with names and backstories also created with AI.

Robert A. Gonsalves 32 Dec 02, 2022
Text-to-Image generation

Generate vivid Images for Any (Chinese) text CogView is a pretrained (4B-param) transformer for text-to-image generation in general domain. Read our p

THUDM 1.3k Dec 29, 2022
Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning. CVPR 2018

Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning Tensorflow code and models for the paper: Large Scale Fine-Grained Categ

Yin Cui 187 Oct 01, 2022
Manifold-Mixup implementation for fastai V2

Manifold Mixup Unofficial implementation of ManifoldMixup (Proceedings of ICML 19) for fast.ai (V2) based on Shivam Saboo's pytorch implementation of

Nestor Demeure 16 Jul 25, 2022
Empower Sequence Labeling with Task-Aware Language Model

LM-LSTM-CRF Check Our New NER Toolkit 🚀 🚀 🚀 Inference: LightNER: inference w. models pre-trained / trained w. any following tools, efficiently. Tra

Liyuan Liu 838 Jan 05, 2023
Raster Vision is an open source Python framework for building computer vision models on satellite, aerial, and other large imagery sets

Raster Vision is an open source Python framework for building computer vision models on satellite, aerial, and other large imagery sets (including obl

Azavea 1.7k Dec 22, 2022
Library to enable Bayesian active learning in your research or labeling work.

Bayesian Active Learning (BaaL) BaaL is an active learning library developed at ElementAI. This repository contains techniques and reusable components

ElementAI 687 Dec 25, 2022
An Easy-to-use, Modular and Prolongable package of deep-learning based Named Entity Recognition Models.

DeepNER An Easy-to-use, Modular and Prolongable package of deep-learning based Named Entity Recognition Models. This repository contains complex Deep

Derrick 9 May 30, 2022
Official PyTorch implementation of "Adversarial Reciprocal Points Learning for Open Set Recognition"

Adversarial Reciprocal Points Learning for Open Set Recognition Official PyTorch implementation of "Adversarial Reciprocal Points Learning for Open Se

Guangyao Chen 78 Dec 28, 2022
PyTorch implementation of DUL (Data Uncertainty Learning in Face Recognition, CVPR2020)

PyTorch implementation of DUL (Data Uncertainty Learning in Face Recognition, CVPR2020)

Mouxiao Huang 20 Nov 15, 2022
Multiwavelets-based operator model

Multiwavelet model for Operator maps Gaurav Gupta, Xiongye Xiao, and Paul Bogdan Multiwavelet-based Operator Learning for Differential Equations In Ne

Gaurav 33 Dec 04, 2022
Implementation of "Learning to Match Features with Seeded Graph Matching Network" ICCV2021

SGMNet Implementation PyTorch implementation of SGMNet for ICCV'21 paper "Learning to Match Features with Seeded Graph Matching Network", by Hongkai C

87 Dec 11, 2022
The author's officially unofficial PyTorch BigGAN implementation.

BigGAN-PyTorch The author's officially unofficial PyTorch BigGAN implementation. This repo contains code for 4-8 GPU training of BigGANs from Large Sc

Andy Brock 2.6k Jan 02, 2023
efficient neural audio synthesis in the waveform domain

neural waveshaping synthesis real-time neural audio synthesis in the waveform domain paper • website • colab • audio by Ben Hayes, Charalampos Saitis,

Ben Hayes 169 Dec 23, 2022
An Official Repo of CVPR '20 "MSeg: A Composite Dataset for Multi-Domain Segmentation"

This is the code for the paper: MSeg: A Composite Dataset for Multi-domain Semantic Segmentation (CVPR 2020, Official Repo) [CVPR PDF] [Journal PDF] J

226 Nov 05, 2022