A few Windows specific scripts for PyTorch

Overview

It is a repo that contains scripts that makes using PyTorch on Windows easier.

Easy Installation

Update: Starting from 0.4.0, you can go to the official site for installation steps. The packages here will not be updated. If you just want to install PyTorch as soon as possible, you can try this one out. The current version of the conda package for PyTorch is 0.3.1. You'll need Anaconda first. And then type in the following commands.

# If your main Python version is not 3.5 or 3.6
conda create -n test python=3.6 numpy pyyaml mkl

# for CPU only packages
conda install -c peterjc123 pytorch-cpu

# for Windows 10 and Windows Server 2016, CUDA 8
conda install -c peterjc123 pytorch

# for Windows 10 and Windows Server 2016, CUDA 9
conda install -c peterjc123 pytorch cuda90

# for Windows 7/8/8.1 and Windows Server 2008/2012, CUDA 8
conda install -c peterjc123 pytorch_legacy

Plus: The support for old NV cards (Compute Capability <= 5.0) is over. There're mainly two ways to resolve this:

  1. You can install legacy packages. See description in this section below.
  2. Install CI packages. However, you have to handle the dependencies by yourself. See About CI packages for details.

If there's conflict against vc14, you may see workground here. Sometimes the new packages may not work, when that happens, you may try the legacy packages here. If you are from China, then the files are stored in Baidu Netdisk. You can access them through this link.

About CI packages

There may be chances that the conda package is stale and you want to try out new features. For this purpose, the CI packages are generated.

System All
Windows CPU (master) Build status
Windows GPU (master) Build status
Windows CPU (0.4.0) Build status
Windows GPU (0.4.0) Build status
Windows CPU (0.4.1) Build status
Windows GPU (0.4.1) Build status

About the nightly packages

An additional CI is setup to run every night so that you can also get the latest packages. The installation steps can be seen here.

Configuration / Python version 3.5 3.6 3.7
Windows x64 CPU (Conda) Build status Build status Build status
Windows x64 CPU (Wheels) Build status Build status Build status
Windows x64 CUDA 9.2 (Conda) Build status Build status Build status
Windows x64 CUDA 9.2 (Wheels) Build status Build status Build status
Windows x64 CUDA 10.0 (Conda) Build status Build status Build status
Windows x64 CUDA 10.0 (Wheels) Build status Build status Build status

And here are the libtorch packages. Only shared build is supported now. Please click the badge to download the latest package for a specific configuration.

Configuration / LibTorch version Shared library
Windows x64 CPU (Release) Build status
Windows x64 CPU (Debug) Build status
Windows x64 CUDA 9.2 (Release) Build status
Windows x64 CUDA 9.2 (Debug) Build status
Windows x64 CUDA 10.0 (Release) Build status
Windows x64 CUDA 10.0 (Debug) Build status

How to find the package and install?

You can first click the icon to get to the main page of the CI system for a certain package. You can see a few jobs under that page. There're several variables that is used to distinguish the packages.

  • PYTHON_VERSION : This one indicates the python version it use.

  • TORCH_CUDA_ARCH_LIST : It implies the architecture of the GPU, only Pascal, Maxwell and Kepler is supported.

After the choice of the jobs, you can see the generated package if you click on Artifact on the navigation bar in the middle of the page.

Installation is simple, but there're some requirements.

# For all versions
# Windows x64
# Python x64 3.5 / 3.6
# MKL/Numpy/PyYAML

# For GPU versions
# CUDA 9 / 9.1
# cuDNN 7
# NVTX (Visual Studio Integration in CUDA. if it fails to be installed, you can extract
#       the CUDA installer exe and found the NVTX installer under the CUDAVisualStudioIntegration)

pip install numpy mkl intel-openmp
pip install pytorch-[version]-cp[pyversion]-cp[pyversion]m-win-amd64.whl
# Add [PythonRoot]\Library\bin into environment variable `PATH` and restart command prompt before using.

Note: You may face with the following issue. First, check that all the dependencies are installed. Second, try if a fresh virtual environment helps. If it helps, it may be an issue of the python version. And you can install VC 2017 Redist. Finally, you may have to manually compile PyTorch from source.

C:\Anaconda2\lib\site-packages\torch\__init__.pyc in ()
     39     os.environ['PATH'] = os.path.dirname(__file__) + '\\lib\\;' + os.environ['PATH']
     40
---> 41     from torch._C import *
     42     __all__ += [name for name in dir(_C)
     43                 if name[0] != '_' and

ImportError: DLL load failed: The specified procedure could not be found.

Compiling Examples

You can download it and put it in the PyTorch directory or use it in a standalone way. There're more details about MSVC 2017 setup in #23.

################################################################################
# These are the flags that you can set before running the scripts.
################################################################################

# You can specify which version you want to build
# If you omit it, it will build for the master branch on default
set PYTORCH_BUILD_VERSION=0.4.1

# If you want to build your own fork of PyTorch
set PYTORCH_REPO=pytorch

# If you want to build for a specific branch
# Note: If you specify PYTORCH_BUILD_VERSION too, it won't be used.
set PYTORCH_BRANCH=master

# If you want to skip the confirmation of the flags configured by our script
set NO_PROMPT=1

# If you want to continue build without a verified VS version (Safe for CPU builds)
set SKIP_VS_VER_CHECK=1

# If you want to skip prompt for building for Python 2.7 (Not recommended)
set SKIP_PY_VER_CHECK=1

# If you want to build for LibTorch
set BUILD_PYTHONLESS=1

################################################################################
# These are the available scripts that you can use.
################################################################################

# If you don't want to override the default settings
auto.bat

# If you don't want to compile with CUDA
cpu.bat

# If you want to compile with CUDA 8
cuda80.bat

# If you want to compile with CUDA 9
cuda90.bat

# If you want to compile with CUDA 9.1
cuda91.bat

# If you want to compile with CUDA 9.2
cuda92.bat

# If you want to compile with CUDA 10.0
cuda100.bat

Using Examples

# The main difference in Python between Windows and Unix systems is multiprocessing
# So please refactor your code into the following structure if you use DataLoader

import torch

def main():
    for i, (x, y) in dataloader:
        # do something here

if __name__ == '__main__':
    main()
Comments
  • issue: Solving environment: failed

    issue: Solving environment: failed

    I get the following error. I can install pytorch on the smaller test environment fine, but when I try to install it outside of the test env. I get the following error. I tried downgrading from Python 3.6.3 to 3.6.0 to no avail. Additionally, if I uninstall for the package that conda questions, I just get the same error with a new package (ex. I uninstalled xeus but qt now appeared. Any suggestions?

    Solving environment: failed
    
    UnsatisfiableError: The following specifications were found to be in conflict:
      - pytorch
      - qt -> icu=58 -> *[track_features=vc14]
      - qt -> icu=58 -> vc==14
    Use "conda info <package>" to see the dependencies for each package.
    
    opened by momonala 25
  • MSBuild can't find the project file for ATEN

    MSBuild can't find the project file for ATEN

    I am trying to install the latest version of PyTorch for Windows 10 by using cpu.bat, as my computer doesn't have a CUDA-compatible GPU. While running the script using VS prompt, I get the following error when it tries to build ATEN:

    -- Build files have been written to: C:/.../pytorch-scripts/pytorch/aten/build
    
    C:\...\pytorch-scripts\pytorch\aten\build>msbuild INSTALL.vcxproj /p:Configuration=Release
    Microsoft (R) Build Engine version 15.6.85.37198 for .NET Framework
    Copyright (C) Microsoft Corporation. All rights reserved.
    
    MSBUILD : error MSB1009: Project file does not exist.
    Switch: INSTALL.vcxproj
    
    C:\...\pytorch-scripts\pytorch\aten\build>IF NOT 1 == 0 exit 1
    

    but I can't find this file anywhere in the directory. Where is it supposed to be?

    opened by franchenstein 13
  • Add Caffe2 building scripts

    Add Caffe2 building scripts

    Hi @peterjc123,

    Finished building Caffe2 with Visual Studio 2015 and the script is ready. At the moment pytorch is cloned from my fork, where the build fully runs. When everything with adapting my pytorch PR will be agreed, of course will switch it to the main repo.

    The only exception found at the moment is CUDA 8 in Debug mode. In Release though, it still is able to build. Successfully tried CUDA 9.0, 9.1, python 2.7 and 3.5, cudnn 7.* in both Debug and Release.

    opened by ArutyunovG 9
  • vc 14.1 package missing DLLs

    vc 14.1 package missing DLLs

    The vc=14.1 package appears to be missing some DLLs so breaks conda build on my Win7 machine:

    See discussion at https://gitter.im/conda/conda-build?at=5a94acef35dd17022efa734d

    I do have VS2017 installed so this doesn't affect my normal day-to-day usage. conda build however creates a clean environment and it appears that when vc=14.1 is installed in this environment python can't start because of the missing DLLs (IIUC).

    opened by dhirschfeld 9
  • ImportError: DLL load failed: The specified module could not be found.

    ImportError: DLL load failed: The specified module could not be found.

    I have installed the right pytorch from pytorch v0.3.1 I have maxwell GPU QuadroM1200 which is of compute capability 5.0. I am using windows10 64bit. Despite of installing all required dependencies, I am getting the below error while importing torch in python interpreter.

    (fastai) C:\Users\Mahesh.Bhosale>python Python 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information.

    >>> import torch
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "C:\Users\Mahesh.Bhosale\AppData\Local\Continuum\anaconda3\envs\fastai\lib\site-packages\torch\__init__.py", line 76, in <module>
        from torch._C import *
    ImportError: DLL load failed: The specified module could not be found.
    >>> exit()
    

    I am not in a position or rather hesitating to build from the source at current moment can you please guide ?

    opened by bhosalems 7
  • Error when running cuda100.bat: Error : Internal Compiler error (codegen):

    Error when running cuda100.bat: Error : Internal Compiler error (codegen): "there was an error in verifying the lgenfe output!"

    Been trying to build Pytorch from source in Windows 10, really stuck here. Using Anaconda.

    INPUT

    (torchTest) C:\Users\pazzu\Documents\pytorch-scripts>cuda100 Ninja not found. It can be used to accelerate builds. You can install ninja using pip install ninja. MAGMA_HOME is set. MAGMA will be included in build. The flags after configuration: NO_CUDA= CMAKE_GENERATOR=Visual Studio 15 2017 Win64 CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0 DISTUTILS_USE_SDK=1 Do you wish to continue? (Y/N)

    OUTPUT .... "C:\Users\pazzu\Documents\pytorch-scripts\pytorch\build\INSTALL.vcxproj" (default target) (1) -> "C:\Users\pazzu\Documents\pytorch-scripts\pytorch\build\ALL_BUILD.vcxproj" (default target) (3) -> "C:\Users\pazzu\Documents\pytorch-scripts\pytorch\build\caffe2\AlgorithmsTest.vcxproj" (default target) (4) -> "C:\Users\pazzu\Documents\pytorch-scripts\pytorch\build\caffe2\caffe2_gpu.vcxproj" (default target) (22) -> (CustomBuild target) -> Error : Internal Compiler error (codegen): "there was an error in verifying the lgenfe output!" [C:\Users\pazzu\Docum ents\pytorch-scripts\pytorch\build\caffe2\caffe2_gpu.vcxproj]

    32851 Warning(s)
    1 Error(s)
    

    Time Elapsed 02:18:20.84

    (torchTest) C:\Users\pazzu\Documents\pytorch-scripts\pytorch\build>IF ERRORLEVEL 1 exit 1 Failed to run 'tools\build_pytorch_libs.bat --use-cuda --use-nnpack --use-mkldnn --use-qnnpack caffe2'

    opened by Maosef 6
  • \paragon was unexpected at this time

    \paragon was unexpected at this time

    I cloned the git and ran cuda91.bat, right after it checks out clang I hit this error:

    Submodule path 'torch/lib/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5'
    aten\src\ATen\common_with_cwrap.py
    1 File(s) copied
    \paragon was unexpected at this time.
    

    Re-running the script just gives me the same error right away.

    Tried searching it before asking, buuut...

    image

    And I tried seeing if it was just a package I needed...

    >conda install paragon
    Solving environment: failed
    
    PackagesNotFoundError: The following packages are not available from current channels:
    
      - paragon
    

    Any idea?

    opened by ghost 5
  • Only one core is used in cpu mode.

    Only one core is used in cpu mode.

    Current environment: windows 7, conda using python 3.6, fx-8350 cpu.

    When i test using torch.get_num_threads() it returns 1. I tried fixing with

    torch.set_num_threads(8)

    and the number returned by get_num_threads() updates - however pytorch still only uses one thread. I think it has to be specific to the windows port, because running pytorch within a virtual linux environment utilizes all available cores in the virtual machine.

    opened by psychosomaticdragon 5
  • PyTorch v0.3.1

    PyTorch v0.3.1

    Thanks for providing the Windows packages!

    Just wanted to mention that I'm trying to cut a new release of https://github.com/maciejkula/spotlight, and I have made the switch to v0.3.1.

    Can I give a hand with pushing the v0.3.1 conda packages in any way?

    opened by maciejkula 5
  • conda install -c peterjc123 pytorch cuda91 ?

    conda install -c peterjc123 pytorch cuda91 ?

    One of your suggestions on the pytorch git was I should build from your repo, but there doesn't seem to be a cuda91 package despite there being a cuda91.bat file present.

    opened by ghost 4
  • python stopped working using pytorch script

    python stopped working using pytorch script

    Hi,

    I don't know if my problem belong here. I'm using pytorch scripts on windows and i'm having issues converting my model to fp16 using ".half()" on the entire model and converting BatchNorm layers to .float() When I run my code it crashes and give me an error "Python stopped working" without showing any exception. I searched in the Event Viewer the reason why and got the following: Faulting application name: python.exe, version: 3.6.4150.1013, time stamp: 0x5a5e439a Faulting module name: _C.cp36-win_amd64.pyd, version: 0.0.0.0, time stamp: 0x5a91cbee Exception code: 0xc0000005 Fault offset: 0x000000000078cd95 Faulting process id: 0x2c78 Faulting application start time: 0x01d3b5ee018887d6

    opened by yana25 4
  • PyTorch for CUDA 9.2/10.2

    PyTorch for CUDA 9.2/10.2

    Hello,

    I ran "conda install -c peterjc123 pytorch" on Ubuntu 16.04 with CUDA 9.2, but Anaconda is showing an install for Pytorch 1.3.1 and not 0.3.1. The same happens for CUDA 10.2. Can I know why this happens?

    opened by CodexForster 0
  • Is the readme.md still up to date?

    Is the readme.md still up to date?

    In the README.md, I see:

    #### 1.

    Plus: The support for old NV cards (Compute Capability <= 5.0) is over. There're mainly two ways to resolve this:

    You can install legacy packages. See description in this section below. Install CI packages. However, you have to handle the dependencies by yourself. See About CI packages for details.

    Is this list perhaps not complete? I thought I can also install the most recent version from source from a Compute Capability of 3.5 onwards. CC 3.5 is only deprecated, but it is still accepted by the most recent versions of CUDA and cuDNN.

    #### 2.

    "About the nightly packages"

    If you click on the badges of the picture, the links do not lead to a download. As 3.8 is now the most recent Python, the badges are not needed anyway in my case, just mentioning it.

    image

    #### 3.

    From the whole readme, I do not get the clear information if I can get a simple installation of a most recent pytorch install from source with CUDA 11.0 and the most recent cuDNN for my older NV card with Compute Capability 3.5.

    For now, I will try instead installing from source, using the cloned git project of pytorch and using the guide of https://github.com/pytorch/pytorch#from-source. I want to compile it with mkl. It is not clear whether the simplified scripts of this project "pytorch-scripts" can help me doing that, since it seems to support only legacy versions of CUDA.

    opened by lorenzznerol 2
  • Static LibTorch mobile build for windows

    Static LibTorch mobile build for windows

    @peterjc123 Would it be possible to build a static LibTorch build for Windows, right now we are facing a couple of issues

    1. The final lib file is too big, it is around 740MB. the corresponding builds in Mac/Ubuntu is around 64MB. Even though we switched off the USE_MKL
    2. It gives us a linking error while loading a touch script model. https://github.com/pytorch/pytorch/issues/14367

    It would be helpful if you have any suggestions to get a static library

    opened by janakg 5
Releases(v0.3.0)
  • v0.3.0(Dec 6, 2017)

    Windows specific changes:

    Bug fixes

    1. Errors in backward leads to deadlock
    2. Memory leak in multiprocessing using DataLoader
    3. An indention bug in torch.cuda

    Features

    1. Add support for new CUDA and cuDNN versions
    2. Add support for Ninja and clcache when compiling

    Known issues

    1. Some tests will fail
    2. There is no support for torch.distributed, nccl and magma
    3. No support for python versions other than 3.5 and 3.6
    4. Don't set num_worker to a number more than 1. ( 0 is safer ) Wrap your main code entry with an if statement like this.
    if __name__ == '__main__':
    

    The packages are accessible here.

    Source code(tar.gz)
    Source code(zip)
Tutorial for surrogate gradient learning in spiking neural networks

SpyTorch A tutorial on surrogate gradient learning in spiking neural networks Version: 0.4 This repository contains tutorial files to get you started

Friedemann Zenke 203 Nov 28, 2022
Differentiable SDE solvers with GPU support and efficient sensitivity analysis.

PyTorch Implementation of Differentiable SDE Solvers This library provides stochastic differential equation (SDE) solvers with GPU support and efficie

Google Research 1.2k Jan 04, 2023
270 Dec 24, 2022
Training PyTorch models with differential privacy

Opacus is a library that enables training PyTorch models with differential privacy. It supports training with minimal code changes required on the cli

1.3k Dec 29, 2022
lookahead optimizer (Lookahead Optimizer: k steps forward, 1 step back) for pytorch

lookahead optimizer for pytorch PyTorch implement of Lookahead Optimizer: k steps forward, 1 step back Usage: base_opt = torch.optim.Adam(model.parame

Liam 318 Dec 09, 2022
Fast and Easy-to-use Distributed Graph Learning for PyTorch Geometric

Fast and Easy-to-use Distributed Graph Learning for PyTorch Geometric

Quiver Team 221 Dec 22, 2022
PyTorch Extension Library of Optimized Scatter Operations

PyTorch Scatter Documentation This package consists of a small extension library of highly optimized sparse update (scatter and segment) operations fo

Matthias Fey 1.2k Jan 07, 2023
Training RNNs as Fast as CNNs (https://arxiv.org/abs/1709.02755)

News SRU++, a new SRU variant, is released. [tech report] [blog] The experimental code and SRU++ implementation are available on the dev branch which

ASAPP Research 2.1k Jan 01, 2023
PyTorch implementation of TabNet paper : https://arxiv.org/pdf/1908.07442.pdf

README TabNet : Attentive Interpretable Tabular Learning This is a pyTorch implementation of Tabnet (Arik, S. O., & Pfister, T. (2019). TabNet: Attent

DreamQuark 2k Dec 27, 2022
A lightweight wrapper for PyTorch that provides a simple declarative API for context switching between devices, distributed modes, mixed-precision, and PyTorch extensions.

A lightweight wrapper for PyTorch that provides a simple declarative API for context switching between devices, distributed modes, mixed-precision, and PyTorch extensions.

Fidelity Investments 56 Sep 13, 2022
A simplified framework and utilities for PyTorch

Here is Poutyne. Poutyne is a simplified framework for PyTorch and handles much of the boilerplating code needed to train neural networks. Use Poutyne

GRAAL/GRAIL 534 Dec 17, 2022
PyTorch toolkit for biomedical imaging

farabio is a minimal PyTorch toolkit for out-of-the-box deep learning support in biomedical imaging. For further information, see Wikis and Docs.

San Askaruly 47 Dec 28, 2022
A code copied from google-research which named motion-imitation was rewrited with PyTorch

motor-system Introduction A code copied from google-research which named motion-imitation was rewrited with PyTorch. More details can get from this pr

NewEra 6 Jan 08, 2022
TorchSSL: A PyTorch-based Toolbox for Semi-Supervised Learning

TorchSSL: A PyTorch-based Toolbox for Semi-Supervised Learning

1k Dec 28, 2022
PyTorch extensions for fast R&D prototyping and Kaggle farming

Pytorch-toolbelt A pytorch-toolbelt is a Python library with a set of bells and whistles for PyTorch for fast R&D prototyping and Kaggle farming: What

Eugene Khvedchenya 1.3k Jan 05, 2023
A PyTorch implementation of Learning to learn by gradient descent by gradient descent

Intro PyTorch implementation of Learning to learn by gradient descent by gradient descent. Run python main.py TODO Initial implementation Toy data LST

Ilya Kostrikov 300 Dec 11, 2022
Implements pytorch code for the Accelerated SGD algorithm.

AccSGD This is the code associated with Accelerated SGD algorithm used in the paper On the insufficiency of existing momentum schemes for Stochastic O

205 Jan 02, 2023
TorchShard is a lightweight engine for slicing a PyTorch tensor into parallel shards

TorchShard is a lightweight engine for slicing a PyTorch tensor into parallel shards. It can reduce GPU memory and scale up the training when the model has massive linear layers (e.g., ViT, BERT and

Kaiyu Yue 275 Nov 22, 2022
A tutorial on "Bayesian Compression for Deep Learning" published at NIPS (2017).

Code release for "Bayesian Compression for Deep Learning" In "Bayesian Compression for Deep Learning" we adopt a Bayesian view for the compression of

Karen Ullrich 190 Dec 30, 2022
3D-RETR: End-to-End Single and Multi-View3D Reconstruction with Transformers

3D-RETR: End-to-End Single and Multi-View 3D Reconstruction with Transformers (BMVC 2021) Zai Shi*, Zhao Meng*, Yiran Xing, Yunpu Ma, Roger Wattenhofe

Zai Shi 36 Dec 21, 2022