PyTorch implementation of neural style transfer algorithm

Last update: Jan 02, 2023

Overview

neural-style-pt

This is a PyTorch implementation of the paper A Neural Algorithm of Artistic Style by Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge. The code is based on Justin Johnson's Neural-Style.

The paper presents an algorithm for combining the content of one image with the style of another image using convolutional neural networks. Here's an example that maps the artistic style of The Starry Night onto a night-time photograph of the Stanford campus:

Applying the style of different images to the same content image gives interesting results. Here we reproduce Figure 2 from the paper, which renders a photograph of the Tubingen in Germany in a variety of styles:

Here are the results of applying the style of various pieces of artwork to this photograph of the golden gate bridge:

Content / Style Tradeoff

The algorithm allows the user to trade-off the relative weight of the style and content reconstruction terms, as shown in this example where we port the style of Picasso's 1907 self-portrait onto Brad Pitt:

Style Scale

By resizing the style image before extracting style features, we can control the types of artistic features that are transfered from the style image; you can control this behavior with the -style_scale flag. Below we see three examples of rendering the Golden Gate Bridge in the style of The Starry Night. From left to right, -style_scale is 2.0, 1.0, and 0.5.

Multiple Style Images

You can use more than one style image to blend multiple artistic styles.

Clockwise from upper left: "The Starry Night" + "The Scream", "The Scream" + "Composition VII", "Seated Nude" + "Composition VII", and "Seated Nude" + "The Starry Night"

Style Interpolation

When using multiple style images, you can control the degree to which they are blended:

Transfer style but not color

If you add the flag -original_colors 1 then the output image will retain the colors of the original image.

Setup:

Dependencies:

PyTorch

Optional dependencies:

For CUDA backend:
- CUDA 7.5 or above
For cuDNN backend:
- cuDNN v6 or above
For ROCm backend:
- ROCm 2.1 or above
For MKL backend:
- MKL 2019 or above
For OpenMP backend:
- OpenMP 5.0 or above

After installing the dependencies, you'll need to run the following script to download the VGG model:

python models/download_models.py

This will download the original VGG-19 model. The original VGG-16 model will also be downloaded. By default the original VGG-19 model is used.

If you have a smaller memory GPU then using NIN Imagenet model will be better and gives slightly worse yet comparable results. You can get the details on the model from BVLC Caffe ModelZoo. The NIN model is downloaded when you run the download_models.py script.

You can find detailed installation instructions for Ubuntu and Windows in the installation guide.

Usage

Basic usage:

python neural_style.py -style_image <image.jpg> -content_image <image.jpg>

cuDNN usage with NIN Model:

python neural_style.py -style_image examples/inputs/picasso_selfport1907.jpg -content_image examples/inputs/brad_pitt.jpg -output_image profile.png -model_file models/nin_imagenet.pth -gpu 0 -backend cudnn -num_iterations 1000 -seed 123 -content_layers relu0,relu3,relu7,relu12 -style_layers relu0,relu3,relu7,relu12 -content_weight 10 -style_weight 500 -image_size 512 -optimizer adam

To use multiple style images, pass a comma-separated list like this:

-style_image starry_night.jpg,the_scream.jpg.

Note that paths to images should not contain the ~ character to represent your home directory; you should instead use a relative path or a full absolute path.

Options:

-image_size: Maximum side length (in pixels) of the generated image. Default is 512.
-style_blend_weights: The weight for blending the style of multiple style images, as a comma-separated list, such as -style_blend_weights 3,7. By default all style images are equally weighted.
-gpu: Zero-indexed ID of the GPU to use; for CPU mode set -gpu to c.

Optimization options:

-content_weight: How much to weight the content reconstruction term. Default is 5e0.
-style_weight: How much to weight the style reconstruction term. Default is 1e2.
-tv_weight: Weight of total-variation (TV) regularization; this helps to smooth the image. Default is 1e-3. Set to 0 to disable TV regularization.
-num_iterations: Default is 1000.
-init: Method for generating the generated image; one of random or image. Default is random which uses a noise initialization as in the paper; image initializes with the content image.
-init_image: Replaces the initialization image with a user specified image.
-optimizer: The optimization algorithm to use; either lbfgs or adam; default is lbfgs. L-BFGS tends to give better results, but uses more memory. Switching to ADAM will reduce memory usage; when using ADAM you will probably need to play with other parameters to get good results, especially the style weight, content weight, and learning rate.
-learning_rate: Learning rate to use with the ADAM optimizer. Default is 1e1.
-normalize_gradients: If this flag is present, style and content gradients from each layer will be L1 normalized.

Output options:

-output_image: Name of the output image. Default is out.png.
-print_iter: Print progress every print_iter iterations. Set to 0 to disable printing.
-save_iter: Save the image every save_iter iterations. Set to 0 to disable saving intermediate results.

Layer options:

-content_layers: Comma-separated list of layer names to use for content reconstruction. Default is relu4_2.
-style_layers: Comma-separated list of layer names to use for style reconstruction. Default is relu1_1,relu2_1,relu3_1,relu4_1,relu5_1.

Other options:

-style_scale: Scale at which to extract features from the style image. Default is 1.0.
-original_colors: If you set this to 1, then the output image will keep the colors of the content image.
-model_file: Path to the .pth file for the VGG Caffe model. Default is the original VGG-19 model; you can also try the original VGG-16 model.
-pooling: The type of pooling layers to use; one of max or avg. Default is max. The VGG-19 models uses max pooling layers, but the paper mentions that replacing these layers with average pooling layers can improve the results. I haven't been able to get good results using average pooling, but the option is here.
-seed: An integer value that you can specify for repeatable results. By default this value is random for each run.
-multidevice_strategy: A comma-separated list of layer indices at which to split the network when using multiple devices. See Multi-GPU scaling for more details.
-backend: nn, cudnn, openmp, or mkl. Default is nn. mkl requires Intel's MKL backend.
-cudnn_autotune: When using the cuDNN backend, pass this flag to use the built-in cuDNN autotuner to select the best convolution algorithms for your architecture. This will make the first iteration a bit slower and can take a bit more memory, but may significantly speed up the cuDNN backend.

Frequently Asked Questions

Problem: The program runs out of memory and dies

Solution: Try reducing the image size: -image_size 256 (or lower). Note that different image sizes will likely require non-default values for -style_weight and -content_weight for optimal results. If you are running on a GPU, you can also try running with -backend cudnn to reduce memory usage.

Problem: -backend cudnn is slower than default NN backend

Solution: Add the flag -cudnn_autotune; this will use the built-in cuDNN autotuner to select the best convolution algorithms.

Problem: Get the following error message:

Missing key(s) in state_dict: "classifier.0.bias", "classifier.0.weight", "classifier.3.bias", "classifier.3.weight". Unexpected key(s) in state_dict: "classifier.1.weight", "classifier.1.bias", "classifier.4.weight", "classifier.4.bias".

Solution: Due to a mix up with layer locations, older models require a fix to be compatible with newer versions of PyTorch. The included donwload_models.py script will automatically perform these fixes after downloading the models.

Memory Usage

By default, neural-style-pt uses the nn backend for convolutions and L-BFGS for optimization. These give good results, but can both use a lot of memory. You can reduce memory usage with the following:

Use cuDNN: Add the flag -backend cudnn to use the cuDNN backend. This will only work in GPU mode.
Use ADAM: Add the flag -optimizer adam to use ADAM instead of L-BFGS. This should significantly reduce memory usage, but may require tuning of other parameters for good results; in particular you should play with the learning rate, content weight, and style weight. This should work in both CPU and GPU modes.
Reduce image size: If the above tricks are not enough, you can reduce the size of the generated image; pass the flag -image_size 256 to generate an image at half the default size.

With the default settings, neural-style-pt uses about 3.7 GB of GPU memory on my system; switching to ADAM and cuDNN reduces the GPU memory footprint to about 1GB.

Speed

Speed can vary a lot depending on the backend and the optimizer. Here are some times for running 500 iterations with -image_size=512 on a Tesla K80 with different settings:

-backend nn -optimizer lbfgs: 117 seconds
-backend nn -optimizer adam: 100 seconds
-backend cudnn -optimizer lbfgs: 124 seconds
-backend cudnn -optimizer adam: 107 seconds
-backend cudnn -cudnn_autotune -optimizer lbfgs: 109 seconds
-backend cudnn -cudnn_autotune -optimizer adam: 91 seconds

Here are the same benchmarks on a GTX 1080:

-backend nn -optimizer lbfgs: 56 seconds
-backend nn -optimizer adam: 38 seconds
-backend cudnn -optimizer lbfgs: 40 seconds
-backend cudnn -optimizer adam: 40 seconds
-backend cudnn -cudnn_autotune -optimizer lbfgs: 23 seconds
-backend cudnn -cudnn_autotune -optimizer adam: 24 seconds

Multi-GPU scaling

You can use multiple CPU and GPU devices to process images at higher resolutions; different layers of the network will be computed on different devices. You can control which GPU and CPU devices are used with the -gpu flag, and you can control how to split layers across devices using the -multidevice_strategy flag.

For example in a server with four GPUs, you can give the flag -gpu 0,1,2,3 to process on GPUs 0, 1, 2, and 3 in that order; by also giving the flag -multidevice_strategy 3,6,12 you indicate that the first two layers should be computed on GPU 0, layers 3 to 5 should be computed on GPU 1, layers 6 to 11 should be computed on GPU 2, and the remaining layers should be computed on GPU 3. You will need to tune the -multidevice_strategy for your setup in order to achieve maximal resolution.

We can achieve very high quality results at high resolution by combining multi-GPU processing with multiscale generation as described in the paper Controlling Perceptual Factors in Neural Style Transfer by Leon A. Gatys, Alexander S. Ecker, Matthias Bethge, Aaron Hertzmann and Eli Shechtman.

Here is a 4016 x 2213 image generated on a server with eight Tesla K80 GPUs:

The script used to generate this image can be found here.

Implementation details

Images are initialized with white noise and optimized using L-BFGS.

We perform style reconstructions using the conv1_1, conv2_1, conv3_1, conv4_1, and conv5_1 layers and content reconstructions using the conv4_2 layer. As in the paper, the five style reconstruction losses have equal weights.

Citation

If you find this code useful for your research, please cite:

@misc{ProGamerGov2018,
  author = {ProGamerGov},
  title = {neural-style-pt},
  year = {2018},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/ProGamerGov/neural-style-pt}},
}

Comments

Cannot use conv layers in vvg-19

Hello, I'm using something like:

python neural_style.py -style_image myStyle.png -content_image myImage.jpg -output_image profile.png -gpu 0 -backend cudnn -num_iterations 5000 -image_size 1000 -style_weight 600 -style_scale 1.2 -style_layers conv1_1

or any convX_Y layer for style or content and I get the following error:

Running optimization with L-BFGS Traceback (most recent call last): File "neural_style.py", line 455, in main() File "neural_style.py", line 257, in main optimizer.step(feval) File "C:\Users[username]\neural-style-pt-master\lib\site-packages\torch\optim\lbfgs.py", line 307, in step orig_loss = closure() File "neural_style.py", line 248, in feval loss.backward() File "C:\Users[username]\neural-style-pt-master\lib\site-packages\torch\tensor.py", line 150, in backward torch.autograd.backward(self, gradient, retain_graph, create_graph) File "C:\Users[username]\neural-style-pt-master\lib\site-packages\torch\autograd_init_.py", line 99, in backward allow_unreachable=True) # allow_unreachable flag RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [64, 791000]], which is output 0 of ViewBackward, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

When I use the ones in your example code (-content_layers relu0,relu3,relu7,relu12 -style_layers relu0,relu3,relu7,relu12), It's just a black image outputted, but no errors at least thrown.

When I try the default which uses (-style_layers relu1_1,relu2_1,relu3_1,relu4_1,relu5_1 -content_layers relu4_2) it works fine.

opened by qwerdbeta 21

Problem with channel_pruning and other models

Hello @ProGamerGov

Thanks for this great repo. VGG and NIN models work like a charm, but using Voltax3 from u/vic8760, I encountered problems with channel_pruning and nyud-fcn32s-color-heavy. With channel_pruning, it returns me this log:

NIN Architecture Detected
Traceback (most recent call last):
  File "neural_style.py", line 409, in <module>
    main()
  File "neural_style.py", line 56, in main
    cnn, layerList = loadCaffemodel(params.model_file, params.pooling, params.gpu)  
  File "/home/GitHub/style_transfer/pytorch_style/CaffeLoader.py", line 136, in loadCaffemodel
    cnn.load_state_dict(torch.load(model_file))
  File "/home/anaconda3/envs/style/lib/python3.6/site-packages/torch/nn/modules/module.py", line 845, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for NIN:
	Missing key(s) in state_dict: "features.4.weight", "features.4.bias", "features.9.weight", "features.9.bias", "features.11.weight", "features.11.bias", "features.16.weight", "features.16.bias", "features.18.weight", "features.18.bias", "features.22.weight", "features.22.bias". 
	Unexpected key(s) in state_dict: "classifier.0.weight", "classifier.0.bias", "classifier.3.weight", "classifier.3.bias", "features.5.weight", "features.5.bias", "features.10.weight", "features.10.bias", "features.12.weight", "features.12.bias", "features.17.weight", "features.17.bias", "features.19.weight", "features.19.bias", "features.21.weight", "features.21.bias", "features.28.weight", "features.28.bias". 
	size mismatch for features.0.weight: copying a param with shape torch.Size([24, 3, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 3, 11, 11]).
	size mismatch for features.0.bias: copying a param with shape torch.Size([24]) from checkpoint, the shape in current model is torch.Size([96]).
	size mismatch for features.2.weight: copying a param with shape torch.Size([22, 24, 3, 3]) from checkpoint, the shape in current model is torch.Size([96, 96, 1, 1]).
	size mismatch for features.2.bias: copying a param with shape torch.Size([22]) from checkpoint, the shape in current model is torch.Size([96]).
	size mismatch for features.7.weight: copying a param with shape torch.Size([51, 41, 3, 3]) from checkpoint, the shape in current model is torch.Size([256, 96, 5, 5]).
	size mismatch for features.7.bias: copying a param with shape torch.Size([51]) from checkpoint, the shape in current model is torch.Size([256]).
	size mismatch for features.14.weight: copying a param with shape torch.Size([111, 89, 3, 3]) from checkpoint, the shape in current model is torch.Size([384, 256, 3, 3]).
	size mismatch for features.14.bias: copying a param with shape torch.Size([111]) from checkpoint, the shape in current model is torch.Size([384]).
	size mismatch for features.24.weight: copying a param with shape torch.Size([512, 228, 3, 3]) from checkpoint, the shape in current model is torch.Size([1024, 1024, 1, 1]).
	size mismatch for features.24.bias: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([1024]).
	size mismatch for features.26.weight: copying a param with shape torch.Size([512, 512, 3, 3]) from checkpoint, the shape in current model is torch.Size([1000, 1024, 1, 1]).
	size mismatch for features.26.bias: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([1000]).

Like model saved hasn't the expected structure. And with nyud-fcn32s-color-heavy, the message isn't clear to me:

Traceback (most recent call last):
  File "neural_style.py", line 409, in <module>
    main()
  File "neural_style.py", line 56, in main
    cnn, layerList = loadCaffemodel(params.model_file, params.pooling, params.gpu)  
  File "/home/GitHub/style_transfer/pytorch_style/CaffeLoader.py", line 135, in loadCaffemodel
    cnn, layerList = modelSelector(str(model_file).lower(), pooling)
  File "/home/GitHub/style_transfer/pytorch_style/CaffeLoader.py", line 119, in modelSelector
    raise ValueError("Model architecture not recognized.")
ValueError: Model architecture not recognized.

In the wiki it's said that these models work. I try different style and content layers names. Maybe I am doing something wrong?

Thanks in advance

opened by JaledMC 10

Memory

I have a 1080 ti with 11 GB of VRAM and I am running into this:

RuntimeError: CUDA out of memory. Tried to allocate 148.00 MiB (GPU 0; 11.00 GiB total capacity; 8.73 GiB already allocated; 44.32 MiB free; 77.81 MiB cached)

I am doing literally nothing else with my computer except styling an image, but it seems that the size I can perform is rather small given my "available" VRAM. I put available in quotes because I know it exists, but I am uncertain about why there is such a low amount that is actually free.

opened by spot92 9
Added support for Multi-GPU and CPU
You can now use multiple GPUs in the same way that you could in the original neural-style.

The -multigpu_strategy parameter was renamed to -multidevice_strategy.

https://github.com/ProGamerGov/neural-style-pt/issues/2

You can use any combination of GPUs and your CPU as devices.

New -disable_check parameter for advanced users.

AMD GPU support.
opened by ProGamerGov 9
Error: `Missing Keys` when loading VGG weights

System Environment OS: Ubuntu 18.04 Python Version: 3.6.7 PyTorch Version: 0.4.1

Attached is a screenshot of the error message when loading VGG weights.

opened by rrmina 9
No images found after running script

The script was working fine for me for months. When I tried to generate some new images it ran as it usually does and went through all the iterations. But the output files are nowhere to be found. I even tried giving a full path in -output. Any ideas??

my script"

python neural_style.py -style_image /mnt/e/GFX\ Textures/Patterns/Photoshop\ Pattern\ Pictures/4.jpg -content_image /media/max/90E83424E8340ACC/Users/Ben\ Stiller/Documents/Photoshop\ Projects/penguins/penguin-29.png -output_image /home/max/Documents/github_downloads/neural-style-pt/penguin-45.png -model_file models/nin_imagenet.pth -gpu 0 -backend cudnn -num_iterations 1000 -seed 123 -content_layers relu0,relu3,relu7,relu12 -style_layers relu0,relu3,relu7,relu12 -content_weight 10 -style_weight 500 -image_size 600 -optimizer adam

"

opened by Mshriver2 7
2nd file not found when using a list of 2 style images

First of all, congratulation for you nice job !

I always have an error "file not found" on the 2nd file of the list, when using a list of 2 style images:

python -u ./neural_style.py -content_image "/c/Users/A455435/w/m/content/IN.jpg" -style_image "/c/Users/A455435/w/m/style/mix/A.jpg,/c/Users/A455435/w/m/style/mix/B.jpg" -style_blend_weights 5,5 -output_image "/c/Users/A455435/w/m/output/IN(Albena_04XX)px100,cw5e0,sw2e2,ss0.8,oc0,it2,i.jpg" -print_iter 1 -save_iter 0 -image_size 100 -num_iterations 2 -content_weight 5e0 -style_weight 2e2 -style_scale 0.8 -original_colors 0 -init image -seed 7 -gpu c VGG-19 Architecture Detected Successfully loaded models/vgg19-d01eb7cb.pth conv1_1: 64 3 3 3 conv1_2: 64 64 3 3 conv2_1: 128 64 3 3 conv2_2: 128 128 3 3 conv3_1: 256 128 3 3 conv3_2: 256 256 3 3 conv3_3: 256 256 3 3 conv3_4: 256 256 3 3 conv4_1: 512 256 3 3 conv4_2: 512 512 3 3 conv4_3: 512 512 3 3 conv4_4: 512 512 3 3 conv5_1: 512 512 3 3 conv5_2: 512 512 3 3 conv5_3: 512 512 3 3 conv5_4: 512 512 3 3 Traceback (most recent call last): File "./neural_style.py", line 468, in main() File "./neural_style.py", line 75, in main img_caffe = preprocess(image, style_size).type(dtype) File "./neural_style.py", line 336, in preprocess image = Image.open(image_name).convert('RGB') File "C:UsersA455435AppDataLocalProgramsPythonPython37libsite-packagesPILImage.py", line 2843, in open fp = builtins.open(filename, "rb") FileNotFoundError: [Errno 2] No such file or directory: '/c/Users/A455435/w/m/style/mix/B.jpg'

While the file exists: $ ls /c/Users/A455435/w/m/style/mix/B.jpg /c/Users/A455435/w/m/style/mix/B.jpg

Any idea of what could be the reason ?

Regards Jerome

opened by jerome8811 5
Sorry dumb question about running the script.

How do I run the scripts through Conda? When I try to run it through Conda, it just keeps trying to open Bash and run it through that.

I edited the starry_stanford.sh script, and am trying to run it, someone help me out please.

opened by Subash-Chandra 5
Missing -normalize_gradient parameter

Hi, I noticed this is based on the jcjohnson code, but his has a param called -normalize_gradient tat is iplemented on lines 130-149 ish on https://github.com/jcjohnson/neural-style/blob/master/neural_style.lua

Would you consider porting his lua code of this feature and perhaps other normalization approaches?

Thanks!

opened by qwerdbeta 5
The num_corrections default value is actually 100, not 0
In neural-style-pt, (and the original neural-style.lua, actually), if params.lbfgs_num_corrections is not set, the default is 0 (line 29). Then, the optim.history object is only updated if params.num_corrections > 0:

#line 56 if params.lbfgs_num_correction > 0: optim_state['history_size'] = params.lbfgs_num_correction

However, the torch.optim.lbfgs class uses a default value of 100 if history_size is not set:

torch.optim.LBFGS(params, lr=1, max_iter=20, max_eval=None, tolerance_grad=1e-05, tolerance_change=1e-09, history_size=100, line_search_fn=None)

See the lbfgs section: https://pytorch.org/docs/stable/optim.html

Thus, if somebody tries to set lbfgs_num_corrections to 0 (the default) to save memory they will actually use size 100.
opened by ajhool 5

Access forbidden to VGG-19 model

Issue

VGG-19 cannot be downloaded using the URL to the S3 bucket.

Code

$ python models/download_models.py

Downloading the VGG-19 model
Downloading: "https://s3-us-west-2.amazonaws.com/jcjohns-models/vgg19-d01eb7cb.pth" to /Users/louis/.cache/torch/checkpoints/vgg19-d01eb7cb.pth
Traceback (most recent call last):
  File "models/download_models.py", line 10, in <module>
    sd = load_url("https://s3-us-west-2.amazonaws.com/jcjohns-models/vgg19-d01eb7cb.pth")
  File ".../env/lib/python3.7/site-packages/torch/hub.py", line 492, in load_state_dict_from_url
    download_url_to_file(url, cached_file, hash_prefix, progress=progress)
  File ".../env/lib/python3.7/site-packages/torch/hub.py", line 391, in download_url_to_file
    u = urlopen(url)
  File ".../env/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File ".../env/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File ".../env/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File ".../env/lib/python3.7/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File ".../env/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File ".../env/lib/python3.7/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

Or using curl:

$ curl https://s3-us-west-2.amazonaws.com/jcjohns-models/vgg19-d01eb7cb.pth

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>6A3FF8D1FF012D4B</RequestId><HostId>bVI6/cnmD0zRB91VGnfWNpATBNLysl/HTErEmZNPqnudNSPrRUC/dKbZ6KANayZ4P8oYYF1vCbo=</HostId></Error>%

resolved

opened by doulouUS 4

HalfTensor (float16) support? tried to implement but failed :(
Is it possible to run this with half precision, to be able to use higher image resolution with limited VRAM? I've tried to do it (similar way to how stable diffusion does):

added "cnn=cnn.half()" after caling loadCaffemodel

replaced all FloatTensor by HalfTensor in neural_style.py It is running, but loss calculation is not working:

Running optimization with L-BFGS Iteration 10 / 1000 Content 1 loss: nan Style 1 loss: nan Style 2 loss: nan Style 3 loss: nan Style 4 loss: nan Style 5 loss: nan Total loss: nan

any idea how to fix that?
opened by arpitest 2
MPS Support
Running neural-style-pt on Apple Silicon seems to require using only CPU or it terminates with this error:

raise AssertionError("Torch not compiled with CUDA enabled")

AssertionError: Torch not compiled with CUDA enabled

Given that PyTorch now supports native Apple Metal acceleration, is there a way to fix this so it'd use MPS?
opened by trbutler 6
Support for additional pre trained models

Hello!

I would like to experiment with different models. Can you please share what steps I would need to take to add support for other models such as these?

opened by Vigilence 1
Multiple Content Image / Video Frame Option

Is there a way to automate inputting multiple content images, that apply the same style image to all of them?

I am rendering each frame separately, of a 500 frame video I wanted to style transfer.

opened by RoronoaZoroSenpai 2
How to reduce the batch size?

I'm wanting to reduce the batch size in order to get around the dreaded "CUDA out of memory" error. I'm wanting to do this to be able to generate much larger images as outputs at least ~10meg. I don't see an Argument for this so thought perhaps this is hard coded?

opened by Bird-NZ 1

Releases(v1.0.0)

v1.0.0(Aug 5, 2022)

Source code(tar.gz)
Source code(zip)

Owner

GitHub Repository

Ensemble Knowledge Guided Sub-network Search and Fine-tuning for Filter Pruning

Ensemble Knowledge Guided Sub-network Search and Fine-tuning for Filter Pruning This repository is official Tensorflow implementation of paper: Ensemb

12 Oct 18, 2022

Bulk2Space is a spatial deconvolution method based on deep learning frameworks

Bulk2Space Spatially resolved single-cell deconvolution of bulk transcriptomes using Bulk2Space Bulk2Space is a spatial deconvolution method based on

60 Dec 27, 2022

A PyTorch implementation of Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks

SVHNClassifier-PyTorch A PyTorch implementation of Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks If

182 Jan 03, 2023

Code for paper "Extract, Denoise and Enforce: Evaluating and Improving Concept Preservation for Text-to-Text Generation" EMNLP 2021

The repo provides the code for paper "Extract, Denoise and Enforce: Evaluating and Improving Concept Preservation for Text-to-Text Generation" EMNLP 2

18 May 24, 2022

Python package for dynamic system estimation of time series

PyDSE Toolset for Dynamic System Estimation for time series inspired by DSE. It is in a beta state and only includes ARMA models right now. Documentat

40 Oct 07, 2022

Using VapourSynth with super resolution models and speeding them up with TensorRT.

VSGAN-tensorrt-docker Using image super resolution models with vapoursynth and speeding them up with TensorRT. Using NVIDIA/Torch-TensorRT combined wi

111 Jan 05, 2023

Code for "Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans" CVPR 2021 best paper candidate

News 05/17/2021 To make the comparison on ZJU-MoCap easier, we save quantitative and qualitative results of other methods at here, including Neural Vo

748 Jan 07, 2023

A PyTorch-based open-source framework that provides methods for improving the weakly annotated data and allows researchers to efficiently develop and compare their own methods.

Knodle (Knowledge-supervised Deep Learning Framework) - a new framework for weak supervision with neural networks. It provides a modularization for se

93 Nov 06, 2022

PyTorch implementation of neural style transfer algorithm

Related tags

Overview

neural-style-pt

Content / Style Tradeoff

Style Scale

Multiple Style Images

Style Interpolation

Transfer style but not color

Setup:

Usage

Frequently Asked Questions

Memory Usage

Speed

Multi-GPU scaling

Implementation details

Citation

Comments

Issue

Code

Releases(v1.0.0)

v1.0.0(Aug 5, 2022)

Owner

Ensemble Knowledge Guided Sub-network Search and Fine-tuning for Filter Pruning

Bulk2Space is a spatial deconvolution method based on deep learning frameworks

A PyTorch implementation of Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks

Code for paper "Extract, Denoise and Enforce: Evaluating and Improving Concept Preservation for Text-to-Text Generation" EMNLP 2021

Python package for dynamic system estimation of time series

Using VapourSynth with super resolution models and speeding them up with TensorRT.

Code for "Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans" CVPR 2021 best paper candidate

Adjusting for Autocorrelated Errors in Neural Networks for Time Series

Official Implementation of "Tracking Grow-Finish Pigs Across Large Pens Using Multiple Cameras"

A real world application of a Recurrent Neural Network on a binary classification of time series data

The official pytorch implementation of our paper "Is Space-Time Attention All You Need for Video Understanding?"

UNION: An Unreferenced Metric for Evaluating Open-ended Story Generation

Python library containing BART query generation and BERT-based Siamese models for neural retrieval.

High-Resolution Image Synthesis with Latent Diffusion Models

A PyTorch implementation of Implicit Q-Learning

Sequential Model-based Algorithm Configuration

Code for Paper: Self-supervised Learning of Motion Capture

Boosted neural network for tabular data

SSL_SLAM2: Lightweight 3-D Localization and Mapping for Solid-State LiDAR (mapping and localization separated) ICRA 2021

A PyTorch-based open-source framework that provides methods for improving the weakly annotated data and allows researchers to efficiently develop and compare their own methods.