Hello,

We just tried upgrading django-pipeline from version 1.3.25 to 1.4.x (multiple minor versions tested, including latest: 1.4.6). Every request rendering static files went from 0.5 sec to about 20 secs to render.

We tried to debug what led to this increase but could not find it. After downgrading to 1.3 the load times were again at 0.5 secs per page.

I could swear this worked in 1.1.27...

When any of my less files specify background: url(''), the url is relative assuming that all compressed CSS will end up in css/*.css, and therefore the image urls being with '../img/'.

However, it seems that pipeline has passed paths off to django staticfiles that are all relative to their original less files, which leads to the image not being found:

Traceback (most recent call last):
...
  File "/Users/steven/virtualenvs/simple-pipeline/lib/python2.7/site-packages/staticfiles/management/commands/collectstatic.py", line 163, in handle_noargs
    collected = self.collect()
  File "/Users/steven/virtualenvs/simple-pipeline/lib/python2.7/site-packages/staticfiles/management/commands/collectstatic.py", line 120, in collect
    for original_path, processed_path, processed in processor:
  File "/Users/steven/virtualenvs/simple-pipeline/lib/python2.7/site-packages/staticfiles/storage.py", line 270, in post_process
    content = pattern.sub(converter, content)
  File "/Users/steven/virtualenvs/simple-pipeline/lib/python2.7/site-packages/staticfiles/storage.py", line 213, in converter
    hashed_url = self.url(unquote(joined_result), force=True)
  File "/Users/steven/virtualenvs/simple-pipeline/lib/python2.7/site-packages/staticfiles/storage.py", line 160, in url
    hashed_name = self.hashed_name(clean_name).replace('\\', '/')
  File "/Users/steven/virtualenvs/simple-pipeline/lib/python2.7/site-packages/staticfiles/storage.py", line 123, in hashed_name
    (clean_name, self))
ValueError: The file 'less/img/wilford-brimley-diabeetus-cat.jpg' could not be found with <simple_pipeline.myapp.pipeline_storage.S3PipelineStorage object at 0x1027f2a90>.

No to go figure out if/where this ever worked...

[1] http://pastebin.com/MGtcj9rA (full stacktrace here)

If I have a file, 'foo.coffee', when I reference that file through the tag, it creates a file next to it called 'foo.js'. The problem with this is it makes it very difficult to have a version control system ignore these files. It is very unlikely that I would want to blanket ignore .css or .js files, so that isn't an option. Instead, I'm left ignoring files individually when I add a new less or coffee file.

It would be nice if pipeline placed the files in their own directory, so I could blanket ignore that directory. As an example, if I have '{static}/foo/bar.coffee', pipeline could place the compiled file in '{static}/{pipelined}/foo/bar.js', where "pipelined" could be a configurable setting.

https://github.com/cyberdelia/django-pipeline/blob/master/pipeline/storage.py#L89 <- bug here

path is string and os.path.dirname(name) is string too.

Searching substring in this case is so confusing - need more complicated logic.

I'm running django-pipeline version 1.1.22 (latest), and the following isn't working yet in .less files:

 @import "bootstrap-1.4/bootstrap.less";

The reason one would do this would be to include the Less library and be able to predefine variables and be able to use the mixins exposed by the library in one's own Less code. For instance:

@basefont: 14px; /* in bootstrap, it's 13 by default, so we override it before it's imported */

@import "bootstrap-1.4/bootstrap.less";

.ghostly { /* we use bootstrap's defined .opacity mixin to create this effect. */
    .opacity(90);
}

Currently, this doesn't work and yields:

Environment:


Request Method: GET
Request URL: http://localhost:8000/

Django Version: 1.3
Python Version: 2.7.1
Installed Applications:
['django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.messages',
 'django.contrib.admin',
 'django.contrib.admindocs',
 'django.contrib.staticfiles',
 'django_bcrypt',
 'pipeline',
 'reversion',
 'south',
 'myproject.web']
Installed Middleware:
('django.middleware.common.CommonMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware')


Template error:
In template /home/rfkrocktk/Desktop/myproject/src/myproject/web/templates/desktop/root.html, error at line 6
   Caught CompilerError while rendering: /var/lib/gems/1.8/gems/less-2.0.7/lib/less/loader.rb:46:in `error': file 'bootstrap-1.4/bootstrap.less' wasn't found. (Less::ParseError)
    from at Function.importer (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/index.js:114:13)
    from at Object.push (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:73:25)
    from at Object.<anonymous> (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/tree/import.js:30:17)
    from at Object.import (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:953:28)
    from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
    from at Object.directive (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:967:29)
    from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
    from at Object.primary (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:428:80)
    from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
    from at Object.parse (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:264:42)
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:65:in `parse'
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:13:in `calljs'
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:27:in `lock'
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:25:in `lock'
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:12:in `calljs'
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:63:in `parse'
    from /var/lib/gems/1.8/gems/less-2.0.7/bin/lessc:22
    from /usr/local/bin/lessc:19:in `load'
    from /usr/local/bin/lessc:19

   1 : {% load static %}


   2 : {% load compressed %}


   3 : <!DOCTYPE html>


   4 : <html>


   5 :  <head>


   6 :      {% block stylesheets %} {% compressed_css 'all' %} {% endblock %}


   7 :      <title>{% block title %}{% endblock %}&nbsp;&raquo;&nbsp;myproject</title>


   8 :  </head>


   9 :  <body>


   10 :         {% block body %}{% endblock %}


   11 :         {% block scripts %}{% compressed_js 'all' %}{% endblock %}


   12 :     </body>


   13 : </html>


   14 : 

Traceback:
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/core/handlers/base.py" in get_response
  111.                         response = callback(request, *callback_args, **callback_kwargs)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/views/generic/simple.py" in direct_to_template
  28.     return HttpResponse(t.render(c), mimetype=mimetype)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
  123.             return self._render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in _render
  117.         return self.nodelist.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
  744.                 bits.append(self.render_node(node, context))
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/debug.py" in render_node
  73.             result = node.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/loader_tags.py" in render
  127.         return compiled_parent._render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in _render
  117.         return self.nodelist.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
  744.                 bits.append(self.render_node(node, context))
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/debug.py" in render_node
  73.             result = node.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/loader_tags.py" in render
  127.         return compiled_parent._render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in _render
  117.         return self.nodelist.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
  744.                 bits.append(self.render_node(node, context))
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/debug.py" in render_node
  73.             result = node.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/loader_tags.py" in render
  64.             result = block.nodelist.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
  744.                 bits.append(self.render_node(node, context))
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/debug.py" in render_node
  73.             result = node.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.22-py2.7.egg/pipeline/templatetags/compressed.py" in render
  30.             package['paths'] = self.packager.compile(package['paths'])
File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.22-py2.7.egg/pipeline/packager.py" in compile
  50.         return self.compiler.compile(paths)
File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.22-py2.7.egg/pipeline/compilers/__init__.py" in compile
  25.                         compiled_content = compiler.compile_file(content, storage.path(path))
File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.22-py2.7.egg/pipeline/compilers/less/__init__.py" in compile_file
  17.         content = self.execute_command(command)
File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.22-py2.7.egg/pipeline/compilers/__init__.py" in execute_command
  88.             raise CompilerError(error)

Exception Type: TemplateSyntaxError at /
Exception Value: Caught CompilerError while rendering: /var/lib/gems/1.8/gems/less-2.0.7/lib/less/loader.rb:46:in `error': file 'bootstrap-1.4/bootstrap.less' wasn't found. (Less::ParseError)
    from at Function.importer (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/index.js:114:13)
    from at Object.push (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:73:25)
    from at Object.<anonymous> (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/tree/import.js:30:17)
    from at Object.import (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:953:28)
    from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
    from at Object.directive (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:967:29)
    from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
    from at Object.primary (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:428:80)
    from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
    from at Object.parse (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:264:42)
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:65:in `parse'
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:13:in `calljs'
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:27:in `lock'
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:25:in `lock'
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:12:in `calljs'
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:63:in `parse'
    from /var/lib/gems/1.8/gems/less-2.0.7/bin/lessc:22
    from /usr/local/bin/lessc:19:in `load'
    from /usr/local/bin/lessc:19

With the below situation, our 'base' bundle is still being compressed and written to a file called 'css/base.css' (where it should be 'css/thatguyismystyle.base.min.css' )

The template tag still writes out the desired filename (which is 'css/thatguyismystyle.base.min.css' )

Django==1.3
django-pipeline==1.2.5
django-staticfiles==1.2.1

    from os.path import join
    from settings import SERVER_ROOT
    from collections import OrderedDict as OD

    NODE_BIN = join(SERVER_ROOT, 'node_modules/.bin')

    #
    # Pipelined CSS Files
    #


    PIPELINE_CSS = OD((
        ('base', {
            'source_filenames': (
                'css/base.less',
            ),

            'output_filename':
                'css/thatguyismystyle.base.min.css',

            'extra_context': {
                'media': 'screen',
            },
        }),
        ('theme', {
            'source_filenames': (
                'css/theme/base.less',
            ),
            'output_filename':
                'css/theme/thatguyismystyle.theme.min.css',

            'extra_context': {
                'media': 'screen',
            },
        }),
        ('theme-light', {
            'source_filenames': (
                'themes/light/css/base.less'
            ),
            'output_filename':
                'themes/thatguyismystyle.colour.light.min.css',
            'extra_context': {
                "media": "screen",
            }
        }),
        ('theme-dark', {
            'source_filenames': (
                'themes/dark/css/base.less'
            ),
            'output_filename':
                'themes/thatguyismystyle.colour.dark.min.css',
            'extra_context': {
                "media": "screen",
            }
        }),
    ))

    #
    # Pipelined JS FILES
    #

    BASE_JS_FRAMEWORK = (
        "js/lib/modernizr.min.js",
        "js/lib/jquery.min.1.7.1.js",
        "js/lib/history.js/jquery.history.js",
        "js/lib/moment.min.pretty.js",
        "js/lib/moment.strftime.min.js",
    )
    BOOTSTRAP_JS = (
        'js/lib/bootstrap/bootstrap-alert.js',
        'js/lib/bootstrap/bootstrap-button.js',
        'js/lib/bootstrap/bootstrap-carousel.js',
        'js/lib/bootstrap/bootstrap-collapse.js',
        'js/lib/bootstrap/bootstrap-dropdown.js',
        'js/lib/bootstrap/bootstrap-modal.js',
        'js/lib/bootstrap/bootstrap-popover.js',
        'js/lib/bootstrap/bootstrap-scrollspy.js',
        'js/lib/bootstrap/bootstrap-tab.js',
        'js/lib/bootstrap/bootstrap-tooltip.js',
        'js/lib/bootstrap/bootstrap-transition.js',
        'js/lib/bootstrap-contrib/*.js',
    )
    CKEDITOR_JS = (
        'js/lib/ckeditor/*.js',
    )
    TINYMCS_JS = (
        'js/lib/tinymce/tiny_mce.js',
        'js/lib/tinymce/langs/*.js',
        'js/lib/tinymce/jquery.tinymce.js',
    )


    PIPELINE_JS = OD((
        ('backend', {
            'source_filenames': (
                'js/lib/jquery.js',
            ) + BOOTSTRAP_JS
              + CKEDITOR_JS + (
                'js/lib/jquery.*.js',
                'js/lib/thatguyismystyle.*.js',
                'js/application.js',
            ),
            'output_filename': 'js/.js',
        }),
        ('profile-pages', {
            'source_filenames': (
            ),
            'output_filename': 'js/profile.js'
        })
    ))
    PIPELINE_JS = dict()

    PIPELINE_DISABLE_WRAPPER = True

    #
    # Compilers
    #

    PIPELINE_STORAGE = 'staticfiles.storage.StaticFilesStorage'
    STATICFILES_STORAGE = 'staticfiles.storage.StaticFilesStorage'
    PIPELINE=True

    PIPELINE_COMPILERS = (
        'pipeline.compilers.less.LessCompiler',
        'pipeline.compilers.coffee.CoffeeScriptCompiler',
    )
    PIPELINE_LESS_BINARY = join( NODE_BIN, 'lessc' )

    #
    # Compressors
    #
    PIPELINE_JS_COMPRESSOR = 'pipeline.compressors.jsmin.JSMinCompressor'
    PIPELINE_CSS_COMPRESSOR = 'pipeline.compressors.cssmin.CssminCompressor'


    STATICFILES_FINDERS = (
     'staticfiles.finders.AppDirectoriesFinder',
    )

    STATICFILES_IGNORE_PATTERNS = (
        "*.txt",
        "tests",
        "css/*.old",
        "/opt/webfiles/common/*.txt",
        "/opt/webfiles/common/temp",
    )

In large projects that use dozens of third-party libraries, invariably, some library will reference a url(../../..somefile) which doesn't exist. When running manage.py collectstatic with django-pipeline, this will result in an incredibly unhelpful error like:

ValueError: The file 'tinymce/js/tinymce/plugins/compat3x/img/icons.gif' could not be found with <pipeline.storage.PipelineCachedStorage object at 0x1ca6750>.

Not only does pipeline not mention which file attempts to include this file, searching for img/icons.gif myself results in 42 matches. Even if I could locate the issue, I can't easily fix the missing file, since it's a third-party library. This means pipeline is effectively blocking all my deployments because of a missing file I don't even use.

Ideally, pipeline should report which resource is referencing a missing file and allow specifying an option to ignore it. This is very similar to the https://github.com/adamcharnock/django-pipeline-forgiving fork, although I think this would be a good idea for the main branch.

I'm using the following static storage:

class S3PipelineStorage(PipelineMixin, S3BotoStorage):
    """S3 storage with pipeline magic"""
    pass

Which is almost the same as the one from pipeline docs (even simpler because it doesnt use CachedFilesMixin) . It's fine with 1.3.x but breaks with 1.4.2 because it tries to run sass compiler using absolute path from s3. S3BotoStorage doesn't support absolute paths and throws this long exception:

/home/bender/myapps/foo/manage.py in <module>()
      8     from django.core.management import execute_from_command_line
      9 
---> 10     execute_from_command_line(sys.argv)

/home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/core/management/__init__.pyc in execute_from_command_line(argv)
    383     """
    384     utility = ManagementUtility(argv)
--> 385     utility.execute()

/home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/core/management/__init__.pyc in execute(self)
    375             sys.stdout.write(self.main_help_text() + '\n')
    376         else:
--> 377             self.fetch_command(subcommand).run_from_argv(self.argv)
    378 
    379 

/home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/core/management/base.pyc in run_from_argv(self, argv)
    286         handle_default_options(options)
    287         try:
--> 288             self.execute(*args, **options.__dict__)
    289         except Exception as e:
    290             if options.traceback or not isinstance(e, CommandError):

/home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/core/management/base.pyc in execute(self, *args, **options)
    336                     not options.get('skip_checks')):
    337                 self.check()
--> 338             output = self.handle(*args, **options)
    339             if output:
    340                 if self.output_transaction:

/home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/core/management/base.pyc in handle(self, *args, **options)
    531         if args:
    532             raise CommandError("Command doesn't accept any arguments")
--> 533         return self.handle_noargs(**options)
    534 
    535     def handle_noargs(self, **options):

/home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.pyc in handle_noargs(self, **options)
    166             raise CommandError("Collecting static files cancelled.")
    167 
--> 168         collected = self.collect()
    169         modified_count = len(collected['modified'])
    170         unmodified_count = len(collected['unmodified'])

/home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.pyc in collect(self)
    112             processor = self.storage.post_process(found_files,
    113                                                   dry_run=self.dry_run)
--> 114             for original_path, processed_path, processed in processor:
    115                 if isinstance(processed, Exception):
    116                     self.stderr.write("Post-processing '%s' failed!" % original_path)
/home/bender/.virtualenvs/foo/lib/python2.7/site-packages/pipeline/storage.pyc in post_process(self, paths, dry_run, **options)
     28         from pipeline.packager import Packager
     29         packager = Packager(storage=self)
---> 30         for package_name in packager.packages['css']:
     31             package = packager.package_for('css', package_name)
     32             output_file = package.output_filename

/home/bender/.virtualenvs/foo/lib/python2.7/site-packages/pipeline/packager.pyc in pack_stylesheets(self, package, **kwargs)
     94         return self.pack(package, self.compressor.compress_css, css_compressed,
     95                          output_filename=package.output_filename,
---> 96                          variant=package.variant, **kwargs)
     97 
     98     def compile(self, paths, force=False):

/home/bender/.virtualenvs/foo/lib/python2.7/site-packages/pipeline/packager.pyc in pack(self, package, compress, signal, **kwargs)
    103         if self.verbose:
    104             print("Saving: %s" % output_filename)
--> 105         paths = self.compile(package.paths, force=True)
    106         content = compress(paths, **kwargs)
    107         self.save_file(output_filename, content)

/home/bender/.virtualenvs/foo/lib/python2.7/site-packages/pipeline/packager.pyc in compile(self, paths, force)
     97 
     98     def compile(self, paths, force=False):
---> 99         return self.compiler.compile(paths, force=force)
    100 
    101     def pack(self, package, compress, signal, **kwargs):

/home/bender/.virtualenvs/foo/lib/python2.7/site-packages/pipeline/compilers/__init__.pyc in compile(self, paths, force)
     54         else:
     55             with futures.ThreadPoolExecutor(max_workers=multiprocessing.cpu_count()) as executor:
---> 56                 return list(executor.map(_compile, paths))
     57 
     58     def output_path(self, path, extension):

/home/bender/.virtualenvs/foo/lib/python2.7/site-packages/concurrent/futures/_base.pyc in map(self, fn, *iterables, **kwargs)
    578             for future in fs:
    579                 if timeout is None:
--> 580                     yield future.result()
    581                 else:
    582                     yield future.result(end_time - time.time())

/home/bender/.virtualenvs/foo/lib/python2.7/site-packages/concurrent/futures/_base.pyc in result(self, timeout)
    398                 raise CancelledError()
    399             elif self._state == FINISHED:
--> 400                 return self.__get_result()
    401 
    402             self._condition.wait(timeout)

/home/bender/.virtualenvs/foo/lib/python2.7/site-packages/concurrent/futures/_base.pyc in __get_result(self)
    357     def __get_result(self):
    358         if self._exception:
--> 359             reraise(self._exception, self._traceback)
    360         else:
    361             return self._result

/home/bender/.virtualenvs/foo/lib/python2.7/site-packages/concurrent/futures/_compat.pyc in reraise(exc, traceback)
    105     def reraise(exc, traceback):
    106         locals_ = {'exc_type': type(exc), 'exc_value': exc, 'traceback': traceback}
--> 107         exec('raise exc_type, exc_value, traceback', {}, locals_)
/home/bender/.virtualenvs/foo/lib/python2.7/site-packages/concurrent/futures/thread.pyc in run(self)
     59 
     60         try:
---> 61             result = self.fn(*self.args, **self.kwargs)
     62         except BaseException:
     63             e, tb = sys.exc_info()[1:]

/home/bender/.virtualenvs/foo/lib/python2.7/site-packages/pipeline/compilers/__init__.pyc in _compile(input_path)
     34                 if compiler.match_file(input_path):
     35                     output_path = self.output_path(input_path, compiler.output_extension)
---> 36                     infile = self.storage.path(input_path)
     37                     outfile = self.output_path(infile, compiler.output_extension)
     38                     outdated = compiler.is_outdated(input_path, output_path)

/home/bender/.virtualenvs/foo/lib/python2.7/site-packages/django/core/files/storage.pyc in path(self, name)
     85         accessed using open() should *not* implement this method.
     86         """
---> 87         raise NotImplementedError("This backend doesn't support absolute paths.")
     88 
     89     # The following methods form the public API for storage systems, but with

NotImplementedError: This backend doesn't support absolute paths

I occasionally (once every 10 runs or so?) get this error when django-pipeline is compressing my JS:

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/django/core/management/base.py", line 415, in handle
    return self.handle_noargs(**options)
  File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 173, in handle_noargs
    collected = self.collect()
  File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 118, in collect
    for original_path, processed_path, processed in processor:
  File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/pipeline/storage.py", line 39, in post_process
    packager.pack_javascripts(package)
  File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/pipeline/packager.py", line 110, in pack_javascripts
    return self.pack(package, self.compressor.compress_js, js_compressed, templates=package.templates, **kwargs)
  File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/pipeline/packager.py", line 104, in pack
    content = compress(paths, **kwargs)
  File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/pipeline/compressors/__init__.py", line 65, in compress_js
    js = getattr(compressor(verbose=self.verbose), 'compress_js')(js)
  File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/pipeline/compressors/yuglify.py", line 13, in compress_js
    return self.compress_common(js, 'js', settings.PIPELINE_YUGLIFY_JS_ARGUMENTS)
  File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/pipeline/compressors/yuglify.py", line 10, in compress_common
    return self.execute_command(command, content)
  File "/home/vagrant/.virtualenvs/VBEZ/local/lib/python2.7/site-packages/pipeline/compressors/__init__.py", line 237, in execute_command
    raise CompressorError(stderr)
pipeline.exceptions.CompressorError: Killed

The only way I have found to continue is to restart the vagrant box that I am developing on.

I'm having the following problem. Not sure if it's a misconfiguration on my end or a bug.

I'm using Bower to install some components. I've added to STATICFILES_DIRS the directory where the components are installed.

Here are some relevant parts from settings.py

STATICFILES_FINDERS = (
    'pipeline.finders.AppDirectoriesFinder',
    'pipeline.finders.FileSystemFinder',
    'pipeline.finders.PipelineFinder',
    'pipeline.finders.CachedFileFinder',
)
STATICFILES_STORAGE = 'pipeline.storage.PipelineCachedStorage'
DEBUG = False

When I run collectstatic, I get the following error:

AttributeError: 'CachedFileFinder' object has no attribute 'storages'

However, If I remove CachedFileFinder, collectstatic works fine.

I've begun implementing Pipeline in our app. I've added the middleware, items to the requirements.txt file, the compressor variable, and the PIPELINE_JS object.

PIPELINE_JS = {
    'activity': {
        'source_filenames': (
            'assets/js/libs/src/core.libs.js',
            'assets/js/libs/src/datepicker.js',
        ),
        'output_filename': 'assets/js/libs/activity.js',
    }
}

I've included the template tag on the page, and output the tag call like so:

{% compressed_js 'activity' %}

The page loads without error, but nothing appears to be happening. I wanted to make sure something should be happening at this point. I don't see any outputted files, and viewing the rendered source of the page shows an empty line where the call to the template was.

So what should I expect is happening?

Also, as a side note, it would be AWESOME if you could include a super simple app using pipeline as a demo. Something that users could download and run locally to see what happens.

Looking forward to your response.

Only a small change was required in the tests to handle this change about removing the type="text/css" attributes for CSS link tags:

It might make sense when releasing a new version to drop support for Python 3.6 and Django 2.2, 3.0 and 3.1 since they have all reached end of life. If you want I can push that in this PR or seperately

collectstatic static fails with the project_infile variable in the compile_file method because the finder looks for static files in the static directories not from static_root.

We had to upgrade django-pipeline from 1.6.14 to at least 2.0.7 since we are currently in the process of upgrading django to version 3.2. After we made the update, our static files were not behaving the same anymore (css not loading). After a few days of investigating I saw that the infile parameter that is passed into the compile_file function in the LessCompiler class were not the same as the value we usually got before we made the update.

Seems like the infile value comes from compile function on the Compiler class in the init file.

    def compile(self, paths, compiler_options={}, force=False):
        def _compile(input_path):
            for compiler in self.compilers:
                compiler = compiler(verbose=self.verbose, storage=self.storage)
                if compiler.match_file(input_path):
                    try:
                        infile = self.storage.path(input_path)
                    except NotImplementedError:
                        infile = finders.find(input_path)
                    project_infile = finders.find(input_path)
                    outfile = compiler.output_path(infile, compiler.output_extension)
                    outdated = compiler.is_outdated(project_infile, outfile)
                    compiler.compile_file(project_infile, outfile,
                                          outdated=outdated, force=force,
                                          **compiler_options)

                    return compiler.output_path(input_path, compiler.output_extension)
            else:
                return input_path

I see the infile value is never passed into the compile_file function. No matter what happens in the try except, another variable called project_infile is created underneath and passed to compile_file, which in our case has the wrong value. Infile has the correct value. On the older version that we used the project_infile didnt exist and the infile value was passed to compile_file. Thank you in advance.

This is a feature request to add self.request to context in render_js on JavascriptNode.

This would be handy for cases where request context variables need to be rendered in the js template, for example, with django-csp you would be able to add a nonce in a custom template without having to extend JavascriptNode and writing a new tag to use it.

My suggestion is to add a key to extra_context which you can declare in settings with a boolean value that defaults to false. If true it adds self.request to context which will then be usable in the template.

If it's something anybody else is interested in, I could create a pull request for it.

Hi there,

I am a bit new to using js. I am having a strange output, where the functions inside a js file are being skipped over during compression using yuglify. I am getting no errors, except when checking the output the functions are not present!

The file which is being ignored

// application.js
function changeValueProp() {
    if (document.getElementById('ShowHideProp').value == 'Show Properties'){
      document.getElementById('ShowHideProp').value = 'Hide Properties';
    }
    else{
      document.getElementById('ShowHideProp').value = 'Show Properties';
    }
  }

In my pipeline config I am using

{
    "JAVASCRIPT": {
     "application": {"source_filenames":  (f'{APP_STATIC_DIR}/application.js',f'{APP_STATIC_DIR}/application01.js',),
                             "output_filename":    'compressed/js/application.js'},
                          },
    "YUGLIFY_BINARY": f"{BASE}/node_modules/yuglify/bin/yuglify",
}

I just checked that if I use "DISABLE_WRAPPER":True, the function from application.js is compressed and put in the output file. In ReadTheDocs, the DISABLE_WRAPPER:False is recommended for improving performance.

Is it possible to use the wrapper without it skipping over the js described above ?

BR

SK