I am using WeasyPrint with Django and Celery and seemingly random errors occur. I have a Django model which has a file field that is a pdf and a celery task which generates these pdfs using WeasyPrint.
The first error occurs on our web server and occurs on seemingly random requests and we receive the traceback below:
Traceback (most recent call last):
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 103, in get_response
resolver_match = resolver.resolve(request.path_info)
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 319, in resolve
for pattern in self.url_patterns:
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 347, in url_patterns
patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 342, in urlconf_module
self._urlconf_module = import_module(self.urlconf_name)
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "/sites/myApp/urls.py", line 16, in <module>
admin.autodiscover()
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/django/contrib/admin/__init__.py", line 29, in autodiscover
import_module('%s.admin' % app)
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/djcelery/admin.py", line 274, in <module>
class PeriodicTaskAdmin(admin.ModelAdmin):
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/djcelery/admin.py", line 276, in PeriodicTaskAdmin
form = periodic_task_form()
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/djcelery/admin.py", line 246, in periodic_task_form
current_app.loader.import_default_modules()
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/djcelery/loaders.py", line 138, in import_default_modules
self.autodiscover()
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/djcelery/loaders.py", line 141, in autodiscover
self.task_modules.update(mod.__name__ for mod in autodiscover() or ())
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/djcelery/loaders.py", line 176, in autodiscover
for app in settings.INSTALLED_APPS])
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/djcelery/loaders.py", line 195, in find_related_module
return importlib.import_module('%s.%s' % (app, related_name))
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/sites/myApp/apps/pdfs/tasks.py", line 6, in <module>
from apps.pdfs.models import PdfModel
File "/sites/myApp/apps/pdfs/models.py", line 15, in <module>
from apps.pdfs.utils import pdf_utils
File "/sites/myApp/apps/pdfs/utils/pdf_utils.py", line 3, in <module>
from weasyprint import HTML, CSS
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/weasyprint/__init__.py", line 304, in <module>
from .css import PARSER, preprocess_stylesheet
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/weasyprint/css/__init__.py", line 29, in <module>
from . import properties
ImportError: cannot import name properties
This exact error occurs with a relatively high frequency, although nowhere near all requests and it starts at seemingly random times. Restarting the uwsgi process which runs the server "solves" the problem.
This issue is also "solved" when I remove djcelery, since djcelery no longer tries to autoload all my tasks modules and WeasyPrint is no longer imported down the line at this point in time, only when it is required.
I originally thought this error would be related to issue #10 but I am not sure because none of the fixes in that issue seemed to work and I am also able to import and use WeasyPrint generally, it just seems to fail in this one specific case.
We also get an error on the machines which run celery workers. We have a Django management command which runs every 3 minutes, and about once a day, we receive the following traceback:
Traceback (most recent call last):
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/django/core/management/base.py", line 222, in run_from_argv
self.execute(*args, **options.__dict__)
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/django/core/management/base.py", line 254, in execute
self.validate()
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/django/core/management/base.py", line 280, in validate
num_errors = get_validation_errors(s, app)
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/django/core/management/validation.py", line 35, in get_validation_errors
for (app_name, error) in get_app_errors().items():
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/django/db/models/loading.py", line 166, in get_app_errors
self._populate()
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/django/db/models/loading.py", line 72, in _populate
self.load_app(app_name, True)
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/django/db/models/loading.py", line 96, in load_app
models = import_module('.models', app_name)
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "/sites/myApp/apps/pdfs/models.py", line 15, in <module>
from apps.pdfs.utils import pdf_utils
File "/sites/myApp/apps/pdfs/utils/pdf_utils.py", line 3, in <module>
from weasyprint import HTML, CSS
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/weasyprint/__init__.py", line 304, in <module>
from .css import PARSER, preprocess_stylesheet
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/weasyprint/css/__init__.py", line 30, in <module>
from . import computed_values
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/weasyprint/css/computed_values.py", line 18, in <module>
from .. import text
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/weasyprint/text.py", line 195, in <module>
gobject = dlopen(ffi, 'gobject-2.0', 'libgobject-2.0-0')
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/weasyprint/text.py", line 192, in dlopen
return ffi.dlopen(names[0]) # pragma: no cover
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/cffi/api.py", line 112, in dlopen
lib, function_cache = _make_ffi_library(self, name, flags)
File "/home/ubuntu/.virtualenvs/myApp/local/lib/python2.7/site-packages/cffi/api.py", line 369, in _make_ffi_library
path = ctypes.util.find_library(name)
File "/usr/lib/python2.7/ctypes/util.py", line 224, in find_library
return _findSoname_ldconfig(name) or _get_soname(_findLib_gcc(name))
File "/usr/lib/python2.7/ctypes/util.py", line 213, in _findSoname_ldconfig
f = os.popen('/sbin/ldconfig -p 2>/dev/null')
OSError: [Errno 12] Cannot allocate memory
I have attempted to check the obvious issue of is the server out of memory which doesn't seem to be the case, and it also seemed odd that the error is the exact same every time. The management command also runs successfully 3 minutes before and 3 minutes after. Also, this occurs across 3 different machines, at the same time.
Both of these errors seem to occur when Django or Celery dynamically import something which eventually imports from WeasyPrint, and both seem to happen randomly as I have been unable to ever reproduce these errors myself.
crash
Of course we will be happy provide Weasyprint contact an every generated PDF (at last few million pages).
76 Dec 12, 2022
1 Nov 09, 2021
27 Jul 13, 2022
1.6k Jan 03, 2023
5k Jan 04, 2023
4 Nov 28, 2022
2.9k Jan 01, 2023
1 Nov 30, 2021
1 Nov 27, 2021
6 Jul 27, 2022
3 Mar 13, 2022
3 Apr 23, 2022
1.8k Dec 29, 2022
22 Nov 21, 2022
4 Dec 05, 2021
1 Nov 08, 2020
5 Jan 22, 2022
0 Nov 03, 2022
1.2k Jan 04, 2023