Hey @GaretJax!

I see that this package hasn't been updated in a while. Is this still actively maintained? If not, would you be OK with transferring maintainership of this project to someone else (either an individual - I am happy to volunteer for that - or an org like jazzband or executablebooks)?

Hi and thanks for sphinx-autobuild,

I have issues retrieving large-ish JS files through tornado that's used behind the scenes by livereload.

My documentation requires mathjax, which files are downloaded and served locally through sphinx-autobuild. When the documentation tries to retrieve MathMenu.js (which is 32.8 kb), I get the following error:

[E 181214 08:51:53 web:1670] Uncaught exception GET /_static/mathjax/extensions/MathMenu.js?V=2.7.5 (127.0.0.1)
    HTTPServerRequest(protocol='http', host='127.0.0.1:8008', method='GET', uri='/_static/mathjax/extensions/MathMenu.js?V=2.7.5', version='HTTP/1.1', remote_ip='127.0.0.1')
    Traceback (most recent call last):
      File "/home/thomas/.buildout/eggs/tornado-5.1.1-py3.5-linux-x86_64.egg/tornado/web.py", line 1592, in _execute
        result = yield result
      File "/home/thomas/.buildout/eggs/tornado-5.1.1-py3.5-linux-x86_64.egg/tornado/gen.py", line 1133, in run
        value = future.result()
      File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
        raise self._exception
      File "/home/thomas/.buildout/eggs/tornado-5.1.1-py3.5-linux-x86_64.egg/tornado/gen.py", line 326, in wrapper
        yielded = next(result)
      File "/home/thomas/.buildout/eggs/tornado-5.1.1-py3.5-linux-x86_64.egg/tornado/web.py", line 2528, in get
        yield self.flush()
      File "/home/thomas/.buildout/eggs/tornado-5.1.1-py3.5-linux-x86_64.egg/tornado/web.py", line 994, in flush
        start_line, self._headers, chunk, callback=callback)
      File "/home/thomas/.buildout/eggs/tornado-5.1.1-py3.5-linux-x86_64.egg/tornado/http1connection.py", line 412, in write_headers
        data += self._format_chunk(chunk)
      File "/home/thomas/.buildout/eggs/tornado-5.1.1-py3.5-linux-x86_64.egg/tornado/http1connection.py", line 424, in _format_chunk
        "Tried to write more data than Content-Length")
    tornado.httputil.HTTPOutputError: Tried to write more data than Content-Length

I'm opening this issue here because as far as I understand tornado can use a static files handler to deal with this kind of problem, and I think it would be good if we could tell the tornado server that it should use that handler for anything in the sphinx static folder.

I haven't looked too much at how livereload and tornado work how we could achieve it so opening the discussion here. From what I saw some modifications to livereload may be needed to access this functionality - you may know that better than I do!

I've just discovered your package and I seems very nice.

My setup:

• python 3.4
• Sphinx v1.2.3
• sphinx-autobuild v0.5
• vim as editor

As I'm using autodoc, I also want to follow the code directory, so I've added the option -z "../mylib". This is easy.

But the autobuild fires every time I edit a file, e.g. doc.rst, as vim modifies .doc.rst.swp file, but does nothing as the file has not been saved. Then when I save it both the doc.rst .doc.rst.un~ are changed and the autobuilds runs for both files. To avoid this I've added

-i 'source/.??*' -i 'source/*~'

to the command line. If I modify one of the python files in mylib, e.g. mylib/module.py, the situation is even worse. The build fires as I type because of mylib/.module.py.swp and, when I save the file, the build needlessly fires a lot of times, mostly because the files in libvhc/__pycache__ are modified when sphinx runs.

I can avoid it with

 -i '../mylib/__pycache__/*' -i '../mylib/.??*' -i '../mylib/*~'

If I add any subdirectory in mylib I'll need to repeat the above line.

For a big project with subpackages and subdirectories in source this gets extremely hard to maintain.

Would it be possible to define the ignore option like

-i '.??*' -i '*~' -i '__pycache__'

such that any file or directory matching the give names is ignored, without to specify in which directory do they live?

First of all, thanks for a great project.

One thing that would be awesome is if sphinx-autobuild . _build/html could also include checking if code has been updated for my project. Right now the autodoc parts of my documentation is not reflected when the docstrings change in the code. What do you think? Would this be possible to implement?

To reproduce, simply instrument a project with the ReadTheDocs Getting Started guide, then try to use sphinx-autobuild . _build/html.

See the documentation about the default behavior for the doctree build output here.

This causes projects to rebuild unnecessarily after the doctrees cache files are generated.

Also, if you force the cache to refresh the environment (for example, with the -E option), the autobuilder will build continuously.

A workaround for now is to also specify the -d option, setting it to _build/doctrees for example.

• editing rst files with Vim + savevers autbackup feature
• this results in a file's timestamp to be changed multiple times on each save
• sphinx-autobuild is triggered by the first change -> rebuild
• all subsequent file system changes are already queued and get processed -> useless rebuilds
• proof of concept quick and dirty hack in SphinxBuilder.call:
  • remember time of last call to build()
  • don't retrigger if delta below a certain threshold (5 seconds?)
• this cures the problem, but I think this could be more cleanly addressed in the file system watchdog layer

Decided to put a few minutes into updating the project.

• dropped Python 3.6, since it's no longer supported,
• added support for Python 3.9 and 3.10,
• added common virtualenv directory names to .gitignore,
• updated tools URLs and versions in .pre-commit-config.yaml,
• migrated project metadata to PEP-621 spec in pyproject.toml,
• moved .coveragerc data to pyproject.toml (and removed unnecessary entries)

Fixes #58 Fixes #74

Some of my CI/CD pipelines were failing because of this. The problem arises if sphinx-build in the local PATH points to a different python version from the main sys.executable.

When working with multiple Sphinx document repositories at one time (e.g., when working with related documents that have cross-referencing intersphinx links), I often need to have multiple browser windows active with multiple sphinx-autobuild processes running. It became tedious and difficult to manually give them all unique port numbers and to start browser windows. These two changes allow a script that includes lines like those below to automatically open browser windows for every Sphinx document I am working on as I run sphinx-autobuild.

Sorry that I didn't include documentation changes and tests. I just don't have the time (but the changes are very minor, with a good bang-for-the-buck value, so I wanted to at least contribute them for consideration).

Clean out any cached content before starting.

make clean 2>/dev/null

Background a trigger for initial build of all files.

(sleep 1 && touch source/*.rst) &

sphinx-autobuild -q
-p 0
--open-browser
--delay 5
--ignore ".swp"
--ignore ".pdf"
--ignore ".log"
--ignore ".out"
--ignore ".toc"
--ignore ".aux"
--ignore ".idx"
--ignore ".ind"
--ignore ".ilg"
--ignore ".tex"
source
build/html

Extracted from my comment on #31: When I save, it triggers the rebuild as expected, but the built content is not actually changed.

Command:

sphinx-autobuild -E -a -z /Users/rouge8/rj/fathom-assets/fathom_assets -n -b html -d /Users/rouge8/rj/fathom-assets/.tox/docs-dev/tmp/doctrees . _build/html

Output:

+--------- ../fathom_assets/query/managers.py changed ---------------------------
| Running Sphinx v1.3.1
| loading intersphinx inventory from https://docs.python.org/2.7/objects.inv...
| building [mo]: all of 0 po files
| building [html]: all source files
| updating environment: 3 added, 0 changed, 0 removed
| reading sources... [ 33%] api
| reading sources... [ 66%] index
| reading sources... [100%] queries
|
| looking for now-outdated files... none found
| pickling environment... done
| checking consistency... done
| preparing documents... done
| writing output... [ 33%] api
| writing output... [ 66%] index
| writing output... [100%] queries
|
| generating indices... genindex py-modindex
| highlighting module code... [ 25%] fathom_assets.query.managers
| highlighting module code... [ 50%] fathom_assets.query.terms
| highlighting module code... [ 75%] fathom_assets.query.query
| highlighting module code... [100%] fathom_assets.query.nodes
|
| writing additional pages... search
| copying static files... done
| copying extra files... done
| dumping search index in English (code: en) ... done
| dumping object inventory... done
| build succeeded.
+--------------------------------------------------------------------------------


+--------- ../fathom_assets/query/managers.py changed ---------------------------
| Running Sphinx v1.3.1
| loading intersphinx inventory from https://docs.python.org/2.7/objects.inv...
| building [mo]: all of 0 po files
| building [html]: all source files
| updating environment: 3 added, 0 changed, 0 removed
| reading sources... [ 33%] api
| reading sources... [ 66%] index
| reading sources... [100%] queries
|
| looking for now-outdated files... none found
| pickling environment... done
| checking consistency... done
| preparing documents... done
| writing output... [ 33%] api
| writing output... [ 66%] index
| writing output... [100%] queries
|
| generating indices... genindex py-modindex
| highlighting module code... [ 25%] fathom_assets.query.managers
| highlighting module code... [ 50%] fathom_assets.query.terms
| highlighting module code... [ 75%] fathom_assets.query.query
| highlighting module code... [100%] fathom_assets.query.nodes
|
| writing additional pages... search
| copying static files... done
| copying extra files... done
| dumping search index in English (code: en) ... done
| dumping object inventory... done
| build succeeded.
+--------------------------------------------------------------------------------

[I 150904 01:55:34 handlers:91] Reload 1 waiters: None
[I 150904 01:55:34 handlers:131] Browser Connected: http://127.0.0.1:8000/api.html#module-fathom_assets.query.managers

Hi! I'm wrapping my sphinx dev environment inside a Docker container to make it portable and easier to use for other users and I'm interested on adding sphinx-autobuild into it.

Up to now, I was able to run a python -m http.server inside the container and access it from my host machine, no problem.

I've made some tries with sphinx-autobuild and the building part works great but I can't access the web server, even though I've EXPOSEd the port I'm using in my Dockerfile and I'm binding a host port to it, as with the http.server module.

It seems that the server the sphinx-autobuild module is starting isn't getting my hosts requests.

Any idea on how to run it on debug mode, or have some logs to look further into this issue?

Describe the bug

context/problem Launching sphinx-autobuild from tox (in conjunction with tox-conda) results in the server continuously relaunching. Specifying
--no-initial results in the server launching as expected until it detects a change and rebuilds the documentation, then this issue persists again. Also, CTRL+C is not responsive, I have to kill the terminal to quit this issue.

expectation This not to occur.

Reproduce the bug

1. Setup and create an Anaconda environment, or at least have the program installed and on PATH
2. Install tox and tox-conda: pip install tox tox-conda
3. Create a tox.ini file with something similar to the following:

[testenv:autodocs]
description =
    Invoke sphinx-autobuild to hot reload docs changes
deps =
    sphinx>=3.2.1
    furo
    sphinx-copybutton
    sphinx-autobuild
commands =
    # Add --no-initial to see that described behavior
    sphinx-autobuild docs docs/_build/html

4. Launch with tox -e autodocs

List your environment

Python: 3.9.12 Windows 10 Anaconda env, likely not necessary Other tools, see above

Context

In the base directory of a repository, there can be files such as CONTRIBUTING.md that are included in the docs by reference:

\```{include} ../CONTRIBUTING.md
\```

Proposal

It would be nice to be able to watch a specific set of files:

--watch *.md

Otherwise, there is going to be a long list of ignores.

Tasks and updates

No response

Describe the bug

context Hi, thanks for developing this very useful tool.

I'm having a bit of an issue where I've added MyST-NB to a project and this causes an infinite build loop, because notebooks are detected as a change, e.g.:

[sphinx-autobuild] Detected change: /home/pimienta/Documents/repos/coding/vocal/crowsetta/doc/_build/jupyter_execute/index.ipynb

expectation

To try and fix this, I use the --ignore option, like so:

sphinx-autobuild --ignore doc/_build/jupyter_execute/*.ipynb doc doc/_build/html

bug But instead I now get:

__main__.py: error: cannot find files ['doc/_build/html']
Command exited with exit code: 2
The server will continue serving the build folder, but the contents being served are no longer in sync with the documentation sources. Please fix the cause of the error above or press Ctrl+C to stop the server.

My interpretation of this error is that somehow the glob I'm writing for --ignore is causing sphinx-autobuild to not see the doc/_build/html folder? It definitely still exists.

$ ls doc/_build/html
api             _downloads  genindex.html  howto-user-format.html  index.html  notebooks    py-modindex.html  scripts      searchindex.js  _static
CHANGELOG.html  formats     howto.html     _images                 _modules    objects.inv  reports           search.html  _sources        tutorial.html

Please let me know if there's some mistake I'm making here

Reproduce the bug

Not sure I can write a MRE for this one.

List your environment

• Python version: Python 3.10
• operating system: PopOS (Ubuntu-like)

$ pip freeze
alabaster==0.7.12
anyio==3.6.1
appdirs==1.4.4
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
asttokens==2.0.5
attrs==21.4.0
audioread==2.1.9
Babel==2.10.1
backcall==0.2.0
beautifulsoup4==4.11.1
birdsong-recognition-dataset==0.3.2
bleach==5.0.0
certifi==2022.5.18.1
cffi==1.15.0
charset-normalizer==2.0.12
click==8.1.3
colorama==0.4.4
commonmark==0.9.1
coverage==6.4
-e git+ssh://[email protected]/vocalpy/[email protected]#egg=crowsetta
cryptography==37.0.2
debugpy==1.6.0
decorator==5.1.1
defusedxml==0.7.1
docutils==0.17.1
entrypoints==0.4
evfuncs==0.3.5
executing==0.8.3
fastjsonschema==2.15.3
flit==3.7.1
flit_core==3.7.1
greenlet==1.1.2
idna==3.3
imagesize==1.3.0
importlib-metadata==4.11.4
importlib-resources==5.7.1
iniconfig==1.1.1
ipykernel==6.13.0
ipython==8.4.0
ipython-genutils==0.2.0
jedi==0.18.1
jeepney==0.8.0
Jinja2==3.1.2
joblib==1.1.0
json5==0.9.8
jsonschema==4.5.1
jupyter-cache==0.5.0
jupyter-client==7.3.1
jupyter-core==4.10.0
jupyter-server==1.17.0
jupyterlab==3.4.2
jupyterlab-pygments==0.2.2
jupyterlab-server==2.14.0
jupytext==1.13.8
keyring==23.5.1
librosa==0.9.1
livereload==2.6.3
llvmlite==0.38.1
markdown-it-py==2.1.0
MarkupSafe==2.1.1
matplotlib-inline==0.1.3
mdit-py-plugins==0.3.0
mdurl==0.1.1
mistune==0.8.4
mypy-extensions==0.4.3
myst-nb==0.15.0
myst-parser==0.17.2
nbclassic==0.3.7
nbclient==0.5.13
nbconvert==6.5.0
nbformat==5.4.0
nest-asyncio==1.5.5
notebook==6.4.11
notebook-shim==0.1.0
numba==0.55.2
numpy==1.22.4
packaging==21.3
pandas==1.4.2
pandera==0.11.0
pandocfilters==1.5.0
parso==0.8.3
pexpect==4.8.0
pickleshare==0.7.5
pkginfo==1.8.2
pluggy==1.0.0
pooch==1.6.0
prometheus-client==0.14.1
prompt-toolkit==3.0.29
psutil==5.9.1
ptyprocess==0.7.0
pure-eval==0.2.2
py==1.11.0
pyarrow==8.0.0
pycparser==2.21
pydantic==1.9.1
pydata-sphinx-theme==0.8.1
Pygments==2.12.0
pyparsing==3.0.9
pyprojroot==0.2.0
pyrsistent==0.18.1
pytest==7.1.2
pytest-cov==3.0.0
python-dateutil==2.8.2
pytz==2022.1
PyYAML==6.0
pyzmq==23.0.0
readme-renderer==35.0
requests==2.27.1
requests-toolbelt==0.9.1
resampy==0.2.2
rfc3986==2.0.0
rich==12.4.4
scikit-learn==1.1.1
scipy==1.8.1
SecretStorage==3.3.2
Send2Trash==1.8.0
six==1.16.0
sniffio==1.2.0
snowballstemmer==2.2.0
SoundFile==0.10.3.post1
soupsieve==2.3.2.post1
Sphinx==4.5.0
sphinx-autobuild==2021.3.14
sphinx-book-theme==0.3.2
sphinx-copybutton==0.5.0
sphinx-tabs==3.3.1
sphinx-togglebutton==0.3.1
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==2.0.0
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.5
sphinxext-opengraph==0.6.3
SQLAlchemy==1.4.36
stack-data==0.2.0
tabulate==0.8.9
terminado==0.15.0
threadpoolctl==3.1.0
tinycss2==1.1.1
toml==0.10.2
tomli==2.0.1
tomli_w==1.0.0
tornado==6.1
traitlets==5.2.1.post0
twine==4.0.0
typing-inspect==0.7.1
typing_extensions==4.2.0
urllib3==1.26.9
wcwidth==0.2.5
webencodings==0.5.1
websocket-client==1.3.2
wrapt==1.14.1
zipp==3.8.0

Describe the bug

context I try to access http://hostname:port//index.html

expectation I expected the webserver to return the content of index.html

bug But instead it tries to access a file named /index.html and returns a HTTP 403.

[W 220503 15:28:56 web:1787] 403 GET //index.html (172.17.0.1): /index.html is not in root static directory
[W 220503 15:28:56 web:2239] 403 GET //index.html (172.17.0.1) 0.86ms

problem This is a problem for people using sphinx themes in which some js resource path have two slashes. See https://github.com/jbms/sphinx-immaterial/issues/94

Reproduce the bug

1. Serve any sphinx doc with sphinx-autobuild
2. Access http://localhost:port/index.html (should work)
3. Access http://localhost:port//index.html (should not work)

List your environment

No response

Description / Summary

Currently the --ignore flag supports globs, but doesn't support recursive **-style globs. This makes it hard to ignore directories generated by autosummary.

Value / benefit

I have to come up with a regular expression to emulate this, which is annoying.

Implementation details

I looked at the implementation and it wasn't straightforward to add because the way it is currently implemented it uses fnmatch, and for whatever reason, recursive globs are not supported by fnmatch. They are only supported by glob, which reads the filesystem, but the current implementation uses a function that doesn't read the file system, presumably to be easier to test. Presumably the only reasonable fix here is to use some external fnmatch module that supports recursive globbing (I don't know of one, but presumably something like it exists), or to somehow automatically convert a recursive glob into a regex.

Tasks to complete

No response

Thanks so much for providing this great tool!

Describe the bug

When I run auto build in VS Code, much of the output is almost the same black color as the background of the terminal. After terminating the server, invoking 'Crtl-C' a couple of times is required to restore the default color to the terminal prompt.

To Reproduce

Run the code: sphinx-autobuild . --port 8020 _build/html

Expected behavior

Expect the terminal output text to be default color - almost white.

Environment

• VS Code 1.54.1
• Python Version [e.g. 3.8.5]:
• Win 10 - 10.0.19041