Build documentation in multiple repos into one site.

Overview

mkdocs-multirepo-plugin

Build documentation in multiple repos into one site.

Setup

Install plugin using pip:

pip install git+https://github.com/jdoiro3/mkdocs-multirepo-plugin

Next, add the plugin to your mkdocs.yml

plugins:
  - multirepo

The plugin introduces the !import statement in your config's nav section. You can now use the import statement to add a documentation section, where the docs are pulled from the source repo.

nav takes precedence over repos (see below).

nav:
  - Home: 'index.md'
  - MicroService: '!import https://github.com/{user}/{repo name}@{branch}'

If you'd prefer MkDocs to build the site nav based on the directory structure, you can define your other repos within the plugins section.

plugins:
  - multirepo:
      cleanup: True # (optional) tells multirepo to cleanup the temporary directory where other repo docs are imported to
      folder_name: multirepo_docs # (optional) tells multirepo what the temp directory should be called
      repos:
        - section: Backstage
          import_url: 'https://github.com/backstage/backstage'
        - section: Monorepo
          import_url: 'https://github.com/backstage/mkdocs-monorepo-plugin'
        - section: 'Techdocs-cli'
          import_url: 'https://github.com/backstage/[email protected]'

Notes

  • If both repos and nav is specified in mkdocs.yml, repos are ignored.

TODO

  • Add tests (no one will use it if it isn't tested well).
  • Add Linux support
    • Looks like git clone --sparse doesn't work with urls on Linux make sure git version is up to date
  • Figure out how tech writers can develop on local copies of repos and see changes in the site before pushing. Maybe using symbolic links.
Comments
  • Adds the access token for git only when executing a bash script for git

    Adds the access token for git only when executing a bash script for git

    The docs for an external repo are supposed to be included in the main menu directly without a docs sub menu entry. To achieve this, the contents of the docs folder inside a repository is moved up directly to the folder of the repo checkout.

    This feature was broken as the code tried to insert the AccessToken for git into the first argument, even though in this case the first argument is a folder name, not a git URL. This led to the the move failing (silently).

    opened by dhs3000 11
  • Building always fails - No such file or directory

    Building always fails - No such file or directory

    Hi, I'd like to thank you for making this awesome plugin, first and foremost! ♥

    Bug Description

    Every configuration I tried leads to a FileNotFoundError: [Errno 2] No such file or directory: exception. I can't figure out whether it's my fault or a real bug in the plugin.

    Let's assume we have the following, "main" doc site we're trying to build/serve:

    site_name: My Docs
    nav:
      - Home: 'index.md'
      - Foo: '!import {url}?branch=docs/mkdocs-bringup'
    plugins:
      - multirepo
    

    Note: I've intentionally hid the URL, but it's an enterprise Gitlab, if that's of any help. The real name of the project, currently represented as Foo, is also hidden.

    Foo contains a valid mkdocs.yml, with Nav properly defined.

    Now, when calling mkdocs build or mkdocs serve, I get this: FileNotFoundError: [Errno 2] No such file or directory: PosixPath('/home/timorg/my-docs/temp_dir/Foo')
    I also get this when moving the import from the Nav section to the plugin's config, i.e. listing sections.

    Logs

    Here's what I got when running mkdocs build --verbose:

    DEBUG    -  Loading configuration file: /home/timorg/service-development-docs/mkdocs.yml
    DEBUG    -  Loaded theme configuration for 'mkdocs' from '/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs/themes/mkdocs/mkdocs_theme.yml': {'static_templates': ['404.html'], 'locale':
                'en', 'include_search_page': False, 'search_index_only': False, 'highlightjs': True, 'hljs_languages': [], 'hljs_style': 'github', 'navigation_depth': 2, 'nav_style': 'primary', 'analytics':
                {'gtag': None}, 'shortcuts': {'help': 191, 'next': 78, 'previous': 80, 'search': 83}}
    DEBUG    -  Config value: 'config_file_path' = '/home/timorg/my-docs/mkdocs.yml'
    DEBUG    -  Config value: 'site_name' = 'Service Development Docs'
    DEBUG    -  Config value: 'nav' = [{'Home': 'index.md'}, {'Datastore': '!import {url}?branch=docs/mkdocs-bringup'}]
    DEBUG    -  Config value: 'pages' = None
    DEBUG    -  Config value: 'site_url' = ''
    DEBUG    -  Config value: 'site_description' = None
    DEBUG    -  Config value: 'site_author' = None
    DEBUG    -  Config value: 'theme' = Theme(name='mkdocs', dirs=['/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs/themes/mkdocs',
                '/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs/templates'], static_templates=['404.html', 'sitemap.xml'], locale=Locale(language='en', territory=''),
                include_search_page=False, search_index_only=False, highlightjs=True, hljs_languages=[], hljs_style='github', navigation_depth=2, nav_style='primary', analytics={'gtag': None}, shortcuts={'help':
                191, 'next': 78, 'previous': 80, 'search': 83})
    DEBUG    -  Config value: 'docs_dir' = '/home/timorg/service-development-docs/docs'
    DEBUG    -  Config value: 'site_dir' = '/home/timorg/service-development-docs/site'
    DEBUG    -  Config value: 'copyright' = None
    DEBUG    -  Config value: 'google_analytics' = None
    DEBUG    -  Config value: 'dev_addr' = Address(host='127.0.0.1', port=8000)
    DEBUG    -  Config value: 'use_directory_urls' = True
    DEBUG    -  Config value: 'repo_url' = ''
    DEBUG    -  Config value: 'repo_name' = ''
    DEBUG    -  Config value: 'edit_uri' = ''
    DEBUG    -  Config value: 'extra_css' = []
    DEBUG    -  Config value: 'extra_javascript' = []
    DEBUG    -  Config value: 'extra_templates' = []
    DEBUG    -  Config value: 'markdown_extensions' = ['toc', 'tables', 'fenced_code']
    DEBUG    -  Config value: 'mdx_configs' = {}
    DEBUG    -  Config value: 'strict' = False
    DEBUG    -  Config value: 'remote_branch' = 'gh-pages'
    DEBUG    -  Config value: 'remote_name' = 'origin'
    DEBUG    -  Config value: 'extra' = {}
    DEBUG    -  Config value: 'plugins' = PluginCollection([('multirepo', <mkdocs_multirepo_plugin.plugin.MultirepoPlugin object at 0x7f3746f10a90>)])
    DEBUG    -  Config value: 'watch' = None
    INFO     -  Multirepo plugin importing docs...
                        :   0%|                                                                                                                                                                | 0/1 [00:00<?, ?it/s]Traceback (most recent call last):
      File "/home/timorg/.local/bin/mkdocs", line 8, in <module>
        sys.exit(cli())
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
        return self.main(*args, **kwargs)
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/click/core.py", line 1055, in main
        rv = self.invoke(ctx)
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/click/core.py", line 760, in invoke
        return __callback(*args, **kwargs)
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs/__main__.py", line 192, in build_command
        build.build(config.load_config(**kwargs), dirty=not clean)
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs/commands/build.py", line 257, in build
        config = config['plugins'].run_event('config', config)
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs/plugins.py", line 102, in run_event
        result = method(item, **kwargs)
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs_multirepo_plugin/plugin.py", line 150, in on_config
        return self.handle_nav_based_import(config)
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs_multirepo_plugin/plugin.py", line 103, in handle_nav_based_import
        asyncio_run(batch_import(repos))
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs_multirepo_plugin/util.py", line 115, in asyncio_run
        asyncio.run(futures)
      File "/home/timorg/.pyenv/versions/3.9.12/lib/python3.9/asyncio/runners.py", line 44, in run
        return loop.run_until_complete(main)
      File "/home/timorg/.pyenv/versions/3.9.12/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
        return future.result()
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs_multirepo_plugin/structure.py", line 255, in batch_import
        repo = await import_async
      File "/home/timorg/.pyenv/versions/3.9.12/lib/python3.9/asyncio/tasks.py", line 611, in _wait_for_one
        return f.result()  # May raise f.exception().
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs_multirepo_plugin/structure.py", line 237, in import_docs
        await execute_bash_script("mv_docs_up.sh", [self.docs_dir.replace("/*", "")], cwd=self.location)
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs_multirepo_plugin/util.py", line 97, in execute_bash_script
        process = await asyncio.create_subprocess_exec(
      File "/home/timorg/.pyenv/versions/3.9.12/lib/python3.9/asyncio/subprocess.py", line 236, in create_subprocess_exec
        transport, protocol = await loop.subprocess_exec(
      File "/home/timorg/.pyenv/versions/3.9.12/lib/python3.9/asyncio/base_events.py", line 1676, in subprocess_exec
        transport = await self._make_subprocess_transport(
      File "/home/timorg/.pyenv/versions/3.9.12/lib/python3.9/asyncio/unix_events.py", line 197, in _make_subprocess_transport
        transp = _UnixSubprocessTransport(self, protocol, args, shell,
      File "/home/timorg/.pyenv/versions/3.9.12/lib/python3.9/asyncio/base_subprocess.py", line 36, in __init__
        self._start(args=args, shell=shell, stdin=stdin, stdout=stdout,
      File "/home/timorg/.pyenv/versions/3.9.12/lib/python3.9/asyncio/unix_events.py", line 789, in _start
        self._proc = subprocess.Popen(
      File "/home/timorg/.pyenv/versions/3.9.12/lib/python3.9/subprocess.py", line 951, in __init__
        self._execute_child(args, executable, preexec_fn, close_fds,
      File "/home/timorg/.pyenv/versions/3.9.12/lib/python3.9/subprocess.py", line 1821, in _execute_child
        raise child_exception_type(errno_num, err_msg, err_filename)
    FileNotFoundError: [Errno 2] No such file or directory: PosixPath('/home/timorg/my-docs/temp_dir/Foo')
                        :   0%|
    

    Note that I've yet again replaced namings in the log to match those defined above. Sorry but it's company-internal info.

    Environment

    OS: Linux (WSL on top of Windows 10) Distro: Ubuntu 20.04 Python: 3.9.12 (Built with pyenv)

    opened by MrPointer 11
  • Issues when running multirepo

    Issues when running multirepo

    When i run mkdocs serve I get the following output.

    INFO     -  Building documentation...
    WARNING  -  Multirepo plugin is ignoring plugins.multirepo.repos. Nav takes precedence
    INFO     -  Multirepo plugin importing docs...
    INFO     -  Cleaning site directory
    INFO     -  Multirepo plugin is cleaning up temp_dir/
    INFO     -  Documentation built in 0.23 seconds
    INFO     -  [16:55:53] Watching paths for changes: 'docs', 'mkdocs.yml'
    INFO     -  [16:55:53] Serving on http://127.0.0.1:8000/
    WARNING  -  [16:55:54] "GET /quix-apis/ HTTP/1.1" code 404
    INFO     -  [16:56:07] Browser connected: http://localhost:8000/
    

    This is my yaml file.. I mostly copied the bits from your readme.

    site_name: My Docs
     
    nav:
      - Home: 'index.md'
      
    plugins:
      - search
      - multirepo:
          # (optional) tells multirepo to cleanup the temporary directory after site is built.
          cleanup: true
          repos:
            - section: Backstage
              # you can define the edit uri path
              import_url: 'https://github.com/backstage/backstage?edit_uri=/blob/master/'
            - section: Monorepo
              import_url: 'https://github.com/backstage/mkdocs-monorepo-plugin?edit_uri=/blob/master/'
            - section: 'Techdocs-cli'
              # note that the branch is still specified in the url
              import_url: 'https://github.com/backstage/techdocs-cli?branch=main&edit_uri=/blob/main/'
            - section: FastAPI
              import_url: 'https://github.com/tiangolo/fastapi?docs_dir=docs/en/docs/*'
            - section: Monorepo Multi Docs
              import_url: https://github.com/backstage/mkdocs-monorepo-plugin?multi_docs=True&docs_dir=sample-docs/*
            - section: 'Django REST'
              section_path: python # Put this under the python menu entry
              import_url: 'https://github.com/encode/django-rest-framework'
            - section: 'Cookiecutter Pypackage'
              section_path: python # Put this under the python menu entry
              import_url: 'https://github.com/zillionare/cookiecutter-pypackage'
            - section: 'Pydantic'
              section_path: python # Put this under the python menu entry
              import_url: 'https://github.com/samuelcolvin/pydantic?branch=main'
    

    I wasn't sure from the docs whether I needed the nav section as well as the plugins config. But without the nav section it fails to build. I noticed WITH the nav section that the output is saying Nav take precedence.

    opened by SteveRosam 10
  • Clarity in the documentation

    Clarity in the documentation

    Hello,

    To an end user of this plugin, it is not clear if the plugin even builds the documentation in the other repositories that it references. It seems to me that the plugin does the following: 1)Reference already built documentation in other repositories and make use of them in this parent repository.

    The plugin does not really build the documentation (i.e run mkdocs build ) in each of the other repositories that the main repository references. This should be made crystal clear so that end users know what the plugin is capable of.

    The current wording in the documentation is ambiguous and is not clear.`

    Thanks, Kiran Hegde

    documentation 
    opened by kirannhegde 9
  • deep nav imports feature

    deep nav imports feature

    needed this for my own personal use - hopefully it will seem like a logical feature to you.

    i ran the tests in my fork and everything was passing, excluding the publish (since i dont have the pypi secret)

    @jdoiro3

    opened by reedsutliff 8
  • nav links to /!import%20url resulting in 404 warning

    nav links to /!import%20url resulting in 404 warning

    Thanks for the plugin. I am trying to set it up with the following parent mkdocs.yml:

    site_name: Documentation
    
    use_directory_urls: false # Either true or false results in the same issue
    
    plugins:
      - search
      - multirepo
    
    nav:
      - Main: index.md
      - DemoRepo: '!import https://github.com/jdoiro3/mkdocs-multirepo-demoRepo1?branch=main'
    

    This builds/serves fine and I can see a temporary folder being created where demo repo is cloned. However, when I try to navigate to the "DemoRepo" menu item I get a 404 page because the link is:

    http://127.0.0.1:8000/!import%20https:/github.com/jdoiro3/mkdocs-multirepo-demoRepo1?branch=main

    I also see a warning:

    WARNING - [15:35:39] "GET /!import%20https:/github.com/jdoiro3/mkdocs-multirepo-demoRepo1?branch=main HTTP/1.1" code 404

    The python version I am using is 3.9.10 on Windows and the following mkdocs versions:

    mkdocs                     1.2.3
    mkdocs-material            8.1.8
    mkdocs-material-extensions 1.0.3
    mkdocs-multirepo-plugin    0.2.6
    

    Do you have any idea what might cause this issue?

    bug additional testing needed 
    opened by fjp 8
  • Adds specific GitHub access token

    Adds specific GitHub access token

    We are using a GitHub App to create an access token rather than using a personal access token (PAT). Using this token with the environment variable AccessToken yields the git error message fatal: could not read Username for 'https://github.com/': No such device or address.

    This PR introduces a new GithubAccessToken environment variable, which is used in the git URL as https://x-access-token:[email protected]/.... without setting the extraheader authorization header (GitHub docs). This approach works with a PAT and an access token from a GitHub app (I tested the former locally from inside a docker container to prevent config leaking).

    The functionality for the existing AccessToken environment variable stays as it is, as it might be different for other environments/git servers.


    • This addresses the issues #13 and initial description of #33.
    • This replaces #54 as the access token is now inserted into the URL inside the bash script and not in util.execute_bash_script which is also used for moving the docs folder.
    opened by dhs3000 7
  • Sparse checkout not working with recent git versions

    Sparse checkout not working with recent git versions

    GitHub Actions recently updated their virtual environments (see ubuntu-latest for example) and now git cli version is 2.37. This is causing problems with git sparse-checkout set used in sparse_clone.sh script because it doesn't admit patterns to be passed.

    Looking at the git docs for a version that works and the one that doesn't seems that we we need to pass --no-cone flag in order to keep using patterns in the sparse checkout. I tested locally modifying the script in my local virtual environment and works but I'm not sure if it's the "real" fix, not very familiar with sparse checkouts.

    opened by rmunoz 7
  • ValueError: max() arg is an empty sequence

    ValueError: max() arg is an empty sequence

    while trying out the plugin, I enabled it after the installation with:

    plugins:
      - multirepo
    

    After that I wanted to start the builtin development server with mkdocs serve. But while doing so I get the following error:

    Error Message

    INFO     -  Building documentation...
    INFO     -  Multirepo plugin importing docs...
    Traceback (most recent call last):
      File "/home/vscode/.local/bin/mkdocs", line 8, in <module>
        sys.exit(cli())
      File "/home/vscode/.local/lib/python3.10/site-packages/click/core.py", line 1128, in __call__
        return self.main(*args, **kwargs)
      File "/home/vscode/.local/lib/python3.10/site-packages/click/core.py", line 1053, in main
        rv = self.invoke(ctx)
      File "/home/vscode/.local/lib/python3.10/site-packages/click/core.py", line 1659, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/home/vscode/.local/lib/python3.10/site-packages/click/core.py", line 1395, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "/home/vscode/.local/lib/python3.10/site-packages/click/core.py", line 754, in invoke
        return __callback(*args, **kwargs)
      File "/home/vscode/.local/lib/python3.10/site-packages/mkdocs/__main__.py", line 177, in serve_command
        serve.serve(dev_addr=dev_addr, livereload=livereload, **kwargs)
      File "/home/vscode/.local/lib/python3.10/site-packages/mkdocs/commands/serve.py", line 54, in serve
        config = builder()
      File "/home/vscode/.local/lib/python3.10/site-packages/mkdocs/commands/serve.py", line 49, in builder
        build(config, live_server=live_server, dirty=dirty)
      File "/home/vscode/.local/lib/python3.10/site-packages/mkdocs/commands/build.py", line 257, in build
        config = config['plugins'].run_event('config', config)
      File "/home/vscode/.local/lib/python3.10/site-packages/mkdocs/plugins.py", line 102, in run_event
        result = method(item, **kwargs)
      File "/home/vscode/.local/lib/python3.10/site-packages/mkdocs_multirepo_plugin/plugin.py", line 165, in on_config
        return self.handle_nav_based_import(config)
      File "/home/vscode/.local/lib/python3.10/site-packages/mkdocs_multirepo_plugin/plugin.py", line 117, in handle_nav_based_import
        asyncio_run(batch_import(docs_repo_objs))
      File "/home/vscode/.local/lib/python3.10/site-packages/mkdocs_multirepo_plugin/util.py", line 117, in asyncio_run
        asyncio.run(futures)
      File "/usr/local/lib/python3.10/asyncio/runners.py", line 44, in run
        return loop.run_until_complete(main)
      File "/usr/local/lib/python3.10/asyncio/base_events.py", line 641, in run_until_complete
        return future.result()
      File "/home/vscode/.local/lib/python3.10/site-packages/mkdocs_multirepo_plugin/structure.py", line 185, in batch_import
        longest_desc = max([len(f"✅ Got {repo.name} Docs") for repo in repos])
    ValueError: max() arg is an empty sequence
    The terminal process "/usr/bin/zsh '-c', 'mkdocs serve'" terminated with exit code: 1.
    

    Am I missing something?

    bug 
    opened by tepene 7
  • Unable to checkout child imports with github actions

    Unable to checkout child imports with github actions

    I am creating a 'documentation' repo in github which hosts a stub mkdocs.yml file under which mkdocs-multirepo-plugin should build documentation from other repos. The .yml is:

    site_name: MyOrg Docs
    theme: readthedocs
    plugins:
      - multirepo
      - search
    
    nav:
      - Home: 'index.md'
      - Included Services:
        - Service1: '!import https://github.com/myOrg/myRepo1?docs_dir=mydocs/*?multi_docs=True?config=mydocs/mkdocs.yml'
        - Service2: '!import https://github.com/myOrg/myRepo2?docs_dir=mydocs/*?multi_docs=True?config=mydocs/mkdocs.yml'
    

    When I run mkdocs buildfrom my machine, it works.

    I want to trigger the build of docs from a github action. Per the documentation, I added github personal access token (PAT) to the repo secrets and included that in my workflow. The workflow .yml is:

    name: Build and Publish Docs 
    
    
    on:
      push:
        branches: ['develop']
    
    env:
      AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
    
    jobs:
    
      build_and_publish:
        runs-on: ubuntu-latest
    
        steps:
          - name: Checkout
            uses: actions/[email protected]
    
          - name: Install Python
            uses: actions/[email protected]
            with:
              python-version: 3.8
    
          - name: Install pip requirements
            run: |
              python -m pip install --upgrade pip
              pip install -r requirements.txt
    
          - name: Run mkdocs
            run: mkdocs build
            working-directory: docs-root
            env:
              AccessToken: ${{ secrets.MY_PAT }}
    
          #- name: Publish to S3
    

    The output of the 'Run mkdocs' step is:

    Run mkdocs build
      mkdocs build
      shell: /usr/bin/bash -e {0}
      env:
        AWS_DEFAULT_REGION: 
        AWS_SECRET_ACCESS_KEY: 
        AWS_ACCESS_KEY_ID: 
        pythonLocation: /opt/hostedtoolcache/Python/3.8.1[2](https://github.com/BuddyTV/papyrus/runs/6177823965?check_suite_focus=true#step:5:2)/x64
        LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/[3](https://github.com/BuddyTV/papyrus/runs/6177823965?check_suite_focus=true#step:5:3).8.12/x6[4](https://github.com/BuddyTV/papyrus/runs/6177823965?check_suite_focus=true#step:5:4)/lib
        AccessToken: ***
    INFO     -  Multirepo plugin importing docs...
    
                          :   0%|          | 0/2 [00:00<?, ?it/s]Traceback (most recent call last):
      File "/opt/hostedtoolcache/Python/3.8.12/x64/bin/mkdocs", line 8, in <module>
        sys.exit(cli())
      File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
        return self.main(*args, **kwargs)
      
    <<snip>>
    
        stdout = await execute_bash_script("sparse_clone.sh", args, self.temp_dir)
      File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/mkdocs_multirepo_plugin/util.py", line 109, in execute_bash_script
        raise GitException(f"\n{stderr}\n")
    mkdocs_multirepo_plugin.util.GitException: 
    Cloning into 'PIN Service'...
    fatal: could not read Username for 'https://github.com': No such device or address
    
    
    Exception ignored in: <function BaseSubprocessTransport.__del__ at 0x7fe74cd8e280>
    Traceback (most recent call last):
      File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/asyncio/base_subprocess.py", line 1[26](https://github.com/BuddyTV/papyrus/runs/6177823965?check_suite_focus=true#step:5:26), in __del__
        self.close()
      File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/asyncio/base_subprocess.py", line 104, in close
        proto.pipe.close()
      
    <<snip>>
    
      File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/asyncio/base_events.py", line [50](https://github.com/BuddyTV/papyrus/runs/6177823965?check_suite_focus=true#step:5:50)8, in _check_closed
    Error:     raise RuntimeError('Event loop is closed')
    RuntimeError: Event loop is closed
    
                          :   0%|          | 0/2 [00:00<?, ?it/s]
    Error: Process completed with exit code 1.
    

    What am I doing wrong?

    Change needed 
    opened by matthewmucker-vizio 6
  • Doesn't work for self hosted github enterprise?

    Doesn't work for self hosted github enterprise?

    I think it doesn't work for self-hosted github enterprise. The URL here is git..com (only vpn accessible), and when running mkdocs serve:

    INFO     -  Multirepo plugin is importing docs for section Section 1
    [Errno 2] No such file or directory: 'where'
    

    This is an example of the mkdocs.yml:

    # Project information
    site_name: Platform Wiki
    site_description: Centralized location for all documentation
    site_author: Platform
    
    # Repository
    repo_name: platform/centralized-docs
    repo_url: https://git.<company>.com/platform/centralized-docs
    
    nav:
      - Home: index.md
      - Login Server: '!import https://git.<company>.com/<org>/<repo>@master'
    
    # Configuration
    theme:
      name: material
      palette:
        primary: indigo
        accent: pink
        scheme: default
    
    plugins:
      - multirepo:
          # (optional) tells multirepo to cleanup the temporary directory after site is built.
          cleanup: true
          # (optional) tells multirepo what the temp directory should be called
          temp_dir: multirepo_docs
    
    
    bug 
    opened by acastro2 5
  • Temp dir issue

    Temp dir issue

    I'm getting this when trying to build.

    The process cannot access the file because it is being used by another process

    I have deleted the temp_dir and still get the same. Has anyone found a work around and is there a build flag or argument to change the temp directory?

    opened by SteveRosam 1
  • Create .git/info dir if it does not exist

    Create .git/info dir if it does not exist

    git init does not necessarily create .git/info, might be version-dependent.

    Fixes #58 (but does not address the general error checking issue of this script, i.e. failing intermediate commands will still get "swallowed").

    opened by hjoukl 2
  • sparse_clone_old.sh might not do a sparse checkout after all

    sparse_clone_old.sh might not do a sparse checkout after all

    git init might not produce a .git/info dir:

    $ mkdir myrepo
    $ cd myrepo/
    $ git init
    Initialized empty Git repository in /tmp/myrepo/.git/
    $ ls -l .git/
    total 8
    -rw-r--r--. 1 holger intern 92 Oct  6 16:19 config
    -rw-r--r--. 1 holger intern 23 Oct  6 16:19 HEAD
    drwxr-xr-x. 2 holger intern 27 Oct  6 16:19 hooks/
    drwxr-xr-x. 4 holger intern 30 Oct  6 16:19 objects/
    drwxr-xr-x. 4 holger intern 31 Oct  6 16:19 refs/
    $ git --version
    git version 2.24.1
    

    Should probably add an mkdir line:

    ...
    git remote add -f origin "$url"                     
    mkdir -p .git/info  # this directory might not exist                           
    for dir in "${dirs[@]}"
    ...
    

    Note that you won't really notice this currently since the shell script will simply continue for the failing commands. This could be mitigated by using set -e errexit or (better) explicit error checking.

    Best, Holger

    opened by hjoukl 2
  • Symlinks from docs dir to toplevel project dir do not work

    Symlinks from docs dir to toplevel project dir do not work

    Hi,

    nice plugin!

    Suppose you have a repo myrepowith toplevel LICENSE and AUTHORS documents that are symlinked from the docs folder, to avoid duplication but use this info in the mkdocs-generated documentation:

    AUTHORS.md
    LICENSE.md
    docs
    ├── about
    │   ├── AUTHORS.md -> ../../AUTHORS.md
    │   └── LICENSE.md -> ../../LICENSE.md
    ├── index.md
    mkdocs.yml
    

    This works with mkdocs serve|build when the repo is used "standalone".

    However, if you want such a repo to be imported with multirepo-plugin:

    # mkdocs.yml of the importing repo
    nav:
      - Home: index.md
      - Documentation:
        - myrepo: '!import https://vcs.example.org/user/myrepo?branch=feature/great-improvements&docs_dir=docs&multi_docs=False&config=mkdocs.yml&extra_imports=["AUTHORS.md", "LICENSE.md"]'
    

    ... the symlinks get broken in the temp_dir sparse checkout:

    $ tree temp_dir/
    temp_dir/
    └── documentation
        └── myrepo
            ├── about
            │   ├── AUTHORS.md -> ../../AUTHORS.md
            │   └── LICENSE.md -> ../../LICENSE.md
            ├── AUTHORS.md
            ├── index.md
            ├── LICENSE.md
            ├── mkdocs.yml
    

    This is due to the way the docs folders contents are put into temp_dir/documentation/myrepo directly, which changes the nesting level. This makes the symlinks invalid.

    Making the links the other way round (e.g. `AUTHORS.md --> docs/about/AUTHORS.md) isn't attractive since you lose the ability to directly render the markdown files in the VCS-GUI (at least in bitbucket, in my case), since the actual git repository content of such symlinks is the path to the symlinked file.

    And you also can't just reference a markdown file in nav: ...outside of the docs_dir.

    Workaround: Duplicate files. Avoid links crossing docs_dir.

    Best regards, Holger

    opened by hjoukl 0
  • including code from paths outside of the mkdocs directory

    including code from paths outside of the mkdocs directory

    Does the plugin support importing code snippets from other paths ?

    We have often things like this in our doc:

    --8<-- "../component/src/test/java/mkdocs/Example.java"
    

    When building a multirepo mkdocs site, it fails:

    ERROR    -  Error reading page 'example/index.md': Snippet at path
                ../component/src/test/java/mkdocs/Example.java could not be found
    

    I am using nav based multirepo import like this:

    Example: '!import [email protected]:myrepo.git?extra_imports=["component/*"]'
    

    The imported repo structure is this:

    component
      src
        test
          java
    mkdocs
      mkdocs.yml
    
    enhancement 
    opened by jorgheymans 3
Releases(v0.4.12)
  • v0.4.12(Oct 21, 2022)

  • v0.4.11(Oct 20, 2022)

    Changes in this Release

    • Added a specific GithubAccessToken environment variable that allows usage of the plugin with GitHub Apps-generated access tokens and personal access tokens.
    • Fixed edit urls for imported repos where the the Mkdocs edit_uri and repo_url aren't set, and the Multirepo edit_uri for an imported repo isn't set. Note that the edit url for imported repos hasn't been robustly tested yet.
    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(Jun 22, 2022)

    Breaking Changes

    • Previously, you would separate config values in an !import statement using multiple ampersands (&). Now, the !import statement will follow general URL syntax, where the url and query string (config values) is separated by one question mark (?) and then each config section is separated by an ampersand. Below is an example !import statement. '!import {url}?branch={branch}&docs_dir={path}&multi_docs={True | False}&config={filename}.yml'

    Other Changes

    • Section names are slugified for clear urls (see PR https://github.com/jdoiro3/mkdocs-multirepo-plugin/pull/28 and Issue https://github.com/jdoiro3/mkdocs-multirepo-plugin/issues/24 for details). This most likely will change in future releases.
    Source code(tar.gz)
    Source code(zip)
  • v0.3.6(May 10, 2022)

  • v0.3.5(Apr 8, 2022)

  • v0.3.3(Apr 1, 2022)

    Additions

    Previously, !import statements could only be included within top level sections in the nav. Now, the !import statement can be located anywhere in the nav.

    nav:
      - Home: index.md
      - microservice:
          - Home: test.md
          - ServiceA: '!import https://github.com/jdoiro3/mkdocs-multirepo-demoRepo1?branch=main'
          - ServiceB: '!import https://github.com/jdoiro3/mkdocs-multirepo-demoRepo1?branch=main'
    
    Source code(tar.gz)
    Source code(zip)
  • v0.3.2(Mar 19, 2022)

    Additions

    • Users can now store the multirepo config file for an imported repo either within the docs directory or the root directory. If the name of the config isn't mkdocs.yml, it can also be specified via the new url parameter: ?config={filename}.yml.

    Changes

    • Fixed the issue where build outputs contained configuration files (see issue #9 for details).
    Source code(tar.gz)
    Source code(zip)
  • v0.3.1(Mar 3, 2022)

    Changes

    • Section names with spaces were causing two directories to be created when the version of git was old (i.e., sparse_clone_old.sh was run).
    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Feb 21, 2022)

  • v0.2.9(Feb 19, 2022)

  • v0.2.7(Feb 18, 2022)

  • v0.2.3(Feb 17, 2022)

  • v0.2.0(Feb 10, 2022)

    Alpha Release

    The minimal features of mkdocs-multirepo-plugin have been implemented but have yet to be fully tested.

    Features

    • Build docs from multiple repos into one site
    • Importing repos that are monorepos (not tested well)
    • Run mkdocs serve within an imported repo for local development (not tested well)
    Source code(tar.gz)
    Source code(zip)
Owner
Joseph Doiron
Not an expert yet but I'm trying.
Joseph Doiron
Coursera learning course Python the basics. Programming exercises and tasks

HSE_Python_the_basics Welcome to BAsics programming Python! You’re joining thousands of learners currently enrolled in the course. I'm excited to have

PavelRyzhkov 0 Jan 05, 2022
OpenAPI (f.k.a Swagger) Specification code generator. Supports C#, PowerShell, Go, Java, Node.js, TypeScript, Python

AutoRest The AutoRest tool generates client libraries for accessing RESTful web services. Input to AutoRest is a spec that describes the REST API usin

Microsoft Azure 4.1k Jan 06, 2023
Repository for learning Python (Python Tutorial)

Repository for learning Python (Python Tutorial) Languages and Tools 🧰 Overview 📑 Repository for learning Python (Python Tutorial) Languages and Too

Swiftman 2 Aug 22, 2022
Que es S4K Builder?, Fácil un constructor de tokens grabbers con muchas opciones, como BTC Miner, Clipper, shutdown PC, Y más! Disfrute el proyecto. <3

S4K Builder Este script Python 3 de código abierto es un constructor del muy popular registrador de tokens que está en [mi GitHub] (https://github.com

SadicX 1 Oct 22, 2021
This contains timezone mapping information for when preprocessed from the geonames data

when-data This contains timezone mapping information for when preprocessed from the geonames data. It exists in a separate repository so that one does

Armin Ronacher 2 Dec 07, 2021
The sarge package provides a wrapper for subprocess which provides command pipeline functionality.

Overview The sarge package provides a wrapper for subprocess which provides command pipeline functionality. This package leverages subprocess to provi

Vinay Sajip 14 Dec 18, 2022
This tutorial will guide you through the process of self-hosting Polygon

Hosting guide This tutorial will guide you through the process of self-hosting Polygon Before starting Make sure you have the following tools installe

Polygon 2 Jan 31, 2022
Cleaner script to normalize knock's output EPUBs

clean-epub The excellent knock application by Benton Edmondson outputs EPUBs that seem to be DRM-free. However, if you run the application twice on th

2 Dec 16, 2022
Type hints support for the Sphinx autodoc extension

sphinx-autodoc-typehints This extension allows you to use Python 3 annotations for documenting acceptable argument types and return value types of fun

Alex Grönholm 462 Dec 29, 2022
API Documentation for Python Projects

API Documentation for Python Projects. Example pdoc -o ./html pdoc generates this website: pdoc.dev/docs. Installation pip install pdoc pdoc is compat

mitmproxy 1.4k Jan 07, 2023
Soccerdata - Efficiently scrape soccer data from various sources

SoccerData is a collection of wrappers over soccer data from Club Elo, ESPN, FBr

Pieter Robberechts 195 Jan 04, 2023
Valentine-with-Python - A Python program generates an animation of a heart with cool texts of your loved one

Valentine with Python Valentines with Python is a mini fun project I have coded.

Niraj Tiwari 4 Dec 31, 2022
Jupyter Notebooks as Markdown Documents, Julia, Python or R scripts

Have you always wished Jupyter notebooks were plain text documents? Wished you could edit them in your favorite IDE? And get clear and meaningful diff

Marc Wouts 5.7k Jan 04, 2023
PySpark Cheat Sheet - learn PySpark and develop apps faster

This cheat sheet will help you learn PySpark and write PySpark apps faster. Everything in here is fully functional PySpark code you can run or adapt to your programs.

Carter Shanklin 168 Jan 01, 2023
A python package to import files from an adjacent folder

EasyImports About EasyImports is a python package that allows users to easily access and import files from sister folders: f.ex: - Project - Folde

1 Jun 22, 2022
An open source utility for creating publication quality LaTex figures generated from OpenFOAM data files.

foamTEX An open source utility for creating publication quality LaTex figures generated from OpenFOAM data files. Explore the docs » Report Bug · Requ

1 Dec 19, 2021
30 days of Python programming challenge is a step-by-step guide to learn the Python programming language in 30 days

30 days of Python programming challenge is a step-by-step guide to learn the Python programming language in 30 days. This challenge may take more than100 days, follow your own pace.

Asabeneh 17.7k Jan 07, 2023
Automated Integration Testing and Live Documentation for your API

Automated Integration Testing and Live Documentation for your API

ScanAPI 1.3k Dec 30, 2022
Members: Thomas Longuevergne Program: Network Security Course: 1DV501 Date of submission: 2021-11-02

Mini-project report Members: Thomas Longuevergne Program: Network Security Course: 1DV501 Date of submission: 2021-11-02 Introduction This project was

1 Nov 08, 2021