A lobby boy will create a VPS server when you need one, and destroy it after using it.

Overview

Lobbyboy

What is a lobby boy? A lobby boy is completely invisible, yet always in sight. A lobby boy remembers what people hate. A lobby boy anticipates the client's needs before the needs are needed. A lobby boy is, above all, discreet to a fault.

--The Grand Budapest Hotel

Test codecov PyPI version Python version

This project is still under testing, it worked but may have bugs.

What is lobbyboy?

Well, lobbyboy is a ssh server. Yes, like sshd. But instead of spawn a new shell on the server like sshd, when you ssh to lobbyboy, lobbyboy will create a new server(VPS) from available providers(meaning to say, DigitalOcean, AWS, GCP, Vultr, etc), then redirect you to the newly created servers. Of course, if lobbyboy finds any servers available already, he will just ask if you want to enter the existing server, or still want to create a new one.

Key Features

  • talks in SSH2 protocol, no need to install any software of configs for client-side, just ssh to lobbyboy!
  • extendable provider: just implement 3 methods, then lobbyboy can work with any provider!
  • destroy the server when you no longer needed.
  • manage ssh keys for you

Installation

Install libkrb5-dev first, this is a dependency for gssapi support.

apt install libkrb5-dev

Install via pip:

pip install lobbyboy

Run server

First, generate a config file:

lobbyboy-config-example > config.toml
# Edit your config before running!

Run the server with:

lobbyboy-server -c config.toml

You can ssh to Lobbyboy now, if you keep the default user Gustave in default config. You can ssh to Lobbyboy via:

ssh [email protected] -p 12200
# Enter the default password "Fiennes"(without quotes)
Welcome to Lobbyboy 0.2.2!
There are 1 available servers:
  0 - Create a new server...
  1 - Enter vagrant lobbyboy-41 127.0.0.1 (0 active sessions)
Please input your choice (number):

You may want to change the password in config.toml or use a public key for authentication. The latter is recommended in a production environment.

Generate a key pair for authentication

Generate a key pair:

ssh-keygen -f lobbyboy_key

Add the content of lobbyboy_key.pub to the end of authorized_keys under [user.Gustave] table. Now you can ssh to the lobbyboy server via:

ssh [email protected] -i lobbyboy_key

Deployment

Lobbyboy is supposed to be a server daemon, so you can manage it by systemd/supervisord or put it into a docker.

Systemd Example

[Unit]
Description=Lobbyboy Server

[Service]
User=me
Group=me
ExecStart=/path/to/lobbyboy-server -c /path/to/lobbyboy/config.toml
Restart=on-failure
WorkingDirectory=/path/to/lobbyboy/

[Install]
WantedBy=multi-user.target

Run in Docker

# Generate a config file
docker run --rm ghcr.io/lobbyboy-ssh/lobbyboy lobbyboy-config-example > lobbyboy_config.toml
# Run the docker container
docker run -v `pwd`/lobbyboy_config.toml:/app/config.toml -p "12200:12200" -d ghcr.io/lobbyboy-ssh/lobbyboy

The lobbyboy server should be active on 12200 port and you can connect to it with

ssh [email protected] -p 12200

The default password for user Gustave is Fiennes. Please change it when you deployed it into production, and consider use ssh key to auth instead of password.

Providers

// TBD

Builtin Providers

Lobbyboy current support multiple Providers:

  • Vagrant (Need vagrant and virtualbox to be installed)
  • Footlosse, in another words, containers (Need footloose and docker to be installed)
  • DigitalOcean
  • Linode

Different Providers support different configs, please see the example config for more detail.

Vagrant Provider

Vagrant Provider won't cost you any money, vagrant is a software runs on your computer along with virtual machine providers, vagrant can provision and control your VM.

This provider can help you to create a new Vagrant instance when you login to Lobbyboy, and destroy the server when you no longer use it.

Supported Features:

  • Create new Vagrant instances
  • You can configure your VM via vagrantfile config (see the config example).

Footloose Provider

footloose can make your docker containers(or Firecracker with ignite) act like virtual machine, so you can ssh to it.

Supported feature:

  • Configurable base image
  • Create a docker container and redirect you in

DigitalOcean Provider

This Provider will create Droplet from DigitalOcean.

Supported Features:

  • Create a new ssh key every time create a droplet.
  • Ask user to input region/droplet size/image when creating.
  • User can save favorite Droplet region/size/image in configs to quick create.
  • Destroy droplet when it is not in use.

Linode Provider

This Provider will create Node from Linode.

Supported Features:

  • Create a new ssh key every time create a droplet.
  • Ask user to input region/node type/image when creating.
  • User can save favorite node region/type/image in configs to quick create.
  • Destroy node when it is not in use.

Please see configs to check available options.

Ignite(Firecracker) Provider

Supported Features:

  • Create a new Firecracker virtual machine
  • Destroy node when it is not in use.

Write Your Own Providers

Providers are VPS vendors, by writing new providers, lobbyboy can work with any VPS vendors.

To make a new Provider work, you need to extend base class `lobbyboy.provider.BaseProvider``, implement 2 methods:

    def create_server(self, channel: Channel) -> LBServerMeta:
        """
        Args:
            channel: paramiko channel

        Returns:
            LBServerMeta: server meta info
        """
        ...


    def destroy_server(self, meta: LBServerMeta, channel: Channel = None) -> bool:
        """
        Args:
            meta: LBServerMeta, we use this to locate one server then destroy it.
            channel: Note that the channel can be None.
                     If called from server_killer, channel will be None.
                     if called when user logout from server, channel is active.

        Returns:
            bool: True if destroy successfully, False if not.
        """
        ...

Then add your Provider to your config file.

Those 3 configs are obligatory, as lobbyboy has to know when should he destroy your spare servers. You can add more configs, and read them from self.provider_config from code, just remember to add docs about it :)

[provider.<your provider name>]
load_module = "lobbyboy.contrib.provider.<your provider module name>::<Provider Class>"
min_life_to_live = "1h"
bill_time_unit = "1h"

Publish Your Own Providers

// TBD

FAQ

Q: Can I use lobbyboy as a proxy, like adding it to my ProxyCommand in ssh config?

A: No. Lobbyboy works like a reverse proxy, meaning to say, for ssh client, it just like a ssh server(sshd maybe), ssh client get a shell from lobbyboy, and doesn't know if it is local shell or it is a nested shell which runs another ssh. (but you know it, right? :D )

I Want to Know More!

Comments
  • LBConfigProvider is inflexible

    LBConfigProvider is inflexible

    My original idea for the provider is that:

    A provider can be defined in a single file, a provider's exceptions, implementations, and config definitions, all can be defined in a single file.

    So that it would be easy to maintain. And also, a provider can be implemented outside lobbyboy's codebase. (let's say, one can upload his provider to pypi, and other users can install them via pip install lobbyboy-aws-ec2-provider, then lobbyboy can load it.

    And one can add custom configs to his provider.

    The current problem is, we have to update the LBConfigProvider if one wants to add more fields to his provider config. It is not possible to do so without updating lobbyboy's source code.

    opened by laixintao 7
  • UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 4093-4094: unexpected end of data

    UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 4093-4094: unexpected end of data

    CRI [20211208-07:42:34.995] thr=140672309044992 lobbyboy.socket_handle:254: *** Socket thread error.                                                                                                                                                                                                                          Traceback (most recent call last):
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 247, in run
        self.user_using(server, proxy_subprocess)
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 188, in user_using
        send_to_channel(self.channel, os.read(master_fd, 10240).decode(), suffix="")
    UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 4093-4094: unexpected end of data
    DEB [20211208-07:42:35.003] thr=140672300570368 paramiko.transport:1819: EOF in transport thread
    
    opened by messense 5
  • Initialization Problems.

    Initialization Problems.

    I am really interested in your idea of lobbyboy. But I know nothing about python.

    So here is my problems during initialization, can you help me to figure it out?

    1. I configure my own data_dir but service start with an error. obbyboy.utils: Error when reading available_servers.json, [Errno 2] No such file or directory: '~/Software/lobbyboy/data/available_servers.json' Even I create the available_servers.json file, it does not work. Until I write {} to the file. But I'm not sure if this is ok for the lobbyboy service
    2. When I ssh with ssh [email protected] -p 12200, the logs shows an error below:
    ERR [20211119-17:40:41.664] thr=123145413386240 paramiko.transport: Unknown exception: too many values to unpack (expected 2)
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport: Traceback (most recent call last):
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:   File "/usr/local/lib/python3.9/site-packages/paramiko/transport.py", line 2109, in run
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:     handler(self.auth_handler, m)
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:   File "/usr/local/lib/python3.9/site-packages/paramiko/auth_handler.py", line 525, in _parse_userauth_request
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:     result = self.transport.server_object.check_auth_publickey(
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:   File "/usr/local/lib/python3.9/site-packages/lobbyboy/server.py", line 103, in check_auth_publickey
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:     _, _key_pub = line.split(" ", 2)
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport: ValueError: too many values to unpack (expected 2)
    

    I did some research but it's really difficult for me to understand python.

    So if you can give me a hand, that's would be nice.

    Thank you.

    opened by kimichen13 4
  • Use pre-commit as the linter

    Use pre-commit as the linter

    Close #48

    Install the commit hooks in dev:

    poetry run pre-commit install
    

    Run in CI: I recommend using pre-commit ci, it can upgrade the hooks and commit linter changes automatically. It is also blazing fast, e2e run time is < 10s

    BTW, the code is changed by the linter so this is better to merge after #55, otherwise there will be lots of conflicts

    opened by frostming 3
  • Recommended use pre-commit to make code style consistency

    Recommended use pre-commit to make code style consistency

    @laixintao @messense @frostming

    Recommended use pre-commit to make code style consistency, especially in the case of multi-person cooperation, what do you think?

    opened by luxiaba 3
  • Bump paramiko from 2.8.1 to 2.10.1

    Bump paramiko from 2.8.1 to 2.10.1

    Bumps paramiko from 2.8.1 to 2.10.1.

    Commits
    • 286bd9f Cut 2.10.1
    • 4c491e2 Fix CVE re: PKey.write_private_key chmod race
    • aa3cc6f Cut 2.10.0
    • e50e19f Fix up changelog entry with real links
    • 02ad67e Helps to actually leverage your mocked system calls
    • 29d7bf4 Clearly our agent stuff is not fully tested yet...
    • 5fcb8da OpenSSH docs state %C should also work in IdentityFile and Match exec
    • 1bf3dce Changelog enhancement
    • f6342fc Prettify, add %C as acceptable controlpath token, mock gethostname
    • 3f3451f Add to changelog
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • OSError: [Errno 9] Bad file descriptor

    OSError: [Errno 9] Bad file descriptor

    INF [20211208-07:44:51.056] thr=140672309044992 lobbyboy.utils:127: user choose 1 for option There are 1 available servers:
    INF [20211208-07:44:51.056] thr=140672309044992 lobbyboy.socket_handle:79: user choose server input=1.
    DEB [20211208-07:44:51.057] thr=140672309044992 lobbyboy.contrib.provider.ignite:86: get ssh to server command for ignite: ['cd dev_datadir/ignite/2021-12-08-0739 && ignite ssh 2021-12-08-0739']
    INF [20211208-07:44:51.057] thr=140672309044992 lobbyboy.socket_handle:103: ssh to server 2021-12-08-0739 127.0.0.1: cd dev_datadir/ignite/2021-12-08-0739 && ignite ssh 2021-12-08-0739
    INF [20211208-07:44:51.062] thr=140672309044992 lobbyboy.socket_handle:176: proxy subprocess created, pid=71892
    CRI [20211208-07:44:51.987] thr=140672309044992 lobbyboy.socket_handle:254: *** Socket thread error.
    Traceback (most recent call last):
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 252, in run
        self.cleanup(t, meta=lb_server, check_destroy=True)
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 194, in cleanup
        self.remove_server_session(t, meta.server_name)
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 223, in remove_server_session
        active_session[server_name] = list(filter(lambda x: x.getpeername() != peer_name, sessions))
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 223, in <lambda>
        active_session[server_name] = list(filter(lambda x: x.getpeername() != peer_name, sessions))
      File "/root/.cache/pypoetry/virtualenvs/lobbyboy-P-fHd6h2-py3.8/lib/python3.8/site-packages/paramiko/transport.py", line 1787, in getpeername
        return gp()
    OSError: [Errno 9] Bad file descriptor
    
    opened by messense 1
  • tests for need_destroy

    tests for need_destroy

    somehow some of my instance didn't get destroyed, I suspect there are some bugs in https://github.com/lobbyboy-ssh/lobbyboy/blob/41bba07e856e6ce31f7164232723e7440a469f18/lobbyboy/server_killer.py#L42

    but I didn't find any, let me add test cases later.

    opened by laixintao 3
  • Security check when lobbyboy server starts.

    Security check when lobbyboy server starts.

    Need to check:

    • user started lobbyboy with default username/password?
    • with the ssh host key https://github.com/lobbyboy-ssh/lobbyboy/blob/main/dev_datadir/ssh_host_rsa_key in git repo (middle man attack)

    If yes, then need to print warning information to stderr.

    related discussion: https://github.com/lobbyboy-ssh/lobbyboy/pull/32

    opened by laixintao 0
  • vultr provider support

    vultr provider support

    i want to add vultr provider and i found python-vultr, but seems that it hasn't been updated for a long time, and there are fatal bugs that can't be used normally, i write a new one pyvultr, can we use this to support vultr provider?

    opened by luxiaba 2
Releases(v0.4.0)
  • v0.4.0(Dec 11, 2021)

    What's Changed

    • fix typo of exmaple config file name. by @laixintao in https://github.com/lobbyboy-ssh/lobbyboy/pull/25
    • Fix typos and add codespell check to CI by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/26
    • Fix the path of example config.toml by @frostming in https://github.com/lobbyboy-ssh/lobbyboy/pull/28
    • Support docker container (based on footloose) by @laixintao in https://github.com/lobbyboy-ssh/lobbyboy/pull/30
    • ci: fix unittest running branch master-> main by @laixintao in https://github.com/lobbyboy-ssh/lobbyboy/pull/36
    • Add instructions to run in docker by @frostming in https://github.com/lobbyboy-ssh/lobbyboy/pull/32
    • Add ignite provider by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/37
    • Revamp provider config by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/41
    • Fix send_to_channel by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/44
    • Add support for enable/disable provider in config by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/45
    • Add a test case for load_config by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/49
    • Add a test case for load_providers by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/50
    • Add coverage report to CI by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/51
    • update readme: new architecture picture, update new providers. by @laixintao in https://github.com/lobbyboy-ssh/lobbyboy/pull/46
    • Define a default provider if there is only one provider by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/47

    New Contributors

    • @messense made their first contribution in https://github.com/lobbyboy-ssh/lobbyboy/pull/26
    • @frostming made their first contribution in https://github.com/lobbyboy-ssh/lobbyboy/pull/28

    Full Changelog: https://github.com/lobbyboy-ssh/lobbyboy/compare/v0.3.0...v0.4.0

    Source code(tar.gz)
    Source code(zip)
A lobby boy will create a VPS server when you need one, and destroy it after using it.

Lobbyboy What is a lobby boy? A lobby boy is completely invisible, yet always in sight. A lobby boy remembers what people hate. A lobby boy anticipate

226 Dec 29, 2022
Containerize a python web application

containerize a python web application introduction this document is part of GDSC at the university of bahrain you don't need to follow along, fell fre

abdullah mosibah 1 Oct 19, 2021
SSH tunnels to remote server.

Author: Pahaz Repo: https://github.com/pahaz/sshtunnel/ Inspired by https://github.com/jmagnusson/bgtunnel, which doesn't work on Windows. See also: h

Pavel White 1k Dec 28, 2022
Hw-ci - Hardware CD/CI and Development Container

Hardware CI & Dev Containter These containers were created for my personal hardware development projects and courses duing my undergraduate degree. Pl

Matthew Dwyer 6 Dec 25, 2022
Automatically capture your Ookla Speedtest metrics and display them in a Grafana dashboard

Speedtest All-In-One Automatically capture your Ookla Speedtest metrics and display them in a Grafana dashboard. Getting Started About This Code This

Aaron Melton 2 Feb 22, 2022
Knock your images before these make you painful.

image-knocker Knock your images before these make you painful. Background One day, I had run my deep learning model training program and got off work

Yonghye Kwon 9 Jul 25, 2022
Honcho: a python clone of Foreman. For managing Procfile-based applications.

___ ___ ___ ___ ___ ___ /\__\ /\ \ /\__\ /\ \ /\__\ /\

Nick Stenning 1.5k Jan 03, 2023
Tiny Git is a simplified version of Git with only the basic functionalities to gain better understanding of git internals.

Tiny Git is a simplified version of Git with only the basic functionalities to gain better understanding of git internals. Implemented Functi

Ahmed Ayman 2 Oct 15, 2021
Manage your SSH like a boss.

--- storm is a command line tool to manage your ssh connections. features adding, editing, deleting, listing, searching across your SSHConfig. command

Emre Yılmaz 3.9k Jan 03, 2023
Kubediff: a tool for Kubernetes to show differences between running state and version controlled configuration.

Kubediff: a tool for Kubernetes to show differences between running state and version controlled configuration.

Weaveworks 1.1k Dec 30, 2022
Helperpod - A CLI tool to run a Kubernetes utility pod with pre-installed tools that can be used for debugging/testing purposes inside a Kubernetes cluster

Helperpod is a CLI tool to run a Kubernetes utility pod with pre-installed tools that can be used for debugging/testing purposes inside a Kubernetes cluster.

Atakan Tatlı 2 Feb 05, 2022
Jenkins-AWS-CICD - Implement Jenkins CI/CD with AWS CodeBuild and AWS CodeDeploy, build a python flask web application.

Jenkins-AWS-CICD - Implement Jenkins CI/CD with AWS CodeBuild and AWS CodeDeploy, build a python flask web application.

Ning 1 Jan 01, 2022
Get Response Of Container Deployment Kube with python

get-response-of-container-deployment-kube 概要 get-response-of-container-deployment-kube は、例えばエッジコンピューティング環境のコンテナデプロイメントシステムにおいて、デプロイ元の端末がデプロイ先のコンテナデプロイ

Latona, Inc. 3 Nov 05, 2021
A cron monitoring tool written in Python & Django

Healthchecks Healthchecks is a cron job monitoring service. It listens for HTTP requests and email messages ("pings") from your cron jobs and schedule

Healthchecks 5.8k Jan 02, 2023
Let's learn how to build, release and operate your containerized applications to Amazon ECS and AWS Fargate using AWS Copilot.

🚀 Welcome to AWS Copilot Workshop In this workshop, you'll learn how to build, release and operate your containerised applications to Amazon ECS and

Donnie Prakoso 15 Jul 14, 2022
MLops tools review for execution on multiple cluster types: slurm, kubernetes, dask...

MLops tools review focused on execution using multiple cluster types: slurm, kubernetes, dask...

4 Nov 30, 2022
The low-level, core functionality of boto 3.

botocore A low-level interface to a growing number of Amazon Web Services. The botocore package is the foundation for the AWS CLI as well as boto3. On

the boto project 1.2k Jan 03, 2023
Azure plugins for Feast (FEAture STore)

Feast on Azure This project provides resources to enable running a feast feature store on Azure. Feast Azure Provider The Feast Azure provider acts li

Microsoft Azure 70 Dec 31, 2022
Find-Xss - Termux Kurulum Dosyası Eklendi Eğer Hata Alıyorsanız Lütfen Resmini Çekip İnstagramdan Bildiriniz

FindXss Waf Bypass Eklendi !!! PRODUCER: Saep UPDATER: Aser-Vant Download: git c

Aser 2 Apr 17, 2022
Deploying a production-ready Django project using Nginx and Gunicorn

django-nginx-gunicorn This project is for deploying a production-ready Django project using Nginx and Gunicorn. Running a local server of Django is no

Arash Sayareh 8 Jul 03, 2022