A lobby boy will create a VPS server when you need one, and destroy it after using it.

Overview

Lobbyboy

What is a lobby boy? A lobby boy is completely invisible, yet always in sight. A lobby boy remembers what people hate. A lobby boy anticipates the client's needs before the needs are needed. A lobby boy is, above all, discreet to a fault.

--The Grand Budapest Hotel

This project is still under testing, it worked but may have bugs.

What is lobbyboy?

Well, lobbyboy is a ssh server. Yes, like sshd. But instead of spawn a new shell on the server like sshd, when you ssh to lobbyboy, lobbyboy will create a new server(VPS) from available providers(meaning to say, DigitalOcean, AWS, GCP, Vultr, etc), then redirect you to the newly created servers. Of course, if lobbyboy finds any servers available already, he will just ask if you want to enter the existing server, or still want to create a new one.

                                                       create
 +------------------+          +--------------------+  new server  +--------------------------+
 |                  |          |                    |------------->|                          |
 User(You!)         |--ssh----->    lobbyboy        |              |  DigitalOcean            |
 |                  |          |                    |------------->|  (or any other providers |
 +------------------+          +--------------------+     ssh      +--------------------------+

Key Features

  • talks in SSH2 protocol, no need to install any software of configs for client-side, just ssh to lobbyboy!
  • extendable provider: just implement 3 methods, then lobbyboy can work with any provider!
  • destroy the server when you no longer needed.
  • manage ssh keys for you

Installation

Install via pip:

pip install lobbyboy

Then generate config file:

lobbyboy-config-example > config.toml
# Edit your config before running!

Run server

lobbyboy-server -c config.toml

Providers

// TBD

Builtin Providers

Lobbyboy current support two Providers:

  • DigitalOcean
  • Vagrant (Need vagrant and virtualbox to be installed)

Different Providers support different configs, please see the example config for more detail.

Write Your Own Providers

Providers are VPS vendors, by writing new providers, lobbyboy can work with any VPS vendors.

To make a new Provider work, you need to extend base class `lobbyboy.provider.BaseProvider``, implement 3 methods:

  def new_server(self, channel):
      """
      Args:
          channel: paramiko channel

      Returns:
          created_server_id: unique id from provision
          created_server_host: server's ip or domain address
      """
      pass

  def destroy_server(self, server_id, server_ip, channel):
      """
      Args:
          channel: Note that the channel can be None.
                    If called from server_killer, channel will be None.
                    if called when user logout from server, channel is active.
      """
      pass

  def ssh_server_command(self, server_id, server_ip):
      """
      Args:
          server_id: the server ssh to, which is returned by you from ``new_server``
          server_ip: ip or domain name.
      Returns:
          list: a command in list format, for later to run exec.
      """
      pass

Then add your Provider to your config file.

Those 3 configs are obligatory, as lobbyboy has to know when should he destroy your spare servers. You can add more configs, and read them from self.provider_config from code, just remember to add docs about it :)

[provider.<your provider name>]
loadmodule = "lobbyboy.contrib.provider.vagrant::VagrantProvider"
min_life_to_live = "1h"
bill_time_unit = "1h"

FAQ

Q: Can I use lobbyboy as a proxy, like adding it to my ProxyCommand in ssh config?

A: No. Lobbyboy works like a reverse proxy, meaning to say, for ssh client, it just like a ssh server(sshd maybe), ssh client get a shell from lobbyboy, and doesn't know if it is local shell or it is a nested shell which runs another ssh. (but you know it, right? :D )

Comments
  • LBConfigProvider is inflexible

    LBConfigProvider is inflexible

    My original idea for the provider is that:

    A provider can be defined in a single file, a provider's exceptions, implementations, and config definitions, all can be defined in a single file.

    So that it would be easy to maintain. And also, a provider can be implemented outside lobbyboy's codebase. (let's say, one can upload his provider to pypi, and other users can install them via pip install lobbyboy-aws-ec2-provider, then lobbyboy can load it.

    And one can add custom configs to his provider.

    The current problem is, we have to update the LBConfigProvider if one wants to add more fields to his provider config. It is not possible to do so without updating lobbyboy's source code.

    opened by laixintao 7
  • UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 4093-4094: unexpected end of data

    UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 4093-4094: unexpected end of data

    CRI [20211208-07:42:34.995] thr=140672309044992 lobbyboy.socket_handle:254: *** Socket thread error.                                                                                                                                                                                                                          Traceback (most recent call last):
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 247, in run
        self.user_using(server, proxy_subprocess)
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 188, in user_using
        send_to_channel(self.channel, os.read(master_fd, 10240).decode(), suffix="")
    UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 4093-4094: unexpected end of data
    DEB [20211208-07:42:35.003] thr=140672300570368 paramiko.transport:1819: EOF in transport thread
    
    opened by messense 5
  • Initialization Problems.

    Initialization Problems.

    I am really interested in your idea of lobbyboy. But I know nothing about python.

    So here is my problems during initialization, can you help me to figure it out?

    1. I configure my own data_dir but service start with an error. obbyboy.utils: Error when reading available_servers.json, [Errno 2] No such file or directory: '~/Software/lobbyboy/data/available_servers.json' Even I create the available_servers.json file, it does not work. Until I write {} to the file. But I'm not sure if this is ok for the lobbyboy service
    2. When I ssh with ssh [email protected] -p 12200, the logs shows an error below:
    ERR [20211119-17:40:41.664] thr=123145413386240 paramiko.transport: Unknown exception: too many values to unpack (expected 2)
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport: Traceback (most recent call last):
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:   File "/usr/local/lib/python3.9/site-packages/paramiko/transport.py", line 2109, in run
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:     handler(self.auth_handler, m)
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:   File "/usr/local/lib/python3.9/site-packages/paramiko/auth_handler.py", line 525, in _parse_userauth_request
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:     result = self.transport.server_object.check_auth_publickey(
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:   File "/usr/local/lib/python3.9/site-packages/lobbyboy/server.py", line 103, in check_auth_publickey
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:     _, _key_pub = line.split(" ", 2)
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport: ValueError: too many values to unpack (expected 2)
    

    I did some research but it's really difficult for me to understand python.

    So if you can give me a hand, that's would be nice.

    Thank you.

    opened by kimichen13 4
  • Use pre-commit as the linter

    Use pre-commit as the linter

    Close #48

    Install the commit hooks in dev:

    poetry run pre-commit install
    

    Run in CI: I recommend using pre-commit ci, it can upgrade the hooks and commit linter changes automatically. It is also blazing fast, e2e run time is < 10s

    BTW, the code is changed by the linter so this is better to merge after #55, otherwise there will be lots of conflicts

    opened by frostming 3
  • Recommended use pre-commit to make code style consistency

    Recommended use pre-commit to make code style consistency

    @laixintao @messense @frostming

    Recommended use pre-commit to make code style consistency, especially in the case of multi-person cooperation, what do you think?

    opened by luxiaba 3
  • Bump paramiko from 2.8.1 to 2.10.1

    Bump paramiko from 2.8.1 to 2.10.1

    Bumps paramiko from 2.8.1 to 2.10.1.

    Commits
    • 286bd9f Cut 2.10.1
    • 4c491e2 Fix CVE re: PKey.write_private_key chmod race
    • aa3cc6f Cut 2.10.0
    • e50e19f Fix up changelog entry with real links
    • 02ad67e Helps to actually leverage your mocked system calls
    • 29d7bf4 Clearly our agent stuff is not fully tested yet...
    • 5fcb8da OpenSSH docs state %C should also work in IdentityFile and Match exec
    • 1bf3dce Changelog enhancement
    • f6342fc Prettify, add %C as acceptable controlpath token, mock gethostname
    • 3f3451f Add to changelog
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • OSError: [Errno 9] Bad file descriptor

    OSError: [Errno 9] Bad file descriptor

    INF [20211208-07:44:51.056] thr=140672309044992 lobbyboy.utils:127: user choose 1 for option There are 1 available servers:
    INF [20211208-07:44:51.056] thr=140672309044992 lobbyboy.socket_handle:79: user choose server input=1.
    DEB [20211208-07:44:51.057] thr=140672309044992 lobbyboy.contrib.provider.ignite:86: get ssh to server command for ignite: ['cd dev_datadir/ignite/2021-12-08-0739 && ignite ssh 2021-12-08-0739']
    INF [20211208-07:44:51.057] thr=140672309044992 lobbyboy.socket_handle:103: ssh to server 2021-12-08-0739 127.0.0.1: cd dev_datadir/ignite/2021-12-08-0739 && ignite ssh 2021-12-08-0739
    INF [20211208-07:44:51.062] thr=140672309044992 lobbyboy.socket_handle:176: proxy subprocess created, pid=71892
    CRI [20211208-07:44:51.987] thr=140672309044992 lobbyboy.socket_handle:254: *** Socket thread error.
    Traceback (most recent call last):
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 252, in run
        self.cleanup(t, meta=lb_server, check_destroy=True)
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 194, in cleanup
        self.remove_server_session(t, meta.server_name)
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 223, in remove_server_session
        active_session[server_name] = list(filter(lambda x: x.getpeername() != peer_name, sessions))
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 223, in <lambda>
        active_session[server_name] = list(filter(lambda x: x.getpeername() != peer_name, sessions))
      File "/root/.cache/pypoetry/virtualenvs/lobbyboy-P-fHd6h2-py3.8/lib/python3.8/site-packages/paramiko/transport.py", line 1787, in getpeername
        return gp()
    OSError: [Errno 9] Bad file descriptor
    
    opened by messense 1
  • tests for need_destroy

    tests for need_destroy

    somehow some of my instance didn't get destroyed, I suspect there are some bugs in https://github.com/lobbyboy-ssh/lobbyboy/blob/41bba07e856e6ce31f7164232723e7440a469f18/lobbyboy/server_killer.py#L42

    but I didn't find any, let me add test cases later.

    opened by laixintao 3
  • Security check when lobbyboy server starts.

    Security check when lobbyboy server starts.

    Need to check:

    • user started lobbyboy with default username/password?
    • with the ssh host key https://github.com/lobbyboy-ssh/lobbyboy/blob/main/dev_datadir/ssh_host_rsa_key in git repo (middle man attack)

    If yes, then need to print warning information to stderr.

    related discussion: https://github.com/lobbyboy-ssh/lobbyboy/pull/32

    opened by laixintao 0
  • vultr provider support

    vultr provider support

    i want to add vultr provider and i found python-vultr, but seems that it hasn't been updated for a long time, and there are fatal bugs that can't be used normally, i write a new one pyvultr, can we use this to support vultr provider?

    opened by luxiaba 2
Releases(v0.4.0)
  • v0.4.0(Dec 11, 2021)

    What's Changed

    • fix typo of exmaple config file name. by @laixintao in https://github.com/lobbyboy-ssh/lobbyboy/pull/25
    • Fix typos and add codespell check to CI by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/26
    • Fix the path of example config.toml by @frostming in https://github.com/lobbyboy-ssh/lobbyboy/pull/28
    • Support docker container (based on footloose) by @laixintao in https://github.com/lobbyboy-ssh/lobbyboy/pull/30
    • ci: fix unittest running branch master-> main by @laixintao in https://github.com/lobbyboy-ssh/lobbyboy/pull/36
    • Add instructions to run in docker by @frostming in https://github.com/lobbyboy-ssh/lobbyboy/pull/32
    • Add ignite provider by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/37
    • Revamp provider config by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/41
    • Fix send_to_channel by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/44
    • Add support for enable/disable provider in config by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/45
    • Add a test case for load_config by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/49
    • Add a test case for load_providers by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/50
    • Add coverage report to CI by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/51
    • update readme: new architecture picture, update new providers. by @laixintao in https://github.com/lobbyboy-ssh/lobbyboy/pull/46
    • Define a default provider if there is only one provider by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/47

    New Contributors

    • @messense made their first contribution in https://github.com/lobbyboy-ssh/lobbyboy/pull/26
    • @frostming made their first contribution in https://github.com/lobbyboy-ssh/lobbyboy/pull/28

    Full Changelog: https://github.com/lobbyboy-ssh/lobbyboy/compare/v0.3.0...v0.4.0

    Source code(tar.gz)
    Source code(zip)
Owner
I love system administration as much as I love programming.
This project shows how to serve an TF based image classification model as a web service with TFServing, Docker, and Kubernetes(GKE).

Deploying ML models with CPU based TFServing, Docker, and Kubernetes By: Chansung Park and Sayak Paul This project shows how to serve a TensorFlow ima

Chansung Park 104 Dec 28, 2022
DataOps framework for Machine Learning projects.

Noronha DataOps Noronha is a Python framework designed to help you orchestrate and manage ML projects life-cycle. It hosts Machine Learning models ins

52 Oct 30, 2022
Create pinned requirements.txt inside a Docker image using pip-tools

Pin your Python dependencies! pin-requirements.py is a script that lets you pin your Python dependencies inside a Docker container. Pinning your depen

4 Aug 18, 2022
Hackergame nc 类题目的 Docker 容器资源限制、动态 flag、网页终端

Hackergame nc 类题目的 Docker 容器资源限制、动态 flag、网页终端 快速入门 配置证书 证书用于验证用户 Token。请确保这里的证书文件(cert.pem)与 Hackergame 平台 配置的证书相同,这样 Hackergame 平台为每个用户生成的 Token 才可以通

USTC Hackergame 68 Nov 09, 2022
gunicorn 'Green Unicorn' is a WSGI HTTP Server for UNIX, fast clients and sleepy applications.

Gunicorn Gunicorn 'Green Unicorn' is a Python WSGI HTTP Server for UNIX. It's a pre-fork worker model ported from Ruby's Unicorn project. The Gunicorn

Benoit Chesneau 8.7k Jan 08, 2023
Pulumi - Developer-First Infrastructure as Code. Your Cloud, Your Language, Your Way 🚀

Pulumi's Infrastructure as Code SDK is the easiest way to create and deploy cloud software that use containers, serverless functions, hosted services,

Pulumi 14.7k Jan 08, 2023
Build and Push docker image in Python (luigi + docker-py)

Docker build images workflow in Python Since docker hub stopped building images for free accounts, I've been looking for another way to do it. I could

Fabien D. 2 Dec 15, 2022
A tool to convert AWS EC2 instances back and forth between On-Demand and Spot billing models.

ec2-spot-converter This tool converts existing AWS EC2 instances back and forth between On-Demand and 'persistent' Spot billing models while preservin

jcjorel 152 Dec 29, 2022
A lobby boy will create a VPS server when you need one, and destroy it after using it.

Lobbyboy What is a lobby boy? A lobby boy is completely invisible, yet always in sight. A lobby boy remembers what people hate. A lobby boy anticipate

226 Dec 29, 2022
Honcho: a python clone of Foreman. For managing Procfile-based applications.

___ ___ ___ ___ ___ ___ /\__\ /\ \ /\__\ /\ \ /\__\ /\

Nick Stenning 1.5k Jan 03, 2023
Oncall is a calendar tool designed for scheduling and managing on-call shifts. It can be used as source of dynamic ownership info for paging systems like http://iris.claims.

Oncall See admin docs for information on how to run and manage Oncall. Development setup Prerequisites Debian/Ubuntu - sudo apt-get install libsasl2-d

LinkedIn 928 Dec 22, 2022
A Kubernetes operator that creates UptimeRobot monitors for your ingresses

This operator automatically creates uptime monitors at UptimeRobot for your Kubernetes Ingress resources. This allows you to easily integrate uptime monitoring of your services into your Kubernetes d

Max 49 Dec 14, 2022
Utilitaire de contrôle de Kubernetes

Utilitaire de contrôle de Kubernetes ** What is this ??? ** Every time we use a word in English our manager tells us to use the French translation of

Théophane Vié 9 Dec 03, 2022
sysctl/sysfs settings on a fly for Kubernetes Cluster. No restarts are required for clusters and nodes.

SysBindings Daemon Little toolkit for control the sysctl/sysfs bindings on Kubernetes Cluster on the fly and without unnecessary restarts of cluster o

Wallarm 19 May 06, 2022
Chef-like functionality for Fabric

/ / ___ ___ ___ ___ | | )| |___ | | )|___) |__ |__/ | __/ | | / |__ -- Chef-like functionality for Fabric About Fabric i

Sébastien Pierre 1.3k Dec 21, 2022
Let's learn how to build, release and operate your containerized applications to Amazon ECS and AWS Fargate using AWS Copilot.

🚀 Welcome to AWS Copilot Workshop In this workshop, you'll learn how to build, release and operate your containerised applications to Amazon ECS and

Donnie Prakoso 15 Jul 14, 2022
Push Container Image To Docker Registry In Python

push-container-image-to-docker-registry 概要 push-container-image-to-docker-registry は、エッジコンピューティング環境において、特定のエッジ端末上の Private Docker Registry に特定のコンテナイメー

Latona, Inc. 3 Nov 04, 2021
Some automation scripts to setup a deployable development database server (with docker).

Postgres-Docker Database Initializer This is a simple automation script that will create a Docker Postgres database with a custom username, password,

Pysogge 1 Nov 11, 2021
SSH tunnels to remote server.

Author: Pahaz Repo: https://github.com/pahaz/sshtunnel/ Inspired by https://github.com/jmagnusson/bgtunnel, which doesn't work on Windows. See also: h

Pavel White 1k Dec 28, 2022
Automatically capture your Ookla Speedtest metrics and display them in a Grafana dashboard

Speedtest All-In-One Automatically capture your Ookla Speedtest metrics and display them in a Grafana dashboard. Getting Started About This Code This

Aaron Melton 2 Feb 22, 2022