A repository containing a short tutorial for Docker (with Python).

Overview

Docker Tutorial for IFT 6758 Lab

image

In this repository, we examine the advtanges of virtualization, what Docker is and how we can deploy simple programs on Docker.

Firstly, we will start by listing some advantages of virtualization. We can discuss them as follows:

  • Minimize hardware costs (Multiple virtual servers on one hardware unit).
  • Easily for HA (High Availability) And Disaster Recovery (DR).
  • Easier to save money and save energy, could be potentially greener.

Problems with Virtualization:

  • Hard to maintain each OS, installation, HyperVisor and host OS compatibility issues, and HyperVisor - container issues.
  • Costly and might take IT/Programming effort.
  • We might need a high storage overhead, even lightweight OSs with all tools installed can bulk up to a high size.

image

Solution?

  • Use containers!

Container definition:

  • is a runnable instance (or sandbox) of an image.
  • can be run on local machines, virtual machines or deployed to the cloud.
  • is portable (can be run on any OS).
  • Containers are isolated from each other and run their own software, binaries, and configurations.

Methodology:

  • These applications share a parent application and its overhead.
  • These apps can run separately, and we can avoid multithreading issues, deadlock issues and other issues with RPCs that may arise.
  • They can't access each other's resources, and they are like apartments in a complex.

Container Architecture:

image

  • Multiple containers run on the same machine virtual / physical machine.
  • All containers share the same OS and Kernel.
  • We can micro-manage processes, and have a higher level and more fine-grained access.
  • We can stop processes using the power of the host OS, and we can even save/move states as we need.
  • We can run containers inside VM! It's interesting as we can combine the power of both if we have a very specialized applicaiton.
  • Higher security as we are working with a very simple isolated block of code.

Compute Comparision:

image

Docker:

Standard definition:

"An open-source project that automates the deployment of software applications inside containers by providing an additional layer of abstraction and automation of OS-level virtualization on Linux."

image

  • Does exactly what we need, and it assists in efficient containerization.
  • Free and Open Source! (now under Moby)
  • Helps in packaing source code, libraries, dependencies, volumes, routes, etc for production.
  • The name Docker had roots from the word "Dock Worker", assists in effective container management.
  • Containers are built from images and saved as images.
  • Each image is identified by a UUID/256 Bit Hash.
  • Each Docker Image has several tags, and we can help in effective versioning.
  • Dockers can be secured by SSH, and additional network security techniques.

image

image

Dockerfile:

  • Dictates the program installation, setup, packaging, instructions, port exposition, etc.
  • We can run a container, list a container, version them, delete them, etc.

Sample Docker Image (for getting started):

docker run -d -p 80:80 docker/getting-started

Tools:

  • Docker Compose: It us used for maintaining and running multi-container Docker applications. We use YAML (mark-up) to dictate the format, dependencies and other nuances of a project. It can aid in building and scaling containers.
  • Docker Volume: We can store and manage separate volumes of code, that avoid deletion when the container is stopped.

Glossary:

  • Docker Daemon: The background service running on the host that manages building, running and distributing Docker containers. The daemon is the process that runs in the operating system which clients talk to.
  • Docker Client: The command line tool that allows the user to interact with the daemon. More generally, there can be other forms of clients too - such as Kitematic which provide a GUI to the users.
  • Docker Hub: A registry of Docker images. You can think of the registry as a directory of all available Docker images. If required, one can host their own Docker registries and can use them for pulling images.
  • Docker Container Image: When running a container, it uses an isolated filesystem. This custom filesystem is provided by a container image. Since the image contains the container's filesystem, it must contain everything needed to run an application - all dependencies, configuration, scripts, binaries, and so on. The image also contains other configuration for the container, such as environment variables, a default command to run, and other metadata.

Folder instructions:

  • Tutorial 1: Demonstrates a basic Docker program.
  • Tutorial 2: Demonstrates simple Redis counter example with compose.
  • Tutorial 3: Docker + NGINX Reverse Proxy + Vanilla MongoDB/NoSQL.
  • Tutorial 4: Same as Tutorial 1, but updated WSGI (Web Server Gateway Interface), instead of a default Flask development WSGI.

References:

  1. https://runnable.com/docker/python/docker-compose-with-flask-apps
  2. https://github.com/docker/awesome-compose
  3. https://docs.docker.com/language/python/build-images/
  4. https://www.cse.wustl.edu/~jain/cse570-18/ftp/m_21cdk.pdf
  5. https://en.wikipedia.org/wiki/Docker_(software)
  6. https://docs.docker.com/compose/
  7. https://docker-curriculum.com/
Owner
Arka Mukherjee
Master's Student in ML and CS @ Mila, University of Montreal
Arka Mukherjee
Micro Data Lake based on Docker Compose

Micro Data Lake based on Docker Compose This is the implementation of a Minimum Data Lake

Abel Coronado 15 Jan 07, 2023
Ganeti is a virtual machine cluster management tool built on top of existing virtualization technologies such as Xen or KVM and other open source software.

Ganeti 3.0 =========== For installation instructions, read the INSTALL and the doc/install.rst files. For a brief introduction, read the ganeti(7) m

395 Jan 04, 2023
Python job scheduling for humans.

schedule Python job scheduling for humans. Run Python functions (or any other callable) periodically using a friendly syntax. A simple to use API for

Dan Bader 10.4k Jan 02, 2023
A declarative Kubeflow Management Tool inspired by Terraform

🍭 KRSH is Alpha version, so many bugs can be reported. If you find a bug, please write an Issue and grow the project together! A declarative Kubeflow

Riiid! 128 Oct 18, 2022
Quick & dirty controller to schedule Kubernetes Jobs later (once)

K8s Jobber Operator Quickly implemented Kubernetes controller to enable scheduling of Jobs at a later time. Usage: To schedule a Job later, Set .spec.

Jukka Väisänen 2 Feb 11, 2022
Google Kubernetes Engine (GKE) with a Snyk Kubernetes controller installed/configured for Snyk App

Google Kubernetes Engine (GKE) with a Snyk Kubernetes controller installed/configured for Snyk App This example provisions a Google Kubernetes Engine

Pas Apicella 2 Feb 09, 2022
A lobby boy will create a VPS server when you need one, and destroy it after using it.

Lobbyboy What is a lobby boy? A lobby boy is completely invisible, yet always in sight. A lobby boy remembers what people hate. A lobby boy anticipate

226 Dec 29, 2022
Copy a Kubernetes pod and run commands in its environment

copypod Utility for copying a running Kubernetes pod so you can run commands in a copy of its environment, without worrying about it the pod potential

Memrise 4 Apr 08, 2022
Big data on k8s

# microsoft azure # https://docs.microsoft.com/en-us/cli/azure/install-azure-cli az account set --subscription [] az aks get-credentials --resource-g

Luan Moreno 22 Dec 24, 2022
MicroK8s is a small, fast, single-package Kubernetes for developers, IoT and edge.

MicroK8s The smallest, fastest Kubernetes Single-package fully conformant lightweight Kubernetes that works on 42 flavours of Linux. Perfect for: Deve

Ubuntu 7.1k Jan 08, 2023
Dockerized iCloud drive

iCloud-drive-docker is a simple iCloud drive client in Docker environment. It uses pyiCloud python library to interact with iCloud

Mandar Patil 376 Jan 01, 2023
framework providing automatic constructions of vulnerable infrastructures

中文 | English 1 Introduction Metarget = meta- + target, a framework providing automatic constructions of vulnerable infrastructures, used to deploy sim

rambolized 685 Dec 28, 2022
Self-hosted, easily-deployable monitoring and alerts service - like a lightweight PagerDuty

Cabot Maintainers wanted Cabot is stable and used by hundreds of companies and individuals in production, but it is not actively maintained. We would

Arachnys 5.4k Dec 23, 2022
A system for managing CI data for Mozilla projects

Treeherder Description Treeherder is a reporting dashboard for Mozilla checkins. It allows users to see the results of automatic builds and their resp

Mozilla 235 Dec 22, 2022
Manage your SSH like a boss.

--- storm is a command line tool to manage your ssh connections. features adding, editing, deleting, listing, searching across your SSHConfig. command

Emre Yılmaz 3.9k Jan 03, 2023
Glances an Eye on your system. A top/htop alternative for GNU/Linux, BSD, Mac OS and Windows operating systems.

Glances - An eye on your system Summary Glances is a cross-platform monitoring tool which aims to present a large amount of monitoring information thr

Nicolas Hennion 22k Jan 08, 2023
pyinfra automates infrastructure super fast at massive scale. It can be used for ad-hoc command execution, service deployment, configuration management and more.

pyinfra automates/provisions/manages/deploys infrastructure super fast at massive scale. It can be used for ad-hoc command execution, service deployme

Nick Barrett 2.1k Dec 29, 2022
Bitnami Docker Image for Python using snapshots for the system packages repositories

Python Snapshot packaged by Bitnami What is Python Snapshot? Python is a programming language that lets you work quickly and integrate systems more ef

Bitnami 1 Jan 13, 2022
Build and Push docker image in Python (luigi + docker-py)

Docker build images workflow in Python Since docker hub stopped building images for free accounts, I've been looking for another way to do it. I could

Fabien D. 2 Dec 15, 2022
Learning and experimenting with Kubernetes

Kubernetes Experiments This repository contains code that I'm using to learn and experiment with Kubernetes. 1. Environment setup minikube kubectl doc

Richard To 10 Dec 02, 2022