The elegance of Airflow + the power of AWS

Overview

Orkestra

The elegance of Airflow + the power of AWS

Docs Codecov

PyPI PyPI - Downloads PyPI - License PyPI - Python Version GitHub issues Mentioned in Awesome CDK

examples/hello_orkestra.py

list: logger.info(item.dict()) return [item.dict()] * 10 @compose(model=Item, is_map_job=True, **default_args) def double_price(item: Item, context): item.price = item.price * 2 return item.dict() @compose(**default_args) def assert_false(event, context): assert False @compose(**default_args) def do_nothing(event, context): logger.info({"doing": "nothing"}) @compose(**default_args) def say_hello(event, context): return "hello, world" @compose(**default_args) def say_goodbye(event, context): return "goodbye" @compose(**default_args) def random_int(event, context): return random.randrange(100) @compose(**default_args) def random_float(event, context): return float(random_int(event, context)) dag() ">
import random
from typing import *
from uuid import uuid4

from aws_lambda_powertools import Logger, Tracer
from pydantic import BaseModel

from orkestra import compose
from orkestra.interfaces import Duration


def dag():
    (
        generate_item
        >> add_price
        >> copy_item
        >> double_price
        >> (do_nothing, assert_false)
        >> say_hello
        >> [random_int, random_float]
        >> say_goodbye
    )


class Item(BaseModel):
    id: str
    name: str
    price: Optional[float] = None

    @classmethod
    def random(cls):
        return cls(
            id=str(uuid4()),
            name=random.choice(
                [
                    "potato",
                    "moon rock",
                    "hat",
                ]
            ),
        )


logger = Logger()

tracer = Tracer()


default_args = dict(
    enable_powertools=True,
    timeout=Duration.seconds(6),
)


@compose(**default_args)
def generate_item(event, context):
    logger.info("generating random item")
    item = Item.random().dict()
    logger.info(item)
    tracer.put_metadata("GenerateItem", "SUCCESS")
    return item


@compose(model=Item, **default_args)
def add_price(item: Item, context):
    price = 3.14
    logger.info(
        "adding price to item",
        extra={
            "item": item.dict(),
            "price": price,
        },
    )
    item.price = price
    return item.dict()


@compose(model=Item, **default_args)
def copy_item(item: Item, context) -> list:
    logger.info(item.dict())
    return [item.dict()] * 10


@compose(model=Item, is_map_job=True, **default_args)
def double_price(item: Item, context):
    item.price = item.price * 2
    return item.dict()


@compose(**default_args)
def assert_false(event, context):
    assert False


@compose(**default_args)
def do_nothing(event, context):
    logger.info({"doing": "nothing"})


@compose(**default_args)
def say_hello(event, context):
    return "hello, world"


@compose(**default_args)
def say_goodbye(event, context):
    return "goodbye"


@compose(**default_args)
def random_int(event, context):
    return random.randrange(100)


@compose(**default_args)
def random_float(event, context):
    return float(random_int(event, context))


dag()

app.py

#!/usr/bin/env python3
from aws_cdk import core as cdk

from examples.hello_orkestra import generate_item


class HelloOrkestra(cdk.Stack):
    def __init__(self, scope, id, **kwargs):

        super().__init__(scope, id, **kwargs)

        generate_item.schedule(
            self,
            expression="rate(5 minutes)",
            state_machine_name="hello_orkestra",
        )


app = cdk.App()


app.synth()

state machine

Owner
Stephan Fitzpatrick
“Talk is cheap. Show me the code.”
Stephan Fitzpatrick
GitHub Actions Docker training

GitHub-Actions-Docker-training Training exercise repository for GitHub Actions using a docker base. This repository should be cloned and used for trai

GitHub School 1 Jan 21, 2022
A discord nitro generator written in python

VerseGenerator A discord nitro generator written in python Usage ・Fork the repo ・Clone it to replit ・Install the required packages and run it ・Input t

NotDrakezz 4 Nov 13, 2021
Zalo AI challenge 2021 5K-Compliance

Zalo AI challenge 2021 5K-Compliance Prepare data: Dữ liệu của ban tổ chức cung

Nguyễn nhật hoàng 32 Nov 21, 2022
Telegram group manager moderen and simple.

Upin Robot A Advanced Powerful, Smart And Intelligent Group Management Bot With New And Powerful Features ... Written with Pyrogram and Telethon... If

Muhammad Nawawi 3 Dec 23, 2021
PyDottie is a version of Dottie.js written in Python 3.

PyDottie is a version of Dottie.js written in Python 3.

Jose Noriega 2 Nov 21, 2021
Non official, but friendly QvaPay library for the Python language.

Python SDK for the QvaPay API Non official, but friendly QvaPay library for the Python language. Setup You can install this package by using the pip t

Carlos Lugones 17 Nov 25, 2022
A simple program to display current playing from Spotify app on your desktop

WallSpot A simple program to display current playing from Spotify app on your desktop How to Use: Linux: Currently Supports GNOME and KDE. If you want

Nannan 4 Feb 19, 2022
Telegram bot for stream music or video on telegram

Anonymous VC Bot + Stream Bot Telegram bot for stream music or video on telegram, powered by PyTgCalls and Pyrogram Features Playlist features Multi L

Anonymous Boy 111 Oct 04, 2022
Client library for accessing IQM quantum computers

IQM Client Client-side library for connecting to an IQM quantum computer. Installation IQM client is not intended to be used directly by human users.

IQM 10 Dec 21, 2022
Python script to decode the EU Covid-19 vaccine certificate

vacdec Python script to decode the EU Covid-19 vaccine certificate This script takes an image with a QR code of a vaccine certificate as the parameter

Hanno Böck 244 Nov 30, 2022
A frame to create discord bots (for myself) that uses cogs, JSON, activities, and more.

dpy-frame A frame to create discord bots (for myself) that uses cogs, JSON, activities, and more. NOTE: Documentation is incomplete, so please wait un

Apple Discord 1 Nov 06, 2021
With this simple app you can customize your presence in Discord.

Discord Rich Presence This a simple console app which can customize your Discord Presence easily, with only one config file! Setup Go to Discord Devel

Mai 1 Jan 06, 2022
Photogrammetry Web API

OpenScanCloud Photogrammetry Web API Overview / Outline: The OpenScan Cloud is intended to be a decentralized, open and free photogrammetry web API. T

Thomas 86 Jan 05, 2023
Send automated wishes to your contacts at scheduled time through WhatsApp. Written for Raspberry pi.

Whatsapp Automated Wishes Helps to send automated wishes to your contacts in Whatsapp at scheduled time using pywhatkit . Written for Raspberry pi. Wi

Uthayamurthy 2 Dec 13, 2022
Python interface to the LinkedIn API

Python LinkedIn Python interface to the LinkedIn API This library provides a pure Python interface to the LinkedIn Profile, Group, Company, Jobs, Sear

ozgur 844 Dec 27, 2022
Account Profiles Dumper for Fortnite.

Fortnite Profile Dumper This program allows you to dump your Fortnite account profiles. How to use it? After starting the FortniteProfileDumper.py, yo

PRO100KatYT 12 Jul 28, 2022
QuickStart specific rules for cfn-python-lint

AWS Quick Start cfn-lint rules This repo provides CloudFormation linting rules specific to AWS Quick Start guidelines, for more information see the Co

AWS Quick Start 12 Jul 30, 2022
Fetch tracking numbers of Amazon orders, for the ease of the logistics.

Amazon-Tracking-Number Fetch tracking numbers of Amazon orders, for the ease of the logistics. Read Me First (How to use this code): Get Amazon "Items

Tony Yao 1 Nov 02, 2021
NiceHash Python Library and Command Line Rest API

NiceHash Python Library and Command Line Rest API Requirements / Modules pip install requests Required data and where to get it Following data is nee

Ashlin Darius Govindasamy 2 Jan 02, 2022
A bot to share Facebook posts.

bot_share_facebook a bot to share Facebook posts. install & clone untuk menjalankan anda bisa melalui terminal contohnya termux, cmd, dan terminal lai

Muhammad Latif Harkat 7 Dec 07, 2022