A Python package which supports global logfmt formatted logging.

Overview

Python Logfmter

pre-commit test

A Python package which supports global logfmt formatted logging.

Install

$ pip install logfmter

Usage

Before integrating this library, you should be familiar with Python's logging functionality. I recommend reading the Basic Logging Tutorial.

This package exposes a single Logfmter class that can be integrated into the standard library logging system similar to any logging.Formatter.

The provided formatter will logfmt encode all logs. Key value pairs are provided via the extra keyword argument or by passing a dictionary as the log message.

Basic

import logging
from logfmter import Logfmter

handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(Logfmter())

logger = logging.getLogger()
logger.addHandler(handler)

logger.error("hello", extra={"alpha": 1}) # at=ERROR msg=hello alpha=1
logger.error({"token": "Hello, World!"}) # at=ERROR token="Hello, World!"

Customize

You can subclass the formatter to change its behavior.

import logging
from logfmter import Logfmter


class CustomLogfmter(Logfmter):
    """
    Provide a custom logfmt formatter which formats
    booleans as "yes" or "no" strings.
    """

    @classmethod
    def format_value(cls, value):
        if isinstance(value, bool):
            return "yes" if value else "no"

	return super().format_value(value)

handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(CustomLogfmter())

logger = logging.getLogger()
logger.addHandler(handler)

logger.error({"example": True}) # at=ERROR example=yes

Development

Required Software

Refer to the links provided below to install these development dependencies:

Getting Started

Setup

$ <runtimes.txt xargs -n 1 pyenv install -s
$ direnv allow
$ pip install -r requirements/dev.txt
$ pre-commit install
$ pip install -e .

Tests

Run the test suite against the active python environment.

$ pytest

Run the test suite against the active python environment and watch the codebase for any changes.

$ ptw

Run the test suite against all supported python versions.

$ tox

Publishing

Build

$ python -m build

Upload

$ twine upload dist/*
Comments
  • Question: How to include default / reserved attributes in the log message?

    Question: How to include default / reserved attributes in the log message?

    Thank you for making this library available. I have been looking at the tests and docs but could not find out how I can include some of the keys you defined as "RESERVED" in my message, such as asctime. I would like to say::

    >>> logger.info("hello")
    "asctime=2022-04-20 msg='hello'"
    

    without having to pass asctime in as an extra. I have tried the below in my dictconfig but did not work::

            'logfmt': {
                'format': 'on=%(asctime)s level=%(levelname)s module=%(name)s msg=%(message)s',
                'class': 'logfmter.Logfmter',
            },
    

    Is this possible?

    opened by meitham 3
  • Include Native Log Record Attributes

    Include Native Log Record Attributes

    We should be able to configure the logger to always include some default keys. For example, a user may want to always include a time or logger key.

    This functionality is seen in the python-json-logger.

    opened by jteppinette 1
  • Add Python 11 Official Support / Testing

    Add Python 11 Official Support / Testing

    I assume this will work on Python 11 just fine, but let's add the Python 11 image to the test scripts and see if there are any improvements that can be made.

    opened by jteppinette 0
  • Update Structure

    Update Structure

    We should move the logfmter directory inside src as suggested by:

    • https://docs.pytest.org/en/7.1.x/explanation/goodpractices.html#choosing-a-test-layout-import-rules
    • https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure%3E

    This is highly recommend and will also support the usage of importlib import mode as recommended here.

    opened by jteppinette 0
  • Exclude Keys

    Exclude Keys

    Some third party libraries may make log calls with keys which we don't want to log. We should be able to exclude these keys from the final log line.

    For example, when overriding uvicorn logging. The color_message key is currently printed (it is used by their custom color formatter). If we could exclude this key, then we could prevent the following output:

    at=INFO msg="Started server process [97819]" color_message="Started server process [%d]"
    at=INFO msg="Waiting for application startup."
    at=INFO msg="Application startup complete."
    at=INFO msg="Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)" color_message="Uvicorn running on %s://%s:%d (Press CTRL+C to quit)"
    
    opened by jteppinette 0
Releases(v0.0.6)
  • v0.0.6(Apr 22, 2022)

  • v0.0.5(Apr 20, 2022)

    • Add support for include native log record attributes in the final log output.
    • Add support for overriding the date format used when formatting the asctime attribute.
    Source code(tar.gz)
    Source code(zip)
  • v0.0.4(Mar 30, 2022)

    • Fix the usage documentation
    • Escape newline characters in all logged values. Previously, you could generate multi-line log statements. This should never be the case.
    • Add support for auto-generating exc_info parameters. If the log record is generated with exc_info, as in when using logging.exception(...), the log message will contain properly formatted exception and traceback information.
    • Add support for logging with an empty message dictionary.
    Source code(tar.gz)
    Source code(zip)
  • v0.0.3(Feb 12, 2022)

  • v0.0.1(Feb 6, 2022)

A simple package that allows you to save inputs & outputs as .log files

wolf_dot_log A simple package that allows you to save inputs & outputs as .log files pip install wolf_dot_log pip3 install wolf_dot_log |Instructions|

Alpwuf 1 Nov 16, 2021
Display tabular data in a visually appealing ASCII table format

PrettyTable Installation Install via pip: python -m pip install -U prettytable Install latest development version: python -m pip install -U git+https

Jazzband 924 Jan 05, 2023
A python library used to interact with webots robocup game web logs

A python library used to interact with webots robocup game web logs

Hamburg Bit-Bots 2 Nov 05, 2021
Json Formatter for the standard python logger

This library is provided to allow standard python logging to output log data as json objects. With JSON we can make our logs more readable by machines and we can stop writing custom parsers for syslo

Zakaria Zajac 1.4k Jan 04, 2023
A Prometheus exporter for monitoring & analyzing Grafana Labs' technical documentation

grafana-docs-exporter A Prometheus exporter for monitoring & analyzing Grafana Labs' technical documentation Here is the public endpoint.

Matt Abrams 5 May 02, 2022
🐑 Syslog Simulator hazır veya kullanıcıların eklediği logları belirtilen adreslere ve port'a seçilen döngüde syslog ile gönderilmesini sağlayan araçtır. | 🇹🇷

syslogsimulator hazır ürün loglarını SIEM veya log toplayıcısına istediğiniz portta belirli sürelerde göndermeyi sağlayan küçük bir araçtır.

Enes Aydın 3 Sep 28, 2021
Discord-Image-Logger - Discord Image Logger With Python

Discord-Image-Logger A exploit I found in discord. Working as of now. Explanatio

111 Dec 31, 2022
Stand-alone parser for User Access Logging from Server 2012 and newer systems

KStrike Stand-alone parser for User Access Logging from Server 2012 and newer systems BriMor Labs KStrike This script will parse data from the User Ac

BriMor Labs 69 Nov 01, 2022
Lazy Profiler is a simple utility to collect CPU, GPU, RAM and GPU Memory stats while the program is running.

lazyprofiler Lazy Profiler is a simple utility to collect CPU, GPU, RAM and GPU Memory stats while the program is running. Installation Use the packag

Shankar Rao Pandala 28 Dec 09, 2022
Structured Logging for Python

structlog makes logging in Python faster, less painful, and more powerful by adding structure to your log entries. It's up to you whether you want str

Hynek Schlawack 2.3k Jan 05, 2023
A watchdog and logger to Discord for hosting ScPrime servers.

ScpDog A watchdog and logger to Discord for hosting ScPrime servers. Designed to work on Linux servers. This is only capable of sending the logs from

Keagan Landfried 3 Jan 10, 2022
Colored terminal output for Python's logging module

coloredlogs: Colored terminal output for Python's logging module The coloredlogs package enables colored terminal output for Python's logging module.

Peter Odding 496 Dec 30, 2022
Python script to scan log files/system for unauthorized access around system

checkLogs Python script to scan log files/system for unauthorized access around Linux systems Table of contents General info Getting started Usage Gen

James Kelly 1 Feb 25, 2022
Espion is a mini-keylogger tool that keeps track of all keys a user presses on his/her keyboard

Espion is a mini-keylogger tool that keeps track of all keys a user presses on his/her keyboard. The details get displayed on the terminal window and also stored in a log file.

Anurag.R.Simha 1 Apr 24, 2022
ScreenshotLogger works just like a keylogger but instead of capturing keystroke,it captures the screen, stores it or sends via email

ScreenshotLogger works just like a keylogger but instead of capturing keystroke,it captures the screen, stores it or sends via email. Scrapeasy is super easy to use and handles everything for you. Ju

Ifechukwudeni Oweh 17 Jul 17, 2022
Pretty and useful exceptions in Python, automatically.

better-exceptions Pretty and more helpful exceptions in Python, automatically. Usage Install better_exceptions via pip: $ pip install better_exception

Qix 4.3k Dec 29, 2022
A colored formatter for the python logging module

Log formatting with colors! colorlog.ColoredFormatter is a formatter for use with Python's logging module that outputs records using terminal colors.

Sam Clements 778 Dec 26, 2022
A Fast, Extensible Progress Bar for Python and CLI

tqdm tqdm derives from the Arabic word taqaddum (تقدّم) which can mean "progress," and is an abbreviation for "I love you so much" in Spanish (te quie

tqdm developers 23.7k Jan 01, 2023
蓝鲸日志平台(BK-LOG)是为解决分布式架构下日志收集、查询困难的一款日志产品,基于业界主流的全文检索引擎

蓝鲸日志平台(BK-LOG)是为解决分布式架构下日志收集、查询困难的一款日志产品,基于业界主流的全文检索引擎,通过蓝鲸智云的专属 Agent 进行日志采集,提供多种场景化的采集、查询功能。

腾讯蓝鲸 102 Dec 22, 2022
Python logging made (stupidly) simple

Loguru is a library which aims to bring enjoyable logging in Python. Did you ever feel lazy about configuring a logger and used print() instead?... I

13.7k Jan 02, 2023