Python codecs extension featuring CLI tools for encoding/decoding anything

Overview

CodExt Tweet

Encode/decode anything.

PyPi Read The Docs Build Status Coverage Status Python Versions Requirements Status Known Vulnerabilities DOI License

This library extends the native codecs library (namely for adding new custom encodings and character mappings) and provides a myriad of new encodings (static or parametrized, like rot or xor), hence its named combining CODecs EXTension.

$ pip install codext
Want to contribute a new codec ? Want to contribute a new macro ?
Check the documentation first
Then PR your new codec
PR your updated version of macros.json

πŸ” Demonstrations

Using CodExt from the command line

Using base tools from the command line

Using the debase command line tool

πŸ’» Usage (main CLI tool) Tweet on codext

$ codext -i test.txt encode dna-1
GTGAGCGGGTATGTGA

$ echo -en "test" | codext encode morse
- . ... -

$ echo -en "test" | codext encode braille
β žβ ‘β Žβ ž

$ echo -en "test" | codext encode base100
πŸ‘«πŸ‘œπŸ‘ͺπŸ‘«

Chaining codecs

$ echo -en "Test string" | codext encode reverse
gnirts tseT

$ echo -en "Test string" | codext encode reverse morse
--. -. .. .-. - ... / - ... . -

$ echo -en "Test string" | codext encode reverse morse dna-2
AGTCAGTCAGTGAGAAAGTCAGTGAGAAAGTGAGTGAGAAAGTGAGTCAGTGAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTTAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTGAGAAAGTC

$ echo -en "Test string" | codext encode reverse morse dna-2 octal
101107124103101107124103101107124107101107101101101107124103101107124107101107101101101107124107101107124107101107101101101107124107101107124103101107124107101107101101101107124103101107101101101107124107101107124107101107124107101107101101101107124124101107101101101107124103101107101101101107124107101107124107101107124107101107101101101107124107101107101101101107124103

$ echo -en "AGTCAGTCAGTGAGAAAGTCAGTGAGAAAGTGAGTGAGAAAGTGAGTCAGTGAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTTAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTGAGAAAGTC" | codext -d dna-2 morse reverse
test string

Using macros

$ codext add-macro my-encoding-chain gzip base63 lzma base64

$ codext list macros
example-macro, my-encoding-chain

$ echo -en "Test string" | codext encode my-encoding-chain
CQQFAF0AAIAAABuTgySPa7WaZC5Sunt6FS0ko71BdrYE8zHqg91qaqadZIR2LafUzpeYDBalvE///ug4AA==

$ codext remove-macro my-encoding-chain

$ codext list macros
example-macro

πŸ’» Usage (base CLI tool) Tweet on debase

$ echo "Test string !" | base122
*.7!ft9οΏ½-f9Γ‚

$ echo "Test string !" | base91 
"ONK;WDZM%Z%xE7L

$ echo "Test string !" | base91 | base85
B2P|BJ6A+nO(j|-cttl%

$ echo "Test string !" | base91 | base85 | base36 | base58-flickr
QVx5tvgjvCAkXaMSuKoQmCnjeCV1YyyR3WErUUErFf

$ echo "Test string !" | base91 | base85 | base36 | base58-flickr | base58-flickr -d | base36 -d | base85 -d | base91 -d
Test string !
$ echo "Test string !" | base91 | base85 | base36 | base58-flickr | debase -m 3
Test string !

$ echo "Test string !" | base91 | base85 | base36 | base58-flickr | debase -f Test
Test string !

πŸ’» Usage (Python)

Getting the list of available codecs:

>> codext.encode("this is a test", "base58-ripple") 'jo9rA2LQwr44eBmZK7E' >>> codext.encode("this is a test", "base58-url") 'JN91Wzkpa1nnDbLyjtf' >>> codecs.encode("this is a test", "base100") 'πŸ‘«πŸ‘ŸπŸ‘ πŸ‘ͺπŸ—πŸ‘ πŸ‘ͺπŸ—πŸ‘˜πŸ—πŸ‘«πŸ‘œπŸ‘ͺπŸ‘«' >>> codecs.decode("πŸ‘«πŸ‘ŸπŸ‘ πŸ‘ͺπŸ—πŸ‘ πŸ‘ͺπŸ—πŸ‘˜πŸ—πŸ‘«πŸ‘œπŸ‘ͺπŸ‘«", "base100") 'this is a test' >>> for i in range(8): print(codext.encode("this is a test", "dna-%d" % (i + 1))) GTGAGCCAGCCGGTATACAAGCCGGTATACAAGCAGACAAGTGAGCGGGTATGTGA CTCACGGACGGCCTATAGAACGGCCTATAGAACGACAGAACTCACGCCCTATCTCA ACAGATTGATTAACGCGTGGATTAACGCGTGGATGAGTGGACAGATAAACGCACAG AGACATTCATTAAGCGCTCCATTAAGCGCTCCATCACTCCAGACATAAAGCGAGAC TCTGTAAGTAATTCGCGAGGTAATTCGCGAGGTAGTGAGGTCTGTATTTCGCTCTG TGTCTAACTAATTGCGCACCTAATTGCGCACCTACTCACCTGTCTATTTGCGTGTC GAGTGCCTGCCGGATATCTTGCCGGATATCTTGCTGTCTTGAGTGCGGGATAGAGT CACTCGGTCGGCCATATGTTCGGCCATATGTTCGTCTGTTCACTCGCCCATACACT >>> codext.decode("GTGAGCCAGCCGGTATACAAGCCGGTATACAAGCAGACAAGTGAGCGGGTATGTGA", "dna-1") 'this is a test' >>> codecs.encode("this is a test", "morse") '- .... .. ... / .. ... / .- / - . ... -' >>> codecs.decode("- .... .. ... / .. ... / .- / - . ... -", "morse") 'this is a test' >>> with open("morse.txt", 'w', encoding="morse") as f: f.write("this is a test") 14 >>> with open("morse.txt",encoding="morse") as f: f.read() 'this is a test' >>> codext.decode(""" = X : x n r y Y y p a ` n | a o h ` g o z """, "whitespace-after+before") 'CSC{not_so_invisible}' >>> print(codext.encode("An example test string", "baudot-tape")) ***.** . * ***.* * . .* * .* . * ** .* ***.** ** .** .* * . * *. * .* * *. * *. * * . * *. * *. * ***. *.* ***.* * .* ">
>>> import codext

>>> codext.list()
['ascii85', 'base85', 'base100', 'base122', ..., 'tomtom', 'dna', 'html', 'markdown', 'url', 'resistor', 'sms', 'whitespace', 'whitespace-after-before']

>>> codext.encode("this is a test", "base58-bitcoin")
'jo91waLQA1NNeBmZKUF'

>>> codext.encode("this is a test", "base58-ripple")
'jo9rA2LQwr44eBmZK7E'

>>> codext.encode("this is a test", "base58-url")
'JN91Wzkpa1nnDbLyjtf'

>>> codecs.encode("this is a test", "base100")
'πŸ‘«πŸ‘ŸπŸ‘ πŸ‘ͺπŸ—πŸ‘ πŸ‘ͺπŸ—πŸ‘˜πŸ—πŸ‘«πŸ‘œπŸ‘ͺπŸ‘«'

>>> codecs.decode("πŸ‘«πŸ‘ŸπŸ‘ πŸ‘ͺπŸ—πŸ‘ πŸ‘ͺπŸ—πŸ‘˜πŸ—πŸ‘«πŸ‘œπŸ‘ͺπŸ‘«", "base100")
'this is a test'

>>> for i in range(8):
        print(codext.encode("this is a test", "dna-%d" % (i + 1)))
GTGAGCCAGCCGGTATACAAGCCGGTATACAAGCAGACAAGTGAGCGGGTATGTGA
CTCACGGACGGCCTATAGAACGGCCTATAGAACGACAGAACTCACGCCCTATCTCA
ACAGATTGATTAACGCGTGGATTAACGCGTGGATGAGTGGACAGATAAACGCACAG
AGACATTCATTAAGCGCTCCATTAAGCGCTCCATCACTCCAGACATAAAGCGAGAC
TCTGTAAGTAATTCGCGAGGTAATTCGCGAGGTAGTGAGGTCTGTATTTCGCTCTG
TGTCTAACTAATTGCGCACCTAATTGCGCACCTACTCACCTGTCTATTTGCGTGTC
GAGTGCCTGCCGGATATCTTGCCGGATATCTTGCTGTCTTGAGTGCGGGATAGAGT
CACTCGGTCGGCCATATGTTCGGCCATATGTTCGTCTGTTCACTCGCCCATACACT
>>> codext.decode("GTGAGCCAGCCGGTATACAAGCCGGTATACAAGCAGACAAGTGAGCGGGTATGTGA", "dna-1")
'this is a test'

>>> codecs.encode("this is a test", "morse")
'- .... .. ... / .. ... / .- / - . ... -'

>>> codecs.decode("- .... .. ... / .. ... / .- / - . ... -", "morse")
'this is a test'

>>> with open("morse.txt", 'w', encoding="morse") as f:
	f.write("this is a test")
14

>>> with open("morse.txt",encoding="morse") as f:
	f.read()
'this is a test'

>>> codext.decode("""
      =            
              X         
   :            
      x         
  n  
    r 
        y   
      Y            
              y        
     p    
         a       
 `          
            n            
          |    
  a          
o    
       h        
          `            
          g               
           o 
   z      """, "whitespace-after+before")
'CSC{not_so_invisible}'

>>> print(codext.encode("An example test string", "baudot-tape"))
***.**
   . *
***.* 
*  .  
   .* 
*  .* 
   . *
** .* 
***.**
** .**
   .* 
*  .  
* *. *
   .* 
* *.  
* *. *
*  .  
* *.  
* *. *
***.  
  *.* 
***.* 
 * .* 

πŸ“ƒ List of codecs

BaseXX

  • ascii85: classical ASCII85 (Python3 only)
  • baseN: see base encodings (incl base32, 36, 45, 58, 62, 63, 64, 91, 100, 122)
  • base-genericN: see base encodings ; supports any possible base

Binary

  • baudot: supports CCITT-1, CCITT-2, EU/FR, ITA1, ITA2, MTK-2 (Python3 only), UK, ...
  • baudot-spaced: variant of baudot ; groups of 5 bits are whitespace-separated
  • baudot-tape: variant of baudot ; outputs a string that looks like a perforated tape
  • bcd: Binary Coded Decimal, encodes characters from their (zero-left-padded) ordinals
  • bcd-extended0: variant of bcd ; encodes characters from their (zero-left-padded) ordinals using prefix bits 0000
  • bcd-extended1: variant of bcd ; encodes characters from their (zero-left-padded) ordinals using prefix bits 1111
  • excess3: uses Excess-3 (aka Stibitz code) binary encoding to convert characters from their ordinals
  • gray: aka reflected binary code
  • manchester: XORes each bit of the input with 01
  • manchester-inverted: variant of manchester ; XORes each bit of the input with 10
  • rotateN: rotates characters by the specified number of bits (N belongs to [1, 7] ; Python 3 only)

Common

  • a1z26: keeps words whitespace-separated and uses a custom character separator
  • cases: set of case-related encodings (including camel-, kebab-, lower-, pascal-, upper-, snake- and swap-case, slugify, capitalize, title)
  • dummy: set of simple encodings (including reverse and word-reverse)
  • octal: dummy octal conversion (converts to 3-digits groups)
  • octal-spaced: variant of octal ; dummy octal conversion, handling whitespace separators
  • ordinal: dummy character ordinals conversion (converts to 3-digits groups)
  • ordinal-spaced: variant of ordinal ; dummy character ordinals conversion, handling whitespace separators

Compression

  • gzip: standard Gzip compression/decompression
  • lz77: compresses the given data with the algorithm of Lempel and Ziv of 1977
  • lz78: compresses the given data with the algorithm of Lempel and Ziv of 1978
  • pkzip_deflate: standard Zip-deflate compression/decompression
  • pkzip_bzip2: standard BZip2 compression/decompression
  • pkzip_lzma: standard LZMA compression/decompression

Cryptography

  • affine: aka Affine Cipher
  • atbash: aka Atbash Cipher
  • bacon: aka Baconian Cipher
  • barbie-N: aka Barbie Typewriter (N belongs to [1, 4])
  • citrix: aka Citrix CTX1 passord encoding
  • rotN: aka Caesar cipher (N belongs to [1,25])
  • scytaleN: encrypts using the number of letters on the rod (N belongs to [1,[)
  • shiftN: shift ordinals (N belongs to [1,255])
  • xorN: XOR with a single byte (N belongs to [1,255])

Languages

  • braille: well-known braille language (Python 3 only)
  • ipsum: aka lorem ipsum
  • leetspeak: based on minimalistic elite speaking rules
  • morse: uses whitespace as a separator
  • navajo: only handles letters (not full words from the Navajo dictionary)
  • radio: aka NATO or radio phonetic alphabet
  • southpark: converts letters to Kenny's language from Southpark (whitespace is also handled)
  • southpark-icase: case insensitive variant of southpark
  • tomtom: similar to morse, using slashes and backslashes

Others

  • dna: implements the 8 rules of DNA sequences (N belongs to [1,8])
  • html: implements entities according to this reference
  • letter-indices: encodes consonants and/or vowels with their corresponding indices
  • markdown: unidirectional encoding from Markdown to HTML
  • url: aka URL encoding

Steganography

  • klopf: aka Klopf code ; Polybius square with trivial alphabetical distribution
  • resistor: aka resistor color codes
  • sms: also called T9 code ; uses "-" as a separator for encoding, "-" or "_" or whitespace for decoding
  • whitespace: replaces bits with whitespaces and tabs
  • whitespace_after_before: variant of whitespace ; encodes characters as new characters with whitespaces before and after according to an equation described in the codec name (e.g. "whitespace+2*after-3*before")

πŸ‘ Supporters

Stargazers repo roster for @dhondta/python-codext

Forkers repo roster for @dhondta/python-codext

Back to top

Comments
  • using Codext guess / Codext rank gives an error

    using Codext guess / Codext rank gives an error

    When I run it on linux and try to use "codext guess" or "codext rank" I get an error message saying:

    # echo "3yZe7d" | codext rank
    Traceback (most recent call last):
      File "/usr/local/bin/codext", line 8, in <module>
        sys.exit(main())
      File "/usr/local/lib/python3.9/dist-packages/codext/__init__.py", line 184, in main
        args.include, args.exclude = __format_list(args.include), __format_list(args.exclude, False)
    AttributeError: 'Namespace' object has no attribute 'include'
    
    opened by GitSunburn 1
  • UU decoding raises an exception in some cases

    UU decoding raises an exception in some cases

    # cat raw.txt 
    1Oh6axLwfecHErbVpRbtNj8t5JACsSQrofdnnMdQtBmoU8cQj6EyLcVRsQLz1MyWfXbqQDIc9wGyyBuH7uV95lBVpFGn3syGIw0IVLx8wJr4ABsIH9Q71hBH4AvIgljx7XnfjfmafahBNrPMDkK3dsJF0r41nzyMnOf7l36NcllAOgRLoB6Qh0APotZu6plYpkSiRCAkDKowOFm0mybKp336TAJe4JiDecD9hNbl5fBDLkGNYhmSkzOQzLBH1aPumW4o
    
    # codext -i raw.txt decode base62
    begin 666 <data>
    M'XL( /H^)V("__-)SBB-<O7*+#;,K8PL\/ H\C#UR2LP= H)[email protected]\*"[email protected]
    M\"T-\*@(#\]V\<A,*R^W\"@I,W9Q-,LU\8XR=$XM*TYR3PG,2G)+RO P#_'.
    ?"LRJS#)US0X(KPJH<[email protected](<TGR  #3A(K<90      
     
    end
    
    # codext -i raw.txt decode base62 | codext decode UU
    Traceback (most recent call last):
      File "/usr/lib/python3.8/encodings/uu_codec.py", line 58, in uu_decode
        data = binascii.a2b_uu(s)
    binascii.Error: Illegal char
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/home/jeane/.local/bin/codext", line 8, in <module>
        sys.exit(main())
      File "/home/jeane/.local/lib/python3.8/site-packages/codext/__init__.py", line 124, in main
        c = getattr(codecs, ["encode", "decode"][args.command == "decode"])(c, encoding, args.errors)
      File "/home/jeane/.local/lib/python3.8/site-packages/codext/__common__.py", line 691, in decode
        return lookup(encoding).decode(obj, errors)[0]
      File "/usr/lib/python3.8/encodings/uu_codec.py", line 62, in uu_decode
        data = binascii.a2b_uu(s[:nbytes])
    binascii.Error: Illegal char
    
    opened by RomainJennes 1
  • Implement Rail Fence Cipher

    Implement Rail Fence Cipher

    Tests pass.

    RegEx might need a fix : rail-5-3- is recognized as a valid codec (ending dash is too much). Only rail-5-3 or rail-5-3-up should be recognized. Couldn't manage to get that right.

    When encoding, there might be trailing whitespaces (which is normal), but users might forget one when copy/pasting. I left them invisible to ease the integration with other tools.

    opened by smarbal 0
  • Add tap/knock language

    Add tap/knock language

    Tap encoding and decoding is implemented and added to the documentation. Couldn't make it work with add_map() due to problems with letter and word separations (single space/double space). Did my own encoding/decoding functions instead.

    opened by smarbal 0
  • [Snyk] Security upgrade markdown2 from 2.3.10 to 2.4.0

    [Snyk] Security upgrade markdown2 from 2.3.10 to 2.4.0

    Snyk has created this PR to fix one or more vulnerable packages in the `pip` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • requirements.txt

    Vulnerabilities that will be fixed

    By pinning:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 768/1000
    Why? Proof of Concept exploit, Recently disclosed, Has a fix available, CVSS 7.5 | Regular Expression Denial of Service (ReDoS)
    SNYK-PYTHON-MARKDOWN2-1063233 | markdown2:
    2.3.10 -> 2.4.0
    | No | Proof of Concept

    (*) Note that the real score may have changed since the PR was raised.

    Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the effected dependencies could be upgraded.

    Check the changes in this PR to ensure they won't cause issues with your project.


    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    πŸ›  Adjust project settings

    πŸ“š Read more about Snyk's upgrade and patch logic

    opened by dhondta 0
  • pip 22.3 / python 3.10 warning on install

    pip 22.3 / python 3.10 warning on install

    Looks like there's a warning on the installation method via pip:

    pip install codext         
    Collecting codext
      Downloading codext-1.14.0.tar.gz (116 kB)
         ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 116.2/116.2 kB 1.7 MB/s eta 0:00:00
      Preparing metadata (setup.py) ... done
    Requirement already satisfied: six in ./venv/lib/python3.10/site-packages (from codext) (1.16.0)
    Collecting markdown2>=2.4.0
      Downloading markdown2-2.4.6-py2.py3-none-any.whl (37 kB)
    Installing collected packages: markdown2, codext
      DEPRECATION: codext is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
      Running setup.py install for codext ... done
    Successfully installed codext-1.14.0 markdown2-2.4.6
    
    opened by mikeatlas 0
Releases(1.10.1)
Owner
Alex
I'm a security professional, passionate about programming (especially Python) and cyber security.
Alex
Yts-cli-streamer - A CLI movie streaming client which works on yts.mx API written in python

YTSP It is a CLI movie streaming client which works on yts.mx API written in pyt

1 Feb 05, 2022
Module for converting 2D Python lists to fancy ASCII tables. Table2Ascii lets you display pretty tables in the terminal and on Discord.

table2ascii Module for converting 2D Python lists to a fancy ASCII/Unicode tables table2ascii πŸ“₯ Installation πŸ§‘β€πŸ’» Usage Convert lists to ASCII table

Jonah Lawrence 40 Jan 03, 2023
CLI utility for updating the EVE Online static data export in a postgres database

EVE SDE Postgres updater CLI utility for updating the EVE Online static data export postgres database. This has been tested with the Fuzzwork postgres

Markus Juopperi 1 Oct 29, 2021
kitty - the fast, feature-rich, cross-platform, GPU based terminal

kitty - the fast, feature-rich, cross-platform, GPU based terminal

Kovid Goyal 17.3k Jan 04, 2023
PyWordle: A Python-made wordle manual solver

PyWordle: A Python-made wordle manual solver How to use it Start the program with python3 pywordlesolver.py. How it works The program has a simple 5-l

Federico Torrielli 5 Nov 24, 2022
Themes for the kitty terminal emulator

Themes for the kitty terminal This is a collection of themes for the kitty terminal emulator. The themes were initially imported from dexpota/kitty-th

Kovid Goyal 190 Jan 05, 2023
Tiny command-line utility for mapping broken keys to other positions.

brokenkey Tiny command-line utility for mapping broken keys to other positions. Installation Clone this repository using git: git clone https://github

0 Oct 04, 2021
Command line interface for unasync

CLI for unasync Command line interface for unasync Getting started Install Run the following command to install the package with pip: pip install unas

Leynier GutiΓ©rrez GonzΓ‘lez 3 Apr 04, 2022
Salesforce object access auditor

Salesforce object access auditor Released as open source by NCC Group Plc - https://www.nccgroup.com/ Developed by Jerome Smith @exploresecurity (with

NCC Group Plc 90 Sep 19, 2022
A command line tool to query source code from your current Python env

wxc wxc (pronounced "which") allows you to inspect source code in your Python environment from the command line. It is based on the inspect module fro

ClΓ©ment Robert 13 Nov 08, 2022
Terminal Colored Text for Python

Terminal Colored Text for Python

R3CKhi-**75 3 Sep 10, 2022
A command-line based, minimal torrent streaming client made using Python and Webtorrent-cli.

ABOUT A command-line based, minimal torrent streaming client made using Python and Webtorrent-cli. Installation pip install -r requirements.txt It use

Janardon Hazarika 17 Dec 11, 2022
MasterDuel Image Recognition Translation Command Line Tool

MasterDuelTranslate(Use Ygo Card DataBase,belong win32 window shot & image match)

PatchouliTC 77 Dec 01, 2022
Pastekan adalah website paste kode / teks sederhana

Pastekan pastekan adalah website paste kode / teks sederhana. install pip install pastekan penggunaan pastekan myfile.txt atau echo "hi" | pastekan

Sekai Kode 1 Dec 24, 2021
Chat In Terminal - Chat-App in python

Chat In Terminal Hello all. πŸ˜‰ Sockets and servers are vey important for connection and importantly chatting with others. πŸ˜‚ 😁 I have thought of maki

Shreejan Dolai 5 Nov 17, 2022
Commandline Python app to Autodownload mediafire folders and files.

Commandline Python app to Autodownload mediafire folders and files.

Tharuk Renuja 3 May 12, 2022
A command line tool that creates a super timeline from SentinelOne's Deep Visibility data

S1SuperTimeline A command line tool that creates a super timeline from SentinelOne's Deep Visibility data What does it do? The script accepts a S1QL q

Juan Ortega 2 Feb 08, 2022
Magnificent app which corrects your previous console command.

The Fuck The Fuck is a magnificent app, inspired by a @liamosaur tweet, that corrects errors in previous console commands. Is The Fuck too slow? Try t

Vladimir Iakovlev 75k Jan 02, 2023
bsp_tool provides a Command Line Interface for analysing .bsp files

bsp_tool Python library for analysing .bsp files bsp_tool provides a Command Line Interface for analysing .bsp files Current development is focused on

Jared Ketterer 64 Dec 28, 2022
Bryce Geiser 4 Aug 04, 2022