FFPuppet is a Python module that automates browser process related tasks to aid in fuzzing

Overview

FFPuppet

Task Status codecov Matrix PyPI

FFPuppet is a Python module that automates browser process related tasks to aid in fuzzing. Happy bug hunting!

Are you fuzzing the browser? Grizzly can help.

Installation

To install the latest version from PyPI
pip install ffpuppet
Xvfb on Linux

On Linux xvfb can be used in order to run headless (this is not the same as Firefox's -headless mode).

To install xvfb on Ubuntu run:

apt-get install xvfb
Installing minidump_stackwalk

minidump_stackwalk is used to extract a crash report when the browser crashes without a debugger (GDB/Valgrind) or instrumentation (ASan). If desired, minidump_stackwalk should be installed in the users path after obtaining it from FirefoxCI. Choose the appropriate tool for your platform (linux64-minidump-stackwalk, macosx64-minidump-stackwalk, win32-minidump-stackwalk, etc.), choose latest, then extract the executable from the downloaded tar archive.

Browser Builds

If you are looking for builds to use with FFPuppet there are a few options.

Downloading builds

fuzzfetch is the recommended method for obtaining builds and is also very helpful in automation.

Taskcluster has a collection of many different build types for multiple platforms and branches. An index of the latest mozilla-central builds can be found here.

Build your own

If you would like to compile your own, build instructions can be found here.

Usage

Once installed FFPuppet can be run using the following command:

python -m ffpuppet
usage: ffpuppet [-h] [-d] [--log-level LOG_LEVEL] [-e EXTENSION] [-p PREFS]
                [-P PROFILE] [-u URL] [--xvfb] [-a ABORT_TOKEN]
                [--launch-timeout LAUNCH_TIMEOUT] [-l LOGS]
                [--log-limit LOG_LIMIT] [-m MEMORY]
                [--poll-interval POLL_INTERVAL] [--save-all]
                [--gdb | --pernosco | --rr | --valgrind]
                binary

FFPuppet - Firefox process launcher and log collector. Happy bug hunting!

positional arguments:
  binary                Firefox binary to launch

optional arguments:
  -h, --help            show this help message and exit
  -d, --display-logs    Display summary of browser logs on process exit.
  --log-level LOG_LEVEL
                        Configure console logging. Options: DEBUG, INFO, WARN,
                        ERROR (default: INFO)

Browser Configuration:
  -e EXTENSION, --extension EXTENSION
                        Install extensions. Specify the path to the xpi or the
                        directory containing the unpacked extension.
  -p PREFS, --prefs PREFS
                        Custom prefs.js file to use (default: profile default)
  -P PROFILE, --profile PROFILE
                        Profile to use. This is non-destructive. A copy of the
                        target profile will be used. (default: temporary
                        profile)
  -u URL, --url URL     Server URL or path to local file to load.
  --xvfb                Use Xvfb. (Linux only)

Issue Detection & Reporting:
  -a ABORT_TOKEN, --abort-token ABORT_TOKEN
                        Scan the browser logs for the given value and close
                        browser if detected. For example '-a ###!!!
                        ASSERTION:' would be used to detect soft assertions.
  --launch-timeout LAUNCH_TIMEOUT
                        Number of seconds to wait for the browser to become
                        responsive after launching. (default: 300)
  -l LOGS, --logs LOGS  Location to save browser logs. A sub-directory
                        containing the browser logs will be created.
  --log-limit LOG_LIMIT
                        Browser log file size limit in MBs (default: 0, no
                        limit)
  -m MEMORY, --memory MEMORY
                        Browser memory limit in MBs (default: 0, no limit)
  --poll-interval POLL_INTERVAL
                        Delay between checks for results (default: 0.5)
  --save-all            Always save logs. By default logs are saved only when
                        an issue is detected.

Available Debuggers:
  --gdb                 Use GDB. (Linux only)
  --pernosco            Use rr. Trace intended to be submitted to Pernosco. (Linux only)
  --rr                  Use rr. (Linux only)
  --valgrind            Use Valgrind. (Linux only)

Replaying a test case
python -m ffpuppet 
   
     -p 
    
      -d -u 
     

     
    
   

This will open the provided test case file in Firefox using the provided prefs.js file and any log data (stderr, stdout, ASan logs... etc) will be dumped to the console when the browser process terminates. Grizzly Replay is recommended for replaying test cases.

Prefs.js files

prefs.js files that can be used for fuzzing or other automated testing can be generated with PrefPicker.

Comments
  • ffpuppet does not work with older version of Firefox

    ffpuppet does not work with older version of Firefox

    I have build Firefox 57.0 and Firefox 63.0.3 (with ASAN) by myself. Running python -m ffpuppet ../../firefox-57.0/objdir-ff-asan/dist/bin/firefox -p $CODE/browsers/prefs.js -d -u /home/ug16zy2/test.html -v --xvfb --log ./out/ gives:

    I ffpuppet [2020-01-01 19:55:44] Launching Firefox... D ffpuppet [2020-01-01 19:55:44] requested location: '/home/ug16zy2/test.html' D ffpuppet [2020-01-01 19:55:44] launch timeout: 300 D ffpuppet [2020-01-01 19:55:44] profile directory: '/tmp/ffprof_VspYuS' D ffpuppet [2020-01-01 19:55:44] using prefs.js: '/home/ug16zy2/fuzz/browsers/prefs.js' D ffpuppet [2020-01-01 19:55:44] launch command: '/home/ug16zy2/firefox-57.0/objdir-ff-asan/dist/bin/firefox -no-remote -profile /tmp/ffprof_VspYuS http://127.0.0.1:46027' D ffpuppet [2020-01-01 19:55:44] launched firefox with pid: 21507 D ffpuppet [2020-01-01 19:55:48] crash report found I ffpuppet [2020-01-01 19:55:48] Shutting down... D ffpuppet [2020-01-01 19:55:48] close(force_close=False) called D ffpuppet [2020-01-01 19:55:48] browser pid: 21507 D ffpuppet [2020-01-01 19:55:48] 1 crash report(s) are available D ffpuppet [2020-01-01 19:55:53] reviewing 0 check(s) D ffpuppet [2020-01-01 19:55:53] scan_path '/tmp/ffprof_VspYuS/minidumps' does not exist D ffpuppet [2020-01-01 19:55:53] exit reason code 'ALERT' I ffpuppet [2020-01-01 19:55:53] Firefox process is closed. (Reason: 'ALERT') I ffpuppet [2020-01-01 19:55:53] Saving logs to '/home/ug16zy2/fuzz/ffpuppet/out' D ffpuppet [2020-01-01 19:55:53] save_logs() called, dest='./out/', logs_only=False, meta=False I ffpuppet [2020-01-01 19:55:53] Displaying logs...

    and the ASAN log is:

    === Dumping 'log_ffp_asan_21493.log.21507.txt' (1.83KB) ==21507==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7f9c17e13646 bp 0x7fffd3f6b340 sp 0x7fffd3f6aab8 T0) ==21507==The signal is caused by a READ memory access. ==21507==Hint: address points to the zero page. #0 0x7f9c17e13645 (/lib/x86_64-linux-gnu/libc.so.6+0xb1645) #1 0x7f9c18ef7146 (/usr/lib/x86_64-linux-gnu/libasan.so.4+0x42146) #2 0x7f9c11652660 (/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0+0x39660) #3 0x7f9c11652806 in g_dgettext (/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0+0x39806) #4 0x7f9c144f13ae in gtk_get_option_group (/usr/lib/x86_64-linux-gnu/libgtk-3.so.0+0x22f3ae) #5 0x7f9c144f15fb in gtk_parse_args (/usr/lib/x86_64-linux-gnu/libgtk-3.so.0+0x22f5fb) #6 0x7f9bfb69192f in XREMain::XRE_mainStartup(bool*) /home/ug16zy2/firefox-57.0/toolkit/xre/nsAppRunner.cpp:3911 #7 0x7f9bfb698931 in XREMain::XRE_main(int, char**, mozilla::BootstrapConfig const&) /home/ug16zy2/firefox-57.0/toolkit/xre/nsAppRunner.cpp:4852 #8 0x7f9bfb699030 in XRE_main(int, char**, mozilla::BootstrapConfig const&) /home/ug16zy2/firefox-57.0/toolkit/xre/nsAppRunner.cpp:4962 #9 0x7f9bfb6c3d6d in mozilla::BootstrapImpl::XRE_main(int, char**, mozilla::BootstrapConfig const&) /home/ug16zy2/firefox-57.0/toolkit/xre/Bootstrap.cpp:45 #10 0x55fa5cd415df in do_main /home/ug16zy2/firefox-57.0/browser/app/nsBrowserApp.cpp:236 #11 0x55fa5cd41c87 in main /home/ug16zy2/firefox-57.0/browser/app/nsBrowserApp.cpp:309 #12 0x7f9c17d83b96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96) #13 0x55fa5cd408d9 in _start (/home/ug16zy2/firefox-57.0/objdir-ff-asan/dist/bin/firefox+0x3e8d9) AddressSanitizer can not provide additional info. SUMMARY: AddressSanitizer: SEGV (/lib/x86_64-linux-gnu/libc.so.6+0xb1645) ==21507==ABORTING

    When I run FFPuppet with /usr/bin/firefox, it works fine and Firefox instance pops up with the input HTML file.

    Does FFPuppet support older version of Firefox?

    opened by ZihanYe 6
  • Added the ability to specify a timeout in order to terminate during hangups.

    Added the ability to specify a timeout in order to terminate during hangups.

    This adds a timeout parameter (-w) by implementing a check called CheckRunningTimeout. The timeout lets someone specify a number of seconds before terminating the browser. Pretty basic really.

    enhancement wontfix 
    opened by arizvisa 6
  • Finish typing support.

    Finish typing support.

    This finishes the work by @nth10sd in #115 . Much of the nicer syntax depends on __future__ annotations which has other caveats, so I've backported this to 3.6 for now.

    This also enables mypy in pre-commit, enables Python 3.10 CI, and enables lint across all Python versions in CI.

    Supersedes #115.

    opened by jschwartzentruber 4
  • Run minidump_stackwalk on any minidumps in the profile directory …

    Run minidump_stackwalk on any minidumps in the profile directory …

    …and append to the log.

    • only works if minidump_stackwalk is in path, otherwise a warning is printed if a minidump is found
    • remove obsolete and incomplete breakpad symbolization
    • symbolize argument to save_log() and clone_log() has been removed
    opened by jschwartzentruber 4
  • Add mypy type information and bump minimum Python version to 3.7

    Add mypy type information and bump minimum Python version to 3.7

    This PR adds mypy type information for ffpuppet, which helps the PR for FuzzManager type info as well. I also deleted type information if present in the comments, since some were already inaccurate IIRC.

    It bumps the minimum Python version to 3.7 since we import annotations from __future__.

    GitHub Actions CI is here.

    To-do:

    • [x] Push a py.typed file on completion
    opened by nth10sd 3
  • Fix #18: --url localfile doesn't work on Windows.

    Fix #18: --url localfile doesn't work on Windows.

    After this change yields:

    windows:
    D ffpuppet [2017-09-07 14:55:24] sending response with redirect url: 'file:///C:/Users/user/Desktop/tc.html'
    linux:
    D ffpuppet [2017-09-07 15:01:45] sending response with redirect url: 'file:/home/user/Desktop/tc.html'
    

    Both redirects work properly.

    opened by jschwartzentruber 3
  • Add support for arbitrary/multiple extensions.

    Add support for arbitrary/multiple extensions.

    Extensions are not going away, and we should support installing them for testing even if we stop using them regularly (which is not planned). This supports arbitrary add-ons by looking up the extension ID in the manifest (may be fragile for legacy add-ons because of XML namespaces).

    Fixes #4.

    enhancement 
    opened by jschwartzentruber 2
  • Only set max_allocation_size_mb and soft_rss_limit_mb for ASan and TSan

    Only set max_allocation_size_mb and soft_rss_limit_mb for ASan and TSan

    Setting these values in UBSAN_OPTIONS as well is a foot gun. It can can lead to confusion if set for multiple sanitizers. It can still be set manually in the environment.

    opened by tysmith 1
  • Reduce the shutdown wait after a crash

    Reduce the shutdown wait after a crash

    This will reduce the wait period for the browser processes to close after a crash log is detected.

    The browser should shutdown quickly after a crash report is written.

    opened by tysmith 1
  • Consider randomizing the resolution passed to xvfb

    Consider randomizing the resolution passed to xvfb

    It might be worth considering setting a random resolution to xvfb as this might affect layout. Similar concept to the TTF Fuzzer outlined here:

    https://media.blackhat.com/bh-eu-12/Lee/bh-eu-12-Lee-GDI_Font_Fuzzing-WP.pdf

    opened by pyoor 1
Releases(0.9.2)
Owner
Mozilla Fuzzing Security
Fuzzing projects at the Mozilla Corporation
Mozilla Fuzzing Security
Automação de Processos (obtenção de informações com o Selenium), atualização de Planilha e Envio de E-mail.

Automação de Processo: Código para acompanhar o valor de algumas ações na B3. O código entra no Google Drive, puxa os valores das ações (pré estabelec

Hemili Beatriz 1 Jan 08, 2022
Based on the selenium automatic test framework of python, the program crawls the score information of the educational administration system of a unive

whpu_spider 该程序基于python的selenium自动化测试框架,对某高校的教务系统的成绩信息实时爬取,在检测到成绩更新之后,会通过电子邮件的方式,将更新的成绩以文本的方式发送给用户,可以使得用户在不必手动登录教务系统网站时,实时获取成绩更新的信息。 该程序仅供学习交流,不可用于恶意攻

1 Dec 30, 2021
pytest plugin for manipulating test data directories and files

pytest-datadir pytest plugin for manipulating test data directories and files. Usage pytest-datadir will look up for a directory with the name of your

Gabriel Reis 191 Dec 21, 2022
WrightEagle AutoTest (Has been updated by Cyrus team members)

Autotest2d WrightEagle AutoTest (Has been updated by Cyrus team members) Thanks go to WrightEagle Members. Steps 1- prepare start_team file. In this s

Cyrus Soccer Simulation 2D Team 3 Sep 01, 2022
Selenium Page Object Model with Python

Page-object-model (POM) is a pattern that you can apply it to develop efficient automation framework.

Mohammad Ifran Uddin 1 Nov 29, 2021
Generates realistic traffic for load testing tile servers

Generates realistic traffic for load testing tile servers. Useful for: Measuring throughput, latency and concurrency of your tile serving stack. Ident

Brandon Liu 23 Dec 05, 2022
A rewrite of Python's builtin doctest module (with pytest plugin integration) but without all the weirdness

The xdoctest package is a re-write of Python's builtin doctest module. It replaces the old regex-based parser with a new abstract-syntax-tree based pa

Jon Crall 174 Dec 16, 2022
Python Projects - Few Python projects with Testing using Pytest

Python_Projects Few Python projects : Fast_API_Docker_PyTest- Just a simple auto

Tal Mogendorff 1 Jan 22, 2022
A command-line tool and Python library and Pytest plugin for automated testing of RESTful APIs, with a simple, concise and flexible YAML-based syntax

1.0 Release See here for details about breaking changes with the upcoming 1.0 release: https://github.com/taverntesting/tavern/issues/495 Easier API t

909 Dec 15, 2022
A browser automation framework and ecosystem.

Selenium Selenium is an umbrella project encapsulating a variety of tools and libraries enabling web browser automation. Selenium specifically provide

Selenium 25.5k Jan 01, 2023
hCaptcha solver and bypasser for Python Selenium. Simple website to try to solve hCaptcha.

hCaptcha solver for Python Selenium. Many thanks to engageub for his hCaptcha solver userscript. This script is solely intended for the use of educati

Maxime Dréan 59 Dec 25, 2022
The Penetration Testers Framework (PTF) is a way for modular support for up-to-date tools.

The PenTesters Framework (PTF) is a Python script designed for Debian/Ubuntu/ArchLinux based distributions to create a similar and familiar distribution for Penetration Testing

trustedsec 4.5k Dec 28, 2022
WEB PENETRATION TESTING TOOL 💥

N-WEB ADVANCE WEB PENETRATION TESTING TOOL Features 🎭 Admin Panel Finder Admin Scanner Dork Generator Advance Dork Finder Extract Links No Redirect H

56 Dec 23, 2022
A Python program that will log into your scheduled Google Meets hands free

Chrome GMautomation General Information This Python program will open up Chrome and log into your scheduled Google Meet with camera and mic turned off

Jonathan Leow 5 Dec 31, 2021
Repository for JIDA SNP Browser Web Application: Local Deployment

JIDA JIDA is a web application that retrieves SNP information for a genomic region of interest in Homo sapiens and calculates specific summary statist

3 Mar 03, 2022
User-interest mock backend server implemnted using flask restful, and SQLAlchemy ORM confiugred with sqlite

Flask_Restful_SQLAlchemy_server User-interest mock backend server implemnted using flask restful, and SQLAlchemy ORM confiugred with sqlite. Backend b

Austin Weigel 1 Nov 17, 2022
A pure Python script to easily get a reverse shell

easy-shell A pure Python script to easily get a reverse shell. How it works? After sending a request, it generates a payload with different commands a

Cristian Souza 48 Dec 12, 2022
Using openpyxl in Python, performed following task

Python-Automation-with-openpyxl Using openpyxl in Python, performed following tasks on an Excel Sheet containing Product Suppliers along with their pr

1 Apr 06, 2022
A twitter bot that simply replies with a beautiful screenshot of the tweet, powered by poet.so

Poet this! Replies with a beautiful screenshot of the tweet, powered by poet.so Installation git clone https://github.com/dhravya/poet-this.git cd po

Dhravya Shah 30 Dec 04, 2022
Doggo Browser

Doggo Browser Quick Start $ python3 -m venv ./venv/ $ source ./venv/bin/activate $ pip3 install -r requirements.txt $ ./sobaki.py References Heavily I

Alexey Kutepov 9 Dec 12, 2022