ARTEMIS: Real-Time Detection and Automatic Mitigation for BGP Prefix Hijacking.

Related tags

Networkingartemis
Overview

Documentation Status Build Status CodeFactor Code style: black Coverage Status Slack Mailing list Release License

ARTEMIS is an open-source tool, that implements a defense approach against BGP prefix hijacking attacks. It is (a) based on accurate and fast detection operated by the AS itself, by leveraging the pervasiveness of publicly available BGP monitoring services, and it (b) enables flexible and fast mitigation of hijacking events. Compared to existing approaches/tools, ARTEMIS combines characteristics desirable to network operators such as comprehensiveness, accuracy, speed, privacy, and flexibility. With the ARTEMIS approach, prefix hijacking can be neutralized within a minute!

Read more at bgpartemis.org and the docs.

Table of Contents

General

Depending on the preferences of the user, ARTEMIS can be used in 3 basic modes according to the combination of enabled micro-services in the user interface:

  1. Passive monitor (monitoring enabled)
  2. Passive detector (monitoring + detection enabled)
  3. Active joint detector and user-triggered mitigator (monitoring + detection + mitigation enabled)

Any of these combinations is valid. To start with, we recommend using mode (2). Mode (3) is under development (currently only a mitigation wrapper is offered).

You can read more about the ARTEMIS methodology, blog posts, presentations, publications and research experiments on the ARTEMIS webpage.

This repository contains the software of ARTEMIS as a tool. ARTEMIS can be run on a server/VM as a modular and extensible multi-container (microservice) application. It has been officially tested at AMS-IX, a major greek ISP, FORTH (a dual-homed edge academic network), and Internet2 (a major US R&E backbone network). Several other network operators use it either in production or in a testing environment.

Features

For a detailed list of supported features please check the CHANGELOG file (sections: "Added"). The following main features are supported:

  • Real-time monitoring of the changes in the BGP routes of the prefixes originated by the AS running ARTEMIS, via:
  • Real-time detection and notifications of BGP prefix hijacking attacks/events of the following types (please refer to the attack taxonomy in our ARTEMIS IEEE/ACM ToN paper):
    • exact-prefix, type 0/1, any data plane manipulation
    • sub-prefix, any type (0/1/-), any data plane manipulation
    • squatting attacks, type 0 (others are N/A), any data plane manipulation
    • policy violations (route leaks) due to long paths towards no-export prefixes
  • Automatic/custom tagging of detected BGP hijack events (ongoing, resolved, ignored, under mitigation, withdrawn, outdated and dormant).
  • Manual or manually controlled mitigation of BGP prefix hijacking attacks.
  • Comprehensive web-based User Interface (UI).
  • Configuration file editable by the operator (directly or via the UI), containing information about: prefixes, ASNs, monitors and ARTEMIS rules ("ASX originates prefix P and advertises it to ASY").
  • Support for both IPv4 and IPv6 prefixes (millions of routed prefixes depending on your resources).
  • Support for both mobile and desktop environments (UI): sample screenshots.
  • Support for docker-compose (local single-server deployment) and Kubernetes (helm charts).
  • Support for multiple modes of operation (passive monitor/detector, active mitigator, etc.).
  • Support for historical BGP update replaying.
  • Support for automated generation of the configuration file.
  • Support for RPKI validation of hijacked prefixes.
  • Compatibility with Grafana charts.
  • Modularity/extensibility by design.
  • CI/CD (Travis CI, Codecov).

Read more at bgpartemis.org and the docs.

System Architecture

Please check this page.

Getting Started

ARTEMIS is built as a multi-container Docker application. The following instructions will get you a containerized copy of the ARTEMIS tool up and running on your local machine using the docker-compose utility. For instructions on how to set up ARTEMIS in a Kubernetes environment, please check the related docs page.

Minimum Technical Requirements

  • CPU: 4 cores (note that needed CPU cores depend on the number of separate processes, e.g., detectors or database modules you spawn)
  • RAM: 4+ GB (note that needed memory depends on the number of configured prefixes/rules/asns and load of incoming BGP updates, see here for more details)
  • HDD: 50 GB (less may suffice, depending on the use case for storing BGP updates and hijack alerts)
  • NETWORK: 1 public-facing network interface (optionally: one internal interface for connection with local route collectors)
  • OS: Ubuntu Linux 16.04+ (other Linux distributions will work too)
  • SW PACKAGES: docker-ce and docker-compose should be pre-installed (see instructions later) and docker should have sudo privileges, if only non-sudo user is allowed
  • Other: SSH server

Moreover, one may optionally configure firewall rules related to the server/VM. We recommend using ufw for this task. Please check the comments in the respective script we provide and set the corresponding <> fields in the file before running:

sudo ./other/ufw_setup.sh

NOTE: For security reasons, we highly recommend protecting your machine with such rules. ARTEMIS tries to minimize external port exposure to minimize the attack surface on the system itself.

How to Install and Setup

To download and install the required software packages, please follow steps 1 through 6 described in this docs section.

To setup the tool (as well as https access to it via the web application), please follow steps 1 through 5 described in this docs section.

Note that specifically for testing purposes, we now support vagrant and VirtualBox VM automation; please check out this docs page for simple instructions on how to spin up a fully functioning ARTEMIS VM, running all needed microservices, within a minute.

How to Run and Configure

  1. Start ARTEMIS:

    docker-compose up -d

    Please consult this docs section if you need to activate additional services.

  2. Visit web UI and configure ARTEMIS:

    https://

    By visiting the system page:

    https://
     
      /admin/system

    you can:

    1. edit the basic configuration file of ARTEMIS that serves as the ground truth for detecting BGP hijacks (consult this docs section first)
    2. control the monitoring, detection and mitigation modules.

  3. Stop ARTEMIS (optional)

    docker-compose stop

Note: We highly recommend going through the detailed docs instructions before using ARTEMIS for the first time. You can further use several other microservices orthogonal to ARTEMIS (like grafana and routinator) by using the main ARTEMIS docker-compose yaml plus the additional yamls:

docker-compose -f docker-compose.yaml -f docker-compose.
   
    .yaml -... 
    
     /
     
      /...

Demo

A running demo of ARTEMIS based on the configuration of our home institute (FORTH) can be found here. You can access the demo as a guest (non-admin) user by using the following credentials:

Please do not request new accounts on the demo portal. Use the given credentials to browse ARTEMIS as a guest user. In case you need admin access, simply clone ARTEMIS locally and use the given configuration file.

Contributing

Please check this file.

Development Team and Contact

We follow a custom Agile approach for our development.

If you need to contact us about a bug, an issue or a question you have; you can reach us over at our Slack Community Channel. Otherwise, you can contact the ARTEMIS developers via e-mail using the ARTEMIS mailing list.

Versioning

Please check this file.

Authors and Contributors

Please check this file.

Documentation

Read more at bgpartemis.org and the docs.

License

The ARTEMIS software is open-sourced under the BSD-3 license. Please check the license file.

Note that all external dependencies are used in a way compatible with BSD-3 (that is, we conform to the compatibility rules of each and every dependency); the associated software packages and their respective licenses are documented in detail in this file, where we provide links to their homepages and licenses. Please let us know in case any of the information contained there is out-of-date to update it.

Side projects

  • Prototype software to enable auto-configuration and auto-mitigation in ARTEMIS using Ansible: Github repo.

Acknowledgements and Funding Sources

This work is supported by the following funding sources on the European side (FORTH):

The following funding sources supported the collaboration with CAIDA UCSD, on the US side:

  • National Science Foundation (NSF) grants OAC-1848641 and CNS-1423659
  • Department of Homeland Security (DHS) Science and Technology Directorate, Cyber Security Division (DHS S&T/CSD) via contract number HHSP233201600012C
  • Comcast Innovation Fund

Powered By

DISCLAIMER: We do not own these logo images. All links to the respective project pages from where the logos were downloaded are contained in this file, together with their respective licenses. The sole purpose of this section is to thank the open-source software projects that enabled ARTEMIS with their functionality and APIs, by making them as visible as possible. The list of project logos is not exhaustive. Image copyright is retained by the respective project's copyright owners.

Comments
  • Keeps redirecting to http instead of https

    Keeps redirecting to http instead of https

    Describe the bug The pages keep redirecting me to http://my_url:443/... I found this: https://stackoverflow.com/questions/14810795/flask-url-for-generating-http-url-instead-of-https

    Am I on the right path?

    I have traefik ingress serving the requests to pods, and deployed your helm chart.

    Affected Component(s)

    • [ ] Back-End (Database, Microservices, Containers, etc)
    • [X] Front-End (Flask, API, etc)
    • [ ] Docs
    • [ ] Build System

    Expected behavior Return https:// scheme in redirects

    frontend p/high 
    opened by dimm0 26
  • Overruling rules (AS announcements)

    Overruling rules (AS announcements)

    Describe the bug I have a list of prefixes with the same origin AS. I want to create two rulesets:

    1. prefixlist with origin AS and a select set of neighbors;
    2. prefixlist with origin AS and an other select list of neighbors for which a no_export is required. Aspected behaviour is that both use-cases is respected.

    Affected Component(s)

    • [X] Back-End (Database, Microservices, Containers, etc)
    • [ ] Front-End (Flask, API, etc)
    • [ ] Docs
    • [ ] Build System

    To Reproduce In config.yaml I have the following setup: prefixes: simple_prefix_list: &my_prefixes

    • 192.168.78.0/24
    • 192.168.10.0/24 <... skip monitors ...> asns: my_asn: &my_asn 1 my_neighbors: &myneighbors
    • 11
    • 12 my_neighbors: &my_noexp_neighbors
    • 21
    • 222 rules:
    • prefixes:
      • *my_prefixes origin_asns:
      • *my_asn neighbors:
      • *my_neighbors mitigation: manual
    • prefixes:
      • *my_prefixes origin_asns:
      • *my_asn neighbors:
      • *my_noexp_neighbors policies:
      • 'no-export' mitigation: manual

    Expected behavior Both situation will be observed in their respective situation (is the combination origin AS and neighbour with given policy). Now only the no-export rule is followed.

    Screenshots If applicable, add screenshots to help explain your problem.

    System (please complete the following information):

    • OS: [e.g. iOS]
    • Browser [e.g. chrome, safari]
    • Version latest

    Additional context Add any other context about the problem here.

    backend detection p/high 
    opened by marcgweg 22
  • Backend and monitor modularization

    Backend and monitor modularization

    Description of PR

    What component(s) does this PR affect?

    • [X] Back-End (Database, Detection/Configuration/etc. Microservices)
    • [ ] Front-End (Flask, API, UI, etc)
    • [X] Monitor (RIPE RIS, BGPStream RV/RIS/CAIDA, etc.)
    • [X] Docs
    • [X] Build System

    Does the PR require changes on other components? If yes, please mark the components:

    • [ ] Back-End (Database, Detection/Configuration/etc. Microservices)
    • [ ] Front-End (Flask, API, UI, etc)
    • [ ] Monitor (RIPE RIS, BGPStream RV/RIS/CAIDA, etc.)
    • [ ] Docs
    • [ ] Build System

    Related Issue

    Resolves #463 #480 #379 #473

    Solution

    Type

    • [ ] Bug fix (non-breaking change which fixes an issue)
    • [ ] New feature (non-breaking change which adds functionality)
    • [X] Breaking change (fix or feature that would cause existing functionality to change)
    • [ ] Docs update
    • [ ] None of the above

    Checklist:

    • [X] I have read the contributing guide and my code conforms to the guidelines.
    • [X] This change requires a change in the documentation.
    • [ ] I have updated the documentation accordingly.
    backend container build p/medium docker refactor kubernetes Epic 
    opened by vkotronis 12
  • Not receiving BGP updates over bgpstreamkafka

    Not receiving BGP updates over bgpstreamkafka

    Hi,

    I'm running Artemis v1.5.0 in kubernetes, deployed via the provided helm chart. I have setup openbmp-collector and openbmp-kafka running on an external VM (using openbmp-kafka as the kafka FQDN and I have created an entry in the artemis-monitor hosts file to map openbmp-kafka to the ip of the VM).

    I am publishing data to the collector using mrt2bmp and I can subscribe to the kafka topics from within the monitor container (using the kafka cli) but no route updates are ingested into Artemis. With appropriate use of print statements :) it looks like the kafka provider hangs on rec = stream.get_next_record() in bgpstreamkafka.py: https://github.com/FORTH-ICS-INSPIRE/artemis/blob/88626b7e2f1e1835b0f9e93f33adb928d2c9215d/monitor/core/taps/bgpstreamkafka.py#L95

    Artemis Monitor Logs:

    messaging - 2020-07-20 16:08:36,198 - INFO @ _receive_callback: Initiating monitor...
messaging - 2020-07-20 16:08:36,198 - INFO @ _receive_callback: Starting building monitor prefix tree...
messaging - 2020-07-20 16:08:36,199 - INFO @ _receive_callback: 3 prefixes integrated in monitor prefix tree in total
messaging - 2020-07-20 16:08:36,199 - INFO @ _receive_callback: Finished building monitor prefix tree.
messaging - 2020-07-20 16:08:36,199 - INFO @ _receive_callback: Calculating monitored prefixes for monitor to supervise...
messaging - 2020-07-20 16:08:36,199 - INFO @ _receive_callback: Calculated monitored prefixes for monitor to supervise.
messaging - 2020-07-20 16:08:36,199 - INFO @ _receive_callback: Initiating configured monitoring instances....
messaging - 2020-07-20 16:08:36,206 - INFO @ _receive_callback: All configured monitoring instances initiated.
messaging - 2020-07-20 16:08:36,206 - INFO @ _receive_callback: Monitor initiated, configured and running.
<string> - 2020-07-20 16:08:36,211 - INFO @ <module>: started
<frozen importlib - 2020-07-20 16:08:36,471 - INFO @ _call_with_frames_removed: Loaded configuration from /etc/artemis/logging.yaml
<frozen importlib - 2020-07-20 16:08:36,475 - INFO @ _call_with_frames_removed: Loaded configuration from /etc/artemis/logging.yaml
exabgp_client - 2020-07-20 16:08:36,481 - INFO @ <module>: Loaded configuration from /etc/artemis/logging.yaml
exabgp_client - 2020-07-20 16:08:36,484 - INFO @ <module>: Starting ExaBGP on 10.254.5.34:5000 for /root/monitor_prefixes.json (auto-conf: False)
bgpstreamkafka - 2020-07-20 16:08:36,489 - INFO @ <module>: Loaded configuration from /etc/artemis/logging.yaml
2020-07-20 16:08:37,491 INFO success: monitor entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

    Artemis Bgpstreamkafka config:

    prefixes:
  ...
monitors:
  ...
  bgpstreamkafka:
    host: 10.254.5.33
    port: 9092
    topic: openbmp.bmp_raw
asns:
  ...
rules:
  ...
    bug p/high bmp 
    opened by MarkMckessock 12
  • Add grafana provisioning and 4 dashboards (resolves #80) and (resolve…

    Add grafana provisioning and 4 dashboards (resolves #80) and (resolve…

    • Add a grafana provisioning directory.

    • Add 4 dashboards.

    • Add env vars for grafana user/pass/open view.

    Description of PR

    This PR adds context to Grafana by provisioning out of the box 4 grafana dashboards that use the local postgres as their datasource. A different default username/password is set in .env file. The option to allow to non-authenticated users to view the dashboards is added to allow for including the graph iframes anywhere needed. NOTE:This is an initial approach to bootstrapping Grafana with dashboards and graphs and things can be improved and updated. There is no dedicated documentation for this PR as Grafana documentation is sufficient:

    • https://grafana.com/docs/administration/provisioning/
    • https://grafana.com/docs/features/datasources/postgres/

    What component(s) does this PR affect?

    • [ ] Back-End (Database, Microservices, Containers, etc)
    • [ ] Front-End (Flask, API, etc)
    • [ ] Docs
    • [ ] Build System
    • Grafana: Switch to latest version of Grafana, take advantage of grafana provisioning as described here: https://grafana.com/docs/administration/provisioning/ and bootstrap the postgres as a datasource plus add 4 dashboards (BGP Updates per prefix, BGP Highjacks per prefix, Service updates and Offending ASes).

    Does the PR require changes on other components? If yes, please mark the components:

    • [ ] Back-End (Database, Microservices, Containers, etc)
    • [ ] Front-End (Flask, API, etc)
    • [ ] Docs
    • [ ] Build System
    • Updates to .env file
    • Updates to grafana docker compose file.
    • Addition of a grafana provisioning directory with datasources and dashboards

    Related Issue

    Resolves #80 and #126

    Solution

    Add grafana dashboards.

    Type

    • [ ] Bug fix (non-breaking change which fixes an issue)
    • [X] New feature (non-breaking change which adds functionality)
    • [ ] Breaking change (fix or feature that would cause existing functionality to change)
    • [ ] Docs update
    • [ ] None of the above

    Checklist:

    • [X] I have read the contributing guide and my code conforms to the guidelines.
    • [ ] This change requires a change in the documentation.
    • [ ] I have updated the documentation accordingly.
    enhancement 
    opened by leopoul 12
  • Trusted feed auto-config from arbitrary paths (origins + prefixes + kafka)

    Trusted feed auto-config from arbitrary paths (origins + prefixes + kafka)

    Description of PR

    What component(s) does this PR affect?

    • [X] Back-End (Database, Detection/Configuration/etc. Microservices)
    • [ ] Front-End (Flask, API, UI, etc)
    • [ ] Monitor (RIPE RIS, BGPStream RV/RIS/CAIDA, etc.)
    • [ ] Docs (incl. wiki)
    • [ ] Build System

    Does the PR require changes on other components? If yes, please mark the components:

    • [ ] Back-End (Database, Detection/Configuration/etc. Microservices)
    • [ ] Front-End (Flask, API, UI, etc)
    • [ ] Monitor (RIPE RIS, BGPStream RV/RIS/CAIDA, etc.)
    • [ ] Docs (incl. wiki)
    • [ ] Build System

    Related Issue

    Resolves #459

    Solution

    Type

    • [ ] Bug fix (non-breaking change which fixes an issue)
    • [X] New feature (non-breaking change which adds functionality)
    • [ ] Breaking change (fix or feature that would cause existing functionality to change)
    • [ ] Docs update
    • [ ] None of the above

    Checklist:

    • [X] I have read the contributing guide and my code conforms to the guidelines.
    • [X] This change requires a change in the documentation.
    • [x] I have updated the documentation accordingly.
    autoconf 
    opened by vkotronis 11
  • Translate local RC feeds (exabgp/hist) into ARTEMIS configuration primitives

    Translate local RC feeds (exabgp/hist) into ARTEMIS configuration primitives

    Description of PR

    What component(s) does this PR affect?

    • [X] Back-End (Database, Microservices, Containers, etc)
    • [ ] Front-End (Flask, API, etc)
    • [ ] Docs
    • [ ] Build System

    Configuration, Monitor, ExaBGP taps

    Does the PR require changes on other components? If yes, please mark the components:

    • [ ] Back-End (Database, Microservices, Containers, etc)
    • [ ] Front-End (Flask, API, etc)
    • [X] Docs
    • [ ] Build System

    Related Issue

    Resolves #253

    Solution

    Instruction to monitor (via configuration) to auto-update configuration based on local monitor feed.

    Type

    • [ ] Bug fix (non-breaking change which fixes an issue)
    • [X] New feature (non-breaking change which adds functionality)
    • [ ] Breaking change (fix or feature that would cause existing functionality to change)
    • [ ] Docs update
    • [ ] None of the above

    Checklist:

    • [X] I have read the contributing guide and my code conforms to the guidelines.
    • [X] This change requires a change in the documentation.
    • [ ] I have updated the documentation accordingly.

    Need to update https://github.com/FORTH-ICS-INSPIRE/artemis/wiki#receiving-bgp-feed-from-local-routerroute-reflectorbgp-monitor-via-exabgp and https://github.com/FORTH-ICS-INSPIRE/artemis/wiki/Configuration-file to let users know that they can use the autoconf flag for exabgp to auto-generate conf file entries. (after completion, testing and merging, ignore for now).

    frontend backend configuration detection database taps mitigation monitor 
    opened by vkotronis 11
  • Autoignore mechanism for hijacks of limited impact/visibility

    Autoignore mechanism for hijacks of limited impact/visibility

    Description of PR

    What component(s) does this PR affect?

    • [X] Back-End (Database, Detection/Configuration/etc. Microservices)
    • [ ] Front-End (Flask, API, UI, etc)
    • [ ] Monitor (RIPE RIS, BGPStream RV/RIS/CAIDA, etc.)
    • [ ] Docs (incl. wiki)
    • [ ] Build System

    database, scheduler, env vars

    Does the PR require changes on other components? If yes, please mark the components:

    • [ ] Back-End (Database, Detection/Configuration/etc. Microservices)
    • [ ] Front-End (Flask, API, UI, etc)
    • [ ] Monitor (RIPE RIS, BGPStream RV/RIS/CAIDA, etc.)
    • [ ] Docs (incl. wiki)
    • [ ] Build System

    Related Issue

    Resolves #346

    Solution

    Type

    • [ ] Bug fix (non-breaking change which fixes an issue)
    • [X] New feature (non-breaking change which adds functionality)
    • [ ] Breaking change (fix or feature that would cause existing functionality to change)
    • [ ] Docs update
    • [ ] None of the above

    Checklist:

    • [X] I have read the contributing guide and my code conforms to the guidelines.

    • [X] This change requires a change in the documentation.

    • [x] I have updated the documentation accordingly.

    • [x] TODO: Add proper tests (TestAutoIgnore)

    • [x] TODO: adjust per configured prefix, consider module separation

    backend automation detection database p/medium kubernetes logging 
    opened by vkotronis 9
  • large config in web editor causes browser to hang

    large config in web editor causes browser to hang

    Describe the bug There seems to be some sort of file size limits for the display engine used to view the config in the system settings. The config I have is currently 415 lines long (22KB on disk). If I go the system tab I get this in firefox:

    Screen Shot 2020-04-24 at 2 52 00 PM

    Chrome hangs for a while before it sends a popup with the same thing, Wait or kill.

    Chrome shows its graphql related:

    Screen Shot 2020-04-24 at 3 13 04 PM

    Affected Component(s)

    • [ ] Back-End (Database, Microservices, Containers, etc)
    • [x] Front-End (Flask, API, etc)
    • [ ] Docs
    • [ ] Build System

    To Reproduce Still trying to nail down exactly where the limit is hit that causes it to stall.

    Expected behavior The config editor shouldn't stall

    Screenshots inline.

    System (please complete the following information):

    • OS: OSX 10.15.4 (Catalina),
    • Browser Chrome v80, firefox v75
    • Version latest on github as of April 23rd 2020

    Additional context Add any other context about the problem here.

    bug frontend p/high autoconf 
    opened by pixitha 9
  • fix: upgrade containers

    fix: upgrade containers

    Updated postgres and the other outdated containers

    Description of PR

    What component(s) does this PR affect?

    • [x] Back-End (Database, Detection/Configuration/etc. Microservices)
    • [ ] Front-End (Flask, API, UI, etc)
    • [ ] Monitor (RIPE RIS, BGPStream RV/RIS/CAIDA, etc.)
    • [ ] Docs
    • [ ] Build System

    Does the PR require changes on other components? If yes, please mark the components:

    • [ ] Back-End (Database, Detection/Configuration/etc. Microservices)
    • [ ] Front-End (Flask, API, UI, etc)
    • [ ] Monitor (RIPE RIS, BGPStream RV/RIS/CAIDA, etc.)
    • [ ] Docs
    • [ ] Build System

    Related Issue

    Resolves #

    Solution

    Type

    • [ ] Bug fix (non-breaking change which fixes an issue)
    • [ ] New feature (non-breaking change which adds functionality)
    • [ ] Breaking change (fix or feature that would cause existing functionality to change)
    • [ ] Docs update
    • [ ] None of the above

    Checklist:

    • [ ] I have read the contributing guide and my code conforms to the guidelines.
    • [ ] This change requires a change in the documentation.
    • [ ] I have updated the documentation accordingly.
    opened by CuriouzK0d3r 8
  • Obvious Memory Usage Reductions

    Obvious Memory Usage Reductions

    It seems like one of the biggest memory users is just the gunicorn workers, so it would be nice to be able to reduce the gunicorn worker count with an env variable (I set it down to 1 and didn't notice any performance change). That plus setting RABBITMQ_IO_THREAD_POOL_SIZE: 16 in docker-compose.yaml seems to have reduced memory usage by about a full GB.

    enhancement configuration p/medium 
    opened by TheBlueMatt 8
  • Standard SYSLOG Handler Causes Multiple Containers to Fail to Startu (Ubuntu 20.04)

    Standard SYSLOG Handler Causes Multiple Containers to Fail to Startu (Ubuntu 20.04)

    Describe the bug When enabling the syslog handler in the config files found at:

    local_config/monitor/logging.yaml
local_config/backend/logging.yaml

    (with detached config).

    Various containers fail to start or get stuck in restart loops.

    riperistap_1         |   File "/usr/lib/python3.7/logging/handlers.py", line 828, in __init__
riperistap_1         |     ress = socket.getaddrinfo(host, port, 0, socktype)
riperistap_1         |   File "/usr/lib/python3.7/socket.py", line 748, in getaddrinfo
riperistap_1         |     for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
riperistap_1         | socket.gaierror: [Errno -8] Servname not supported for ai_socktype
riperistap_1         |
riperistap_1         | The above exception was the direct cause of the following exception:
riperistap_1         |
riperistap_1         | Traceback (most recent call last):
riperistap_1         |   File "<string>", line 1, in <module>
riperistap_1         |   File "core/ripe_ris.py", line 11, in init core.ripe_ris
riperistap_1         |     from artemis_utils import get_ip_version
riperistap_1         |   File "/usr/local/lib/python3.7/dist-packages/artemis_utils/__init__.py", line 36, in <module>
riperistap_1         |     log = get_logger()
riperistap_1         |   File "/usr/local/lib/python3.7/dist-packages/artemis_utils/__init__.py", line 25, in get_logger
riperistap_1         |     logging.config.dictConfig(config)
riperistap_1         |   File "/usr/lib/python3.7/logging/config.py", line 799, in dictConfig
riperistap_1         |     dictConfigClass(config).configure()

    Affected Component(s)

    • [ ] Back-End Microservices

    To Reproduce Steps to reproduce the behavior:

    1. Enable SYSLOG handler in local configuration files (logging.yaml)
    2. docker-compose up -d
    3. Containers appear to start and then restart repeatedly

    To resolve:

        syslog_handler:
        class: logging.handlers.SysLogHandler
        level: DEBUG
        formatter: simple
        address: ['<ip of dockerhost>', 514]

    Expected behavior

    microservice containers should start as expected

    System (please complete the following information):

    • OS: Ubuntu 20.04
    • Docker Version: 20.10.17
    • Docker Compose Version: 1.29.2
    opened by chewsterB 4
  • Routinator Fails to Start

    Routinator Fails to Start

    Describe the bug When trying to start routinator - it fails with:

    Failed to read config file /home/routinator/.routinator.conf: Is a directory (os error 21)
Failed to read config file /home/routinator/.routinator.conf: Is a directory (os error 21)

    Affected Component(s)

    • [x] routinator

    To Reproduce Steps to reproduce the behavior:

    1. Follow the instructions here: https://bgpartemis.readthedocs.io/en/latest/rpkivalidconf/
    2. Routinator fails with the above log (docker logs routinator)
    3. This line in the instructions creates a directory called routinator.conf with the routinator.conf inside it cp other/routinator/routinator.conf local_configs/routinator/routinator.conf &&
    4. Edit the docker-compose.routinator.yaml and change the volumes line to:
    volumes:
           - ./local_configs/routinator/tals:/home/routinator/.rpki-cache/tals
           - ./local_configs/routinator/routinator.conf/routinator.conf:/home/routinator/.routinator.conf

    Expected behavior Routinator should start, as per instructions

    System (please complete the following information):

    • OS: Ubuntu 20.04
    • Docker: 20.10.17
    • Docker-compose: 1.29.2
    opened by chewsterB 1
  • ARTEMIS Lite design and PoC

    ARTEMIS Lite design and PoC

    Is your feature request related to a problem? Please describe. Lighter ARTEMIS version with bare minimum reqs:

    • Configuration primitives (YAML) + memory importer
    • Simple RIS Live websocket monitor
    • Detector (comparing input from RIS live with the YAML file in-mem info)
    • Alerter with integrations
    • No DB state, no per-hijack state (simple triggers + alerts)
    • No multi-user DB, should be runnable with very little reqs on Linux or Mac
    • Only UI
    • for speed: YAML file should be stored in an in-mem structure (prefix trie?) which can also support quick lookups for incoming BGP updates.
    • The user should be able to see: BGP updates, BGP hijacks/alerts
    • Needs some intelligence on alert creation (e.g., batch processing of several alerts saying the same thing)

    Describe the solution you'd like Rust-based implementation (not Python for performance reasons)

    Describe alternatives you've considered Rust, golang

    Additional context N/A (TBD)

    enhancement p/low 
    opened by vkotronis 5
  • Bump mkdocs from 1.1.2 to 1.2.3 in /other

    Bump mkdocs from 1.1.2 to 1.2.3 in /other

    Bumps mkdocs from 1.1.2 to 1.2.3.

    Release notes

    Sourced from mkdocs's releases.

    1.2.3

    MkDocs 1.2.3 is a bugfix release for MkDocs 1.2.

    Aside: MkDocs has a new chat room on Gitter/Matrix. More details.

    Improvements:

    • Built-in themes now also support these languages:

    • Third-party plugins will take precedence over built-in plugins with the same name (#2591)

    • Bugfix: Fix ability to load translations for some languages: core support (#2565) and search plugin support with fallbacks (#2602)

    • Bugfix (regression in 1.2): Prevent directory traversal in the dev server (#2604)

    • Bugfix (regression in 1.2): Prevent webserver warnings from being treated as a build failure in strict mode (#2607)

    • Bugfix: Correctly print colorful messages in the terminal on Windows (#2606)

    • Bugfix: Python version 3.10 was displayed incorrectly in --version (#2618)

    Other small improvements; see commit log.

    1.2.2

    MkDocs 1.2.2 is a bugfix release for MkDocs 1.2 -- make sure you've seen the "major" release notes as well.

    • Bugfix (regression in 1.2): Fix serving files/paths with Unicode characters (#2464)

    • Bugfix (regression in 1.2): Revert livereload file watching to use polling observer (#2477)

      This had to be done to reasonably support usages that span virtual filesystems such as non-native Docker and network mounts.

      This goes back to the polling approach, very similar to that was always used prior, meaning most of the same downsides with latency and CPU usage.

    • Revert from 1.2: Remove the requirement of a site_url config and the restriction on use_directory_urls (#2490)

    • Bugfix (regression in 1.2): Don't require trailing slash in the URL when serving a directory index in mkdocs serve server (#2507)

      Instead of showing a 404 error, detect if it's a directory and redirect to a path with a trailing slash added, like before.

    • Bugfix: Fix gh_deploy with config-file in the current directory (#2481)

    • Bugfix: Fix reversed breadcrumbs in "readthedocs" theme (#2179)

    • Allow "mkdocs.yaml" as the file name when '--config' is not passed (#2478)

    ... (truncated)

    Commits
    • d167eab Release 1.2.3 (#2614)
    • 5629b09 Re-format translation files to pass a lint check (#2621)
    • 2c4679b Re-format translation files to pass a lint check (#2620)
    • 9262cc5 Fix the code to abbreviate Python's version (#2618)
    • 8345850 Add hint about -f/--config-file in configuration documentation (#2616)
    • 815af48 Added translation for Brazilian Portuguese (#2535)
    • 6563439 Update contact instructions: announce chat, preference for issues (#2610)
    • 6b72eef We can again announce support of zh_CN locale (#2609)
    • b18ae29 Drop assert_mock_called_once compat method from tests (#2611)
    • 7a27572 Isolate strict warning counter to just the ongoing build (#2607)
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Re-evaluate the RPKI state when new routing advertisements come in

    Re-evaluate the RPKI state when new routing advertisements come in

    Have Artemis re-evaluate the RPKI state when new routing advertisements come in and potentially clear the RPKI INVALID status in the alerts. Similar to how Artemis can change the status of an alert from Ongoing to Withdrawn.

    One example of this is where we had the prefix 76.178.98.0/23 advertised but Artemis had the config to monitor for 76.178.64.0/18. This generated an alert and was RPKI invalid. A ROA was then created and 76.178.98.0/23 was added to the Artemis config. It would have been nice if the alert would have cleared and shown RPKI valid image

    opened by racompton 2
Releases(2.3.0)

  • 2.3.0(Nov 24, 2022)

    Added

    • SSO support with Google SSO PoC (frontend)
    • accessibility (frontend)
    • firebase notifications (for mobile applications)
    • every API call now has a rate limit. The limit is per API endpoint, so it is not a global counter. The default value is 20 requests per 15 minutes. The user has to define 2 extra environmental variables:
      • LIMIT_WINDOW specifies the time window in ms.
      • LIMIT_REQUESTS specifies the number of allowed requests in that secific time window.
    • GRIP API integration. When there is 1 or more GRIP events related to the ARTEMIS event, a dropdown menu with the event ids is added to the hijack page. When clicked, a new tab to grip event page is opened.
    • signup and Login forms now include a captcha field for bot protection. The user has to additionally define the following .env variables:
      • CAPTCHA_SECRET. This would preferably be a long random hash value.
      • CAPTCHA_WINDOW specifies the time where the false login attempt count is valid.
      • CAPTCHA_TRIES is the number of unsuccessful login attempts that are needed to trigger the CAPTCHA generation.
    • new env var ARTEMIS_WEB_BASE_DIR specifies the ARTEMIS web base directory (default: empty).
    • condition to limit GRIP events association to 1 hour time window w.r.t. ARTEMIS alerts.
    • a web command line search experience based on super-cmd. To trigger the search modal:
      • Windows + Linux OS: ctrl + Windows key (super) + k
      • MacOS : cmd + k
    • logout alerts (frontend)
    • live button in dashboard page (frontend)
    • 401 notification for unauthorized users (frontend)
    • visual testing on cypress (frontend)
    • install_pybgpstreamv2_for_ubuntu.sh utility for pybgpstrem installation

    Changed

    • upgraded NGINX to 1.23
    • upgraded RabbitMQ to 3.9.20
    • (breaking) upgraded timescale DB to 2.8.1-pg14
    • upgraded postgrest to v10
    • upgraded hasura GraphQL engine v2.10.0
    • changed bgpstream_retrieve_prefix_records.py to support BGPStream v2 (new version)
    • enforce ssl_protocols TLSv1.2 TLSv1.3 in default nginx.conf
    • bumped caida/bgpstream to 2.2.0 and monitor containers to related 1.0.3
    • migrated our web application to latest Nextjs, React and Webpack
    • updated artemis-utils to 1.0.14 (service name fix for docker-compose)
    • password policy change
    • increased size limit for POST /api/config to host larger configuration files
    • removed process parallelism for internal reconfiguration tasks for thread-safety
    • default frontend password for admin

    Fixed

    • Split prefixtree recalculation indicator and fixed prefix trie bug with v4/v6
    • LDAP behavior and correct vars (frontend)
    • Admin login bug (frontend)
    • Bug with logout warning popup
    • GQL ports in frontend
    • login and JWT auth calls and documentation
    • LDAP button is hidden when no LDAP service is being used (frontend)
    • tooltips bug (frontend)
    • timezone and timestamps bug (frontend)
    • pending users treatment
    • hijack time started timestamp fix (frontend)
    • broken hijack URL in logs

    Backwards Incompatible Changes

    :rotating_light: Since we migrated to a new version of the backend database, you will need to do the following process to upgrade ARTEMIS:

    • Deactivate current running instance: docker-compose -f ... down
    • Stash any local changes that should not conflict with upstream: git stash
    • Checkout the master branch git checkout master
    • Pull most recent code (including .env, versions, etc.) git pull origin master
    • (IMPORTANT) Delete old DB data sudo rm -r postgres-data-current && sudo rm -r postgres-data-backup
    • Pop local changes and resolve conflicts by preferring upstream: git stash pop
    • Pull latest containers docker-compose -f ... pull
    • Start ARTEMIS! docker-compose -f ... up -d
    Source code(tar.gz)
    Source code(zip)

  • 2.1.0(May 17, 2021)

    Added

    • new frontend (React) in new repo and container
    • modular component-oriented UI
    • CSRF protection
    • improved LDAP support
    • responsive layout and window scaling fix
    • user feedback when tables are not properly rendered
    • allow creation of new user from admin user management page
    • admins can now set user's password
    • correct auto-logout
    • configurable session and inactivity timeout in .env
    • "json" encoding accepted for messages coming from frontend (ignore/resolve/seen/delete/(un-)mitigate)
    • frontend REST API (OpenAPI-swagger)

    Changed

    • changes in "dataplane_msms" table and "view_dataplane_msms" view, in order to support the new design of the "dataplane_view" module.
    • improved server-side auth
    • upgraded artemis-utils to 1.0.10 to include the slacker-log-handler==1.7.1 dep
    • migrating from travis to GH actions
    • downgraded to six==1.11.0 to achieve compatibility
    • changed tap invalid format logging level to debug (instead of warning)
    • updated artemis-utils to 1.0.11
    • updated k8s documentation
    • updated artemis-chart
    • replaced bcrypt with argon2

    Fixed

    • autoconfiguration subprefix bug in prefixtree plus new autoconf tests
    • vagrant docker-compose.yaml file fix (sync with master)
    • session timeout behavior

    Deprecated

    • old frontend (Flask, custom JS)

    Security

    • updated PyYAML to 5.4 (fixing sec vulnerabilitiy)

    Backwards Incompatible Changes

    :rotating_light: You can follow the classic update process for the transition to the latest version, however note that the new frontend requires the following changes that are not backwards compatible:

    Deprecation of initial frontend user DB

    We have migrated to mongoDB from sqlite. Due to security implications for migrating salted hashed passwords between the two databases based on the algorithms we used initially and the ones we use currently, we opted for simply deprecating the sqlite DB. Locally managed users will have to be recreated per instance. However, to ease this process, the new frontend now provides admins with the capability to create new users and set their password accordingly.

    New and deleted env variables

    • CONFIG_HOST=configuration [NEW, NO ACTION REQUIRED]
    • CONFIG_PORT=3000 [NEW, NO ACTION REQUIRED]
    • DATABASE_HOST=database [NEW, NO ACTION REQUIRED]
    • BIND_IP=0.0.0.0 [DELETED]
    • WEBAPP_PORT=4200 [CORRECT VALUE CHANGED FROM 8000 to 4200]
    • ADMIN_USER=admin [DELETED, NOW USING ONLY EMAIL]
    • JS_VERSION=1.0.4 [DELETED]
    • GUNICORN_WORKERS=4 [DELETED]
    • SESSION_TIMEOUT=1800 [NEW]
    • INACTIVITY_TIMEOUT=900 [NEW]
    • MONGODB_USER=admin [NEW]
    • MONGODB_PASS=pass [NEW, SECRET]
    • MONGODB_HOST=mongodb [NEW]
    • MONGODB_PORT=27017 [NEW]
    • MONGODB_NAME=artemis-web [NEW]
    • LDAP_ENABLED=true [NEW]
    • LDAP_HOST=ldap [NEW]
    • LDAP_PORT=10389 [NEW]
    • LDAP_PROTOCOL=ldap [NEW]
    • LDAP_BIND_DN="cn=admin,dc=planetexpress,dc=com" [NEW]
    • LDAP_BIND_SECRET="GoodNewsEveryone" [NEW, SECRET]
    • LDAP_SEARCH_BASE="ou=people,dc=planetexpress,dc=com" [NEW]
    • LDAP_SEARCH_FILTER="(mail={{username}})" [NEW]
    • LDAP_SEARCH_ATTRIBUTES="mail, uid" [NEW]
    • LDAP_EMAIL_FIELDNAME=mail [NEW]
    • LDAP_ADMIN_GROUP=admin_staff [NEW]
    • LDAP_USER_GROUP= [NEW]
    • FLASK_SECRET_KEY [DELETED]
    • SECURITY_PASSWORD_SALT [DELETED]
    • CSRF_SECRET [NEW, SECRET]
    • API_KEY [NEW, SECRET]

    Deprecated/removed files

    • Entire frontend code section. The current code is in the mono-repo https://github.com/FORTH-ICS-INSPIRE/artemis-web .
    • local_configs/frontend/webapp.cfg
    • local_configs/frontend/config.py
    • local_configs/frontend/logging.yaml
    • local_configs/frontend/__init__.py
    • frontend/db

    docker-compose.yaml

    • Heavy revisions in frontend section to sync with new env variables. Please make sure you properly resolve conflicts with your on-premise ARTEMIS instance.
    • Addition of mongodb section, user data is now kept at the mapped ./mongo_data folder.
    Source code(tar.gz)
    Source code(zip)

  • 2.0.0(Jan 26, 2021)

    To upgrade, please follow the guide here and pay attention to the important 2.0.0 upgrade instructions.

    Added

    • Action and necessary logic to un-mitigate a hijack
    • Ability to reload targeted services based on what conf section changed
    • GET config endpoints to backend and monitor services
    • Makefile for local commands (dev/ci)
    • Added healthchecks to docker-compose (docker-compose version bumped to 3.4)
    • Service reconfiguring flag and status
    • Redis no-data-received key expiry logic (default: 60 seconds)
    • Hijack detail JSON export

    Changed

    • Decoupled microservice architecture for backend and frontend
    • Possible states constraint on DB hijacks table to allow orthogonal mitigation tagging
    • Orthogonal mitigation/unmitigation tagging
    • Heavily refactored docs
    • Replaced bulk timer thread with separate process
    • Replaced autoignore checker thread with separate process
    • Replaced exabgp autoconf thread with separate process
    • Display info on new microservices, grouping backend/monitor
    • Upgraded requests to 2.25.1
    • Upgraded to new bgpstream containers (using also ris and routeviews live sources)
    • Full refactoring and upgrade of artemis-utils, refactored all imports (1.0.7)
    • Upgraded Flask-JWT-Extended to 3.25.0 to address JWT issue with new browsers
    • Upgraded artemis-utils to 1.0.8

    Fixed

    • Discard messages with timestamps in future datetime (updated to artemis-utils==1.0.5)
    • Check for very old timestamps in DB update handling
    • Replaced sets that are not supported by shared manager with lists
    • Removed unneeded read locks
    • Non-gracefull SIGKILL for taps in case graceful stop delays a lot
    • Redis configuration file input

    Deprecated

    • 'caida' project in bgpstreamlive
    • supervisor support and configuration

    Notes:

    1. if you encounter issues with detection essentially getting "stuck" (see https://github.com/FORTH-ICS-INSPIRE/artemis/issues/564) please comment under this issue and give us a heads-up in Slack. This was a bug we encountered only in a certain local deployment and we would like to double-check with the users whether it is an actual problem. If it is, please consult the issue workaround comment and we will patch on demand.
    2. If you see any redis-related errors probably you have not set the configuration file of redis (new 2.0.0 addition correctly). Just run:
    cp backend-services/configs/redis.conf local_configs/backend/redis.conf

    and restart artemis.

    Source code(tar.gz)
    Source code(zip)

  • 1.6.0(Nov 2, 2020)

    To upgrade, follow the guide here.

    Added

    • Redirection to login page when session is expired (#404)
    • Support RADIUS integration for user authentication (#409)
    • Support for GRAPHQL_URI to be set via environment variables, or to take a lazy approach and set HASURA_HOST and HASURA_PORT env variables
    • Flag to override BGPStream timestamp (#415)
    • Hijack alert auto-ignore mechanism for alerts of limited impact/visibility (#373)
    • Env variable to invoke intended process states recovery mechanism
    • Reinstated RPKI tests for PRs
    • Implicit withdrawals are now generated also on correct (direct) subprefix routing
    • Support for fork PRs in travis CI/CD builds and tests
    • New dataplane-related table and view (pings and traceroutes)
    • PoC mitigation mechanism and setup (deaggregation)
    • Coverage for unit tests
    • AS Path pre-pending sequence matching support in configuration
    • Docs pages

    Changed

    • Refactor: Changed exchange / queue names
    • Heavily refactored and improved exaBGP-based auto-configuration for speed/scale

    Fixed

    • Deleting user also deletes the role mapping (#412)
    • Backend Makefile to use currently installed python version
    • Frontend Makefile to use specific closure compiler version
    • Minor opt fix in Redis RTT calls in DB
    • Observer fix (RMQ connection establishment) to deal with threading issues

    Removed

    • Autoconf support in historical BGPStream (due to deprecation)

    Deprecated

    • Wiki pages
    Source code(tar.gz)
    Source code(zip)

  • 1.5.0(Jun 3, 2020)

    Added

    • Additional ports (8080, 8443) to nginx configuration for rootless docker support
    • Application names for all postgres connections
    • Database migration check between latest release and current version
    • Mechanism to show which modules are (re)loading upon configuration change
    • GUNICORN_WORKERS and RABBITMQ_IO_THREAD_POOL_SIZE variables smaller deployments
    • ZAP Active and Passive scanning
    • Select Page button on hijacks page to select all hijacks on current page

    Changed

    • Upgraded nginx to latest release (v1.17)
    • Upgrade rabbitmq to latest release (v3.8.3)
    • Upgrade postgrest to latest release version (v7.0.0)
    • Upgraded hasura-engine to v1.1.0
    • Updated version of BGPStream on custom image (mavromat/bgpstream-redis:v1.3-bgpstream)

    Fixed

    • LIMIT query planner bug (http://datamangling.com/2014/01/17/limit-1-and-performance-in-a-postgres-query/)
    • Added stderr output of supervisor processes on docker-compose logs (Please update your custom supervisor confs if using local_configs)
    • Issue with multiple rules for same prefix(es) in detection
    • Update RIPE Stat API call for ASN to country mapping
    • Minor issue with deprecated version of routinator (upgraded to 0.6.4)
    • Observer ujson encoding bug

    Security

    • Secured "proxy_api" endpoint from unauthorized access. Thanks to @marcgweg for finding this issue
    Source code(tar.gz)
    Source code(zip)

  • 1.4.0(Dec 2, 2019)

    This release includes the following notable changes:

    • Important monitoring + detection performance boost
    • Grafana chart support
    • Passive autoconfiguration via exaBGP or bgpstreamhist, via "autoconf" flag
    • ENV variable "WITHDRAWN_HIJACK_THRESHOLD" to consider hijacks withdrawn based on a withdrawn monitor percentage threshold
    • TravisCI and CodeCov support (replaced SemaphoreCI and coveralls)
    • Database fixes and optimizations
    • Replaced betabmp client with "caida" project within bgpstreamlive
    • Added bgpstreamkafka tap (can be used for private BMP feeds)
    • RPKI status in hijack alerts; can be enabled via ENV variables
    • Optional routinator RPKI validator microservice
    • Automatic YAML config file formatting upon config modification
    • ENV vars "DB_AUTOCLEAN" and "DB_HIJACK_DORMANT" set to 1 day (24h) by default
    • ARTEMIS recovery upon server rebooting
    • Fixed important RabbitMQ ack issue with messages
    Source code(tar.gz)
    Source code(zip)

  • 1.3.1(Oct 4, 2019)

    Note for this version: please take extra care since we have introduced the following backwards-incompatible changes (logging and Kubernetes):

    • Changed frontend logger from webapp_logger to artemis_logger in logging.yaml and config.py under frontend/webapp/configs. For old versions you need to replace the name from webapp_logger to artemis_logger in these two files under your local_configs/frontend folder.
    • Changed helm chart labels to support multiple releases per namespace and this breaks helm upgrades. To migrate to the new version you need to deploy a new installation and migrate the Persistent Volumes Claims (PVC) to the new release by attaching them on the current Persistent Volumes.

    This release includes the following changes:

    Added

    • IPv6 tests (backend testing)
    • PR labeler (GitHub actions)
    • Env variable and formatter for selecting hijack log fields
    • Env variable for ARTEMIS web host for backend logging
    • Logging also withdrawn and outdated hijacks in mail and hijack loggers
    • Extra end_tag, outdated_parent and hijack_url in hijacks for logging
    • Reinstating intended modules on ARTEMIS startup
    • Support of multiple artemis instances on a single namespace (k8s)
    • Added benchmarker and PR commenter

    Changed

    • py-radix, substituted with pytricia tree
    • dep-licenses updated (with currently used modules)
    • Fixed bug with outdated hijacks that remain hijacks
    • Fixed bug with BGP updates related to more than one hijacks
    • Added outdated_parent logging field to inform logging systems

    Fixed

    • Support for millions of prefixes in configuration file
    • Ensure ARTEMIS config file is not clobbered on pod restart

    Removed

    • ipaddress requirement from frontend (not needed)
    Source code(tar.gz)
    Source code(zip)

  • 1.3.0(Sep 12, 2019)

    This release includes the following changes:

    Added

    • Slack logging package and example
    • ARTEMIS logo
    • Monitor peers count in stats table (overview)
    • Implicit withdrawals generated for benign BGP updates correcting hijacked peer-prefix combinations
    • Support for configured ASN ranges
    • Community-based annotation support for annotating hijacks
    • Local pre-commit hook to check versioning
    • Bgpstream historical BGP record collector (per prefix)
    • Support for RIPE whois service-resolved AS-SETs in configuration
    • Ingress resource for helm-chart (by default supports NGINX Ingress Controller)
    • Redis ping/pong check on connection initialization
    • In hijack page count the number of peers seen announcement and withdrawal

    Changed

    • Refactoring frontend (views, templates and static files are organized inside the folder render)
    • Update hasura (1.0.0alpha42 -> 1.0.0alpha45)
    • Replaced $.ajax with fetch
    • Hijack logger output is now a JSON string
    • Decoupled monitor module to another container
    • Optional deployment of NGINX container or NGINX Ingress on Kubernetes

    Fixed

    • Bug with rule learning (hijack to rule dict) when empty neighbor
    • Minor fix in custom mitigation trigger
    • Updated configurations for k8s deployment
    • Fixed flake8 warnings
    • Wrong data.sql for testing
    • Bug with single-quoted communities being interpreted as sexagesimal upon automated conf reload
    • Monitoring of monitors and restarting them if not seen any BGP updates within user-defined interval
    • Avoid detectors being stuck on blpop if token erased
    • Fix for historical replay of withdrawals plus bgpstreamhist rate-limit
    • Minor: creating implicit withdrawals 1 second after the related benign announcement
    • Minor: in handling BGP withdrawals, consider announcements that are strictly newer than the withdrawals
    • Various helm (k8s) chart fixes and improvements (see #221)

    Security

    • Bumped SQLAlchemy from 1.2.16 to 1.3.3 in /frontend

    You can view a detailed list of changes in the CHANGELOG (https://github.com/FORTH-ICS-INSPIRE/artemis/blob/master/CHANGELOG.md). You can follow the wiki guide here (https://github.com/FORTH-ICS-INSPIRE/artemis/wiki#upgrading-artemis-to-a-new-version) to upgrade to the new version.

    Note for this version: please take extra care following the wiki upgrade guide in order to properly build/alter the local_configs to include the new monitor container configuration. Make sure you check docker-compose.yaml and make sure that the local configs are properly mapped.

    Source code(tar.gz)
    Source code(zip)

  • 1.2.0(Apr 10, 2019)

    This release includes the following changes:

    Added

    • Support for dormant flags in hijacks
    • Storing hijack update (origin, neighbor) combinations in redis
    • Learn rule action for ongoing hijacks in frontend after ignore action
    • Misc UI additions
    • Initial kubernetes/helm (helm-charts) support
    • Reject old updates from taps and have a "HISTORIC" variable to enable/disable
    • Initial support for LDAP authentication
    • Hijack deletion functionality
    • Abuse contact details for each ASN (Extracted from RIPEStat)
    • Functionality to copy ASN details on clickboard
    • Support to filter BGP Updates based on their AS Path
    • Display distinct values of BGP Updates for the following fields: "Origin AS", "Peer AS" and "service" in hijack view

    Changed

    • Using prefix lists in json file format as monitoring taps input to avoid problematic ultra long arguments
    • Refactored environment variables
    • Use of RIPE RIS firehose stream instead of the websocket clientui8
    • Misc UI improvements
    • Update hasura (1.0.0alpha31 -> 1.0.0alpha42)
    • Hijack view now uses hasura graphql to fetch BGP Updates

    Fixed

    • Correct RFC2622 translation when needed in frontend and backend
    • When learning ignore rule, escape special character ":" (IPv6)
    • Problematic start of RIS and exaBGP monitors, even if not configured
    • BGP update redis bootstraping from DB
    • UI support for multiple instances of a module in overview and system page
    • Misc UI bugs

    Removed

    • Configured prefix graph visualization (needs redesign)
    • Config data field from configs DB table

    Security

    • Using yaml dump and safe_load instead of pickling/unpickling

    You can view a more detailed list of changes in the CHANGELOG (https://github.com/FORTH-ICS-INSPIRE/artemis/blob/master/CHANGELOG.md). You can follow the wiki guide here (https://github.com/FORTH-ICS-INSPIRE/artemis/wiki#upgrading-artemis-to-a-new-version) to upgrade to the new version. Note: this release requires a DB migration, which is automatically handled by the system.

    Source code(tar.gz)
    Source code(zip)

  • 1.1.1(Feb 28, 2019)

    Added

    • Tooltip support for peers seen BGP Announcement/Withdrawal on hijack view.
    • Support for rfc2622 ^+, ^-, ^n and ^n-m prefix operators in configuration
    • Coverage tests for frontend actions
    • Pre-commit hooks
    • Policy dimension on hijacks (route leak detection based on no-export)
    • Support for auto-cleaning unneeded BGP updates
    • Automated DB backups
    • View hijack by key
    • Enable sorting for columns: # Peers Seen/# ASes Infected
    • Added DB version on overview page

    Changed

    • Testing refactoring
    • RIPE RIS live python websocket client

    Fixed

    • Updated/optimized db query for removing withdrawn peers (newer announcement)
    • Support for different user/pass on rabbitmq
    • Solved bug with randomized config hashing
    • Fixed expected behavior when trying to run old containers on new DBs

    You can view a more detailed list of changes in the CHANGELOG (https://github.com/FORTH-ICS-INSPIRE/artemis/blob/master/CHANGELOG.md). You can follow the wiki guide here (https://github.com/FORTH-ICS-INSPIRE/artemis/wiki#upgrading-artemis-to-a-new-version) to upgrade to the new version. Note: this release requires a DB migration, which is automatically handled by the system.

    Source code(tar.gz)
    Source code(zip)

  • 1.1.0(Feb 20, 2019)

    This release includes the following important changes:

    Added:

    • CI/CD container
    • SemaphoreCI testing for backend
    • Automation of system and DB migration
    • Multi-process Database support through supervisor
    • Custom monitor for high-throughput measuring
    • Support for wildcards (origin_asns, neighbors) in configuration
    • Enabled POST request on /jwt/auth to retrieve authentication token

    Changed:

    • Moved static js libraries to CDN
    • Upgraded requirements in frontend
    • Upgraded requirements in backend
    • Moved js minifier to container builder
    • Revised detection logic to account for hijack dimensions
    • Using the Seen/Acknowledged to confirm true or false hijack

    Fixed:

    • Misc code quality improvements and bug-fixes
    • Fetch API support for older browsers

    You can view a more detailed list of changes in the CHANGELOG (https://github.com/FORTH-ICS-INSPIRE/artemis/blob/master/CHANGELOG.md).

    You can follow the wiki guide here (https://github.com/FORTH-ICS-INSPIRE/artemis/wiki#upgrading-artemis-to-a-new-version) to upgrade to the new version. Note: this release requires a DB migration, which is automatically handled by the system.

    Source code(tar.gz)
    Source code(zip)

  • 1.0.0(Dec 20, 2018)

    First release of the ARTEMIS real-time open-source tool against BGP prefix hijacking. This release introduces the following features:

    • Real-time monitoring of the changes in the BGP routes of the prefixes originated by the AS running ARTEMIS.
    • Real-time detection and notifications of BGP prefix hijacking attacks/events of the following types (please refer to the attack taxonomy in our ARTEMIS ToN paper):
      • exact-prefix type-0/1
      • sub-prefix of any type
      • squatting attacks.
    • Automatic/custom tagging of detected BGP hijack events (ongoing, resolved, ignored, under mitigation, withdrawn and outdated).
    • Manual or manually controlled mitigation of BGP prefix hijacking attacks.
    • Comprehensive web-based User Interface (UI).
    • Configuration file editable by the operator (directly or via the UI), containing information about: prefixes, ASNs, monitors and ARTEMIS rules ("ASX originates prefix P and advertises it to ASY").
    • Support for both IPv4 and IPv6 prefixes.
    • Support for both mobile and desktop environments (UI).
    • Modularity/extensibility by design.
    Source code(tar.gz)
    Source code(zip)
Owner
INSPIRE Group @FORTH-ICS
Internet Security Privacy and Intelligence Research Group
INSPIRE Group @FORTH-ICS
GitHub Repository
Nexum is an open-source, remote administration tool written in Python 3

A full-featured remote administration tool written in Python 3. The goal of this project is to make the use of a remote administration tool as simple

z3phyrus 2 Nov 26, 2021
This is a simple python code to get the list of banned IP addresses from Fail2ban

Fail2ban Scripts Usage banned_list.py This script tries to get the banned list of IP addresses by Fail2ban for the service freeswitch. You can modify

Yehor Smoliakov 9 Dec 28, 2022
Usbkill - an anti-forensic kill-switch that waits for a change on your USB ports and then immediately shuts down your computer.

Usbkill - an anti-forensic kill-switch that waits for a change on your USB ports and then immediately shuts down your computer.

Hephaestos 4.1k Dec 30, 2022
msgspec is a fast and friendly implementation of the MessagePack protocol for Python 3.8+

msgspec msgspec is a fast and friendly implementation of the MessagePack protocol for Python 3.8+. In addition to serialization/deserializat

Jim Crist-Harif 414 Jan 06, 2023
Godzilla traffic decoder Godzilla Decoder 是一个用于 哥斯拉Godzilla 加密流量分析的辅助脚本。

Godzilla Decoder 简介 Godzilla Decoder 是一个用于 哥斯拉Godzilla 加密流量分析的辅助脚本。 Godzilla Decoder 基于 mitmproxy，是mitmproxy的addon脚本。 目前支持 哥斯拉3.0.3 PhpDynamicPayload的

He Ruiliang 40 Dec 25, 2022
EV: IDS Evasion via Packet Manipulation

EV: IDS Evasion via TCP/IP Packet Manipulation 中文文档 Introduction EV is a tool that allows you crafting TCP packets and leveraging some well-known TCP/

256 Dec 08, 2022
The AKS cluster provisioner provisions AKS clusters :-)

Overview The AKS cluster provisioner provisions AKS clusters :-) It uses the Azure CLI to configure VNet and subnets before creating the cluster itsel

Gigi Sayfan 1 Nov 10, 2021
Web service load balancing simulation experiment.

Web service load balancing simulation experiment.

NicestZK 1 Nov 12, 2021
ExtDNS synchronizes labeled records in docker-compose with DNS providers.

ExtDNS for docker-compose ExtDNS synchronizes labeled records in docker-compose with DNS providers. Inspired by External DNS, ExtDNS makes resources d

DNTSK 6 Dec 24, 2022
Utility for converting IP Fabric webhooks into a Teams format.

IP Fabric Webhook Integration for Microsoft Teams and/or Slack Setup IP Fabric Setup Go to Settings Webhooks Add webhook Provide a name URL will b

Community Fabric 1 Jan 26, 2022
Script and library to wait for a DNS authority server to get its configuration.

DNSWait dnswait is a small script to wait for the "propagation" of a namserver configuration. Installing It's as easy as: python -m pip install dnswai

Julien Palard 14 Jan 17, 2022
Socialhome is best described as a federated personal profile with social networking functionality

Description Socialhome is best described as a federated personal profile with social networking functionality. Users can create rich content using Mar

Jason Robinson 332 Dec 30, 2022
Wifijammer - Continuously jam all wifi clients/routers

wifijammer Continuously jam all wifi clients and access points within range. The effectiveness of this script is constrained by your wireless card. Al

Dan McInerney 3.5k Dec 31, 2022
a safe proxy over tls

TlsProxys 基于TLS协议的http流量代理 安装 ※ 需要python3.7+ linux: python3.9 -m pip install TlsProxys windows: pip install TlsProxys 基本用法 服务器端: $ tpserver [command]

56 Nov 30, 2022
Home Assistant integration for MyEnergi devices

myenergi for Home Assistant myenergi custom component for Home Assistant This is a very early release, will add more documentations soon! This compone

Johan Isacsson 70 Dec 18, 2022
Automated network configuration backups using Github actions and git-scraping

Network Config Scraper This repository demonstrates the use of Github Actions and git-scraping to build an automated backup solution for network confi

WWT 19 Dec 14, 2022
Simple app that redirect fixed URL to changing URL, configurable via POST requests

This is a basic URL redirection service. It stores associations between apps and redirection URLs, for apps with changing URLs. You can then use GET r

Maxime Weyl 2 Jan 28, 2022
This application aims to read all wifi passwords and visualizes the complexity in graph formation by taking into account several criteria and help you generate new random passwords.

This application aims to read all wifi passwords and visualizes the complexity in graph formation by taking into account several criteria and help you generate new random passwords.

Njomza Rexhepi 0 May 29, 2022
Openconnect VPN RPi Gateway

Openconnect-VPN-RPi-Gateway See the blog (Chinese) for how to build an Openconne

Zhongze Tang 2 Jan 30, 2022
A repository dedicated to IoT(internet of things ) and python scripts

📑 Introduction Week of Learning is a weekly program in which you will get all the necessary knowledge about Circuit-Building, Arduino and Micro-Contr

27 Nov 22, 2022