EODAG is a command line tool and a plugin-oriented Python framework for searching, aggregating results and downloading remote sensed images while offering a unified API for data access regardless of the data provider

Overview
https://eodag.readthedocs.io/en/latest/_static/eodag_bycs.png

https://img.shields.io/conda/vn/conda-forge/eodag https://readthedocs.org/projects/eodag/badge/?version=latest&style=flat


Checkout the new Jupyterlab extension: eodag-labextension! This will bring a fiendly UI to your notebook and help you search and browse for EO products using eodag.

EODAG (Earth Observation Data Access Gateway) is a command line tool and a plugin-oriented Python framework for searching, aggregating results and downloading remote sensed images while offering a unified API for data access regardless of the data provider. The EODAG SDK is structured around three functions:

  • List product types: list of supported products and their description
  • Search products (by product type or uid) : searches products according to the search criteria provided
  • Download products : download product “as is"

EODAG is developed in Python. It is structured according to a modular plugin architecture, easily extensible and able to integrate new data providers. Three types of plugins compose the tool:

  • Catalog search plugins, responsible for searching data (OpenSearch, CSW, ...), building paths, retrieving quicklook, combining results
  • Download plugins, allowing to download and retrieve data locally (via FTP, HTTP, ..), always with the same directory organization
  • Authentication plugins, which are used to authenticate the user on the external services used (JSON Token, Basic Auth, OAUTH, ...).

Since v2.0 EODAG can be run as STAC client or server.

Read the documentation for more insights.

EODAG overview

Installation

EODAG is available on PyPI:

python -m pip install eodag

And with conda from the conda-forge channel:

conda install -c conda-forge eodag

Usage

For downloading you will need to fill your credentials for the desired providers in your eodag user configuration file. The file will automatically be created with empty values on the first run.

Python API

Example usage for interacting with the api in your Python code:

from eodag import EODataAccessGateway

dag = EODataAccessGateway()

search_results, total_count = dag.search(
    productType='S2_MSI_L1C',
    geom={'lonmin': 1, 'latmin': 43.5, 'lonmax': 2, 'latmax': 44}, # accepts WKT polygons, shapely.geometry, ...
    start='2021-01-01',
    end='2021-01-15'
)

product_paths = dag.download_all(search_results)

This will search for Sentinel 2 level-1C products on the default provider and return the found products first page and an estimated total number of products matching the search criteria. And then it will download these products. Please check the Python API User Guide for more details.

STAC REST API

An eodag installation can be exposed through a STAC compliant REST api from the command line:

$ eodag serve-rest --help
Usage: eodag serve-rest [OPTIONS]

  Start eodag HTTP server

Options:
  -f, --config PATH   File path to the user configuration file with its
                      credentials
  -d, --daemon TEXT   run in daemon mode
  -w, --world         run flask using IPv4 0.0.0.0 (all network interfaces),
                      otherwise bind to 127.0.0.1 (localhost). This maybe
                      necessary in systems that only run Flask  [default:
                      False]
  -p, --port INTEGER  The port on which to listen  [default: 5000]
  --debug             Run in debug mode (for development purpose)  [default:
                      False]
  --help              Show this message and exit.

# run server
$ eodag serve-rest

# list available product types for ``peps`` provider:
$ curl "http://127.0.0.1:5000/collections?provider=peps" | jq ".collections[].id"
"S1_SAR_GRD"
"S1_SAR_OCN"
"S1_SAR_SLC"
"S2_MSI_L1C"
"S2_MSI_L2A"
"S3_EFR"
"S3_ERR"
"S3_LAN"
"S3_OLCI_L2LFR"
"S3_OLCI_L2LRR"
"S3_SLSTR_L1RBT"
"S3_SLSTR_L2LST"

# search for items
$ curl "http://127.0.0.1:5000/search?collections=S2_MSI_L1C&bbox=0,43,1,44&datetime=2018-01-20/2018-01-25" \
| jq ".context.matched"
6

# browse for items
$ curl "http://127.0.0.1:5000/S2_MSI_L1C/country/FRA/year/2021/month/01/day/25/cloud_cover/10/items" \
| jq ".context.matched"
9

# get download link
$ curl "http://127.0.0.1:5000/S2_MSI_L1C/country/FRA/year/2021/month/01/day/25/cloud_cover/10/items" \
| jq ".features[0].assets.downloadLink.href"
"http://127.0.0.1:5000/S2_MSI_L1C/country/FRA/year/2021/month/01/day/25/cloud_cover/10/items/S2A_MSIL1C_20210125T105331_N0209_R051_T31UCR_20210125T130733/download"

# download
$ wget "http://127.0.0.1:5000/S2_MSI_L1C/country/FRA/year/2021/month/01/day/25/cloud_cover/10/items/S2A_MSIL1C_20210125T105331_N0209_R051_T31UCR_20210125T130733/download"

You can also browse over your STAC API server using STAC Browser. Simply run:

git clone https://github.com/CS-SI/eodag.git
cd eodag
docker-compose up
# or for a more verbose logging:
EODAG_LOGGING=3 docker-compose up

And browse http://127.0.0.1:5001:

STAC browser example

For more information, see STAC REST API usage.

Command line interface

Start playing with the CLI:

  • To search for some products:

    eodag search --productType S2_MSI_L1C --box 1 43 2 44 --start 2021-03-01 --end 2021-03-31
    

    The request above searches for S2_MSI_L1C product types in a given bounding box, in March 2021. It saves the results in a GeoJSON file (search_results.geojson by default).

    Results are paginated, you may want to get all pages at once with --all, or search products having 20% of maximum coud cover with --cloudCover 20. For more information on available options:

    eodag search --help
    
  • To download the result of the previous call to search:

    eodag download --search-results search_results.geojson
    
  • To download only the result quicklooks of the previous call to search:

    eodag download --quicklooks --search-results search_results.geojson
    
  • To list all available product types and supported providers:

    eodag list
    
  • To list available product types on a specified supported provider:

    eodag list -p sobloo
    
  • To see all the available options and commands:

    eodag --help
    
  • To print log messages, add -v to eodag master command. e.g. eodag -v list. The more v given (up to 3), the more verbose the tool is. For a full verbose output, do for example: eodag -vvv list

Contribute

Have you spotted a typo in our documentation? Have you observed a bug while running EODAG? Do you have a suggestion for a new feature?

Don't hesitate and open an issue or submit a pull request, contributions are most welcome!

For guidance on setting up a development environment and how to make a contribution to eodag, see the contributing guidelines.

License

EODAG is licensed under Apache License v2.0. See LICENSE file for details.

Authors

EODAG has been created by CS GROUP - France.

Credits

EODAG is built on top of amazingly useful open source projects. See NOTICE file for details about those projects and their licenses. Thank you to all the authors of these projects !

Comments
  • progress bar with tqdm

    progress bar with tqdm

    Original report by Baptiste Meylheuc (Bitbucket: bmeylheuc, ).


    See to replace the direct use of tqdm by a python callback that allows to choose the progress bar type (Using tqdm_notebook allows to display the download progress on a single bar).

    enhancement 
    opened by sbrunato 36
  • Support OpenSearch keys in search interface

    Support OpenSearch keys in search interface

    Original report by Mickaël Savinaud (Bitbucket: savmickael, GitHub: savmickael).

    The original report had attachments: coll_def.ods


    The search interface must support OpenSearch geo extension :

    • box instead of geometry currently
    • geometry as WKT (perhaps limited to polygon type)
    • lat / lon and radius
    • name
    • relation is quite over-kill I think (intersects is default value)

    For temporal extension key :

    • start instead of startTimeFromAscendingNode
    • end instead of completionTimeFromAscendingNode
    • relation is quite over-kill I think (intersects is default value)

    OpenSearch Geo and Temporal extention : https://portal.opengeospatial.org/files/?artifact_id=56866

    enhancement 
    opened by sbrunato 17
  • Capella Bucket Download issue

    Capella Bucket Download issue

    Bucket Issue Hi there, I'm trying to download Capella's Open data from: https://www.stacindex.org/catalogs/capella-space-open-data#/BrVjRL9LGRjg8ipuaKLhdHr1kD4sbvyd8QTYGwbMTTtkR4fzPd4yeiApxyrQkiq/LGziSG3zruu42mUQkBqrXkRPLdBZyfQ9WBqmakLW8v6Q2jtwGXLNgDPJyeVfFR6XkZHa8qL5E75MA6d8vzhnxgNLQVUuVFNrq?t=2 But I got a "NoSuchKey" error from the "bucket_name" = "data"

    I tried unsuccesfull workarounds... If someone got an idea, it's very welcome.

    Thanks, and thanks a lot EOdag devs! Ari

    Code To Reproduce

    from eodag import setup_logging
    setup_logging(verbose=3)
    
    from eodag.api.core import EODataAccessGateway
    
    # Create an EODAG custom STAC provider
    dag = EODataAccessGateway()
    
    # Add the custom STAC provider + output + login an password
    catalog_path = "https://capella-open-data.s3.us-west-2.amazonaws.com/stac/capella-open-data-by-product-type/capella-open-data-slc/collection.json"
    # catalog_path = 'https://capella-open-data.s3.us-west-2.amazonaws.com/stac/catalog.json'
    base_uri = "https://capella-open-data.s3.us-west-2.amazonaws.com"
    outputs_prefix = r"D:\Radar\data\EODAG"
    aws_access_key_id = "change_me"
    aws_secret_access_key = "change_me"
    
    
    dag.update_providers_config("""
    capella:
        search:
            type: StaticStacSearch
            api_endpoint: %s
        products:
            GENERIC_PRODUCT_TYPE:
                productType: '{productType}'
        download:
            type: AwsDownload
            base_uri: %s
            flatten_top_dirs: True
            outputs_prefix: %s
        auth:
            type: AwsAuth
            credentials:
                aws_access_key_id: %s
                aws_secret_access_key: %s
    """ % (catalog_path, base_uri, outputs_prefix, aws_access_key_id, aws_secret_access_key ))
    
    # # Set the custom STAC provider as preferred
    dag.set_preferred_provider("capella")
    
    # Query every product from inside the catalog
    # all_products, _ = dag.search()
    
    capella_prod, _ = dag.search(start="2021-12-01", end="2021-12-15", **{"sar:product_type": "SLC", "sensorMode": "spotlight"})  # , locations=dict(country="BRA"), start="2021-01-01", end="2020-05-30", items_per_page=50)
    
    # [(key, asset["href"]) for key, asset in capella_prod[0].assets.items()]
    
    paths = dag.download_all(capella_prod)
    
    

    TroubleShoot

    2022-10-07 16:47:56,830 eodag.config                     [INFO    ] (config           ) Loading user configuration from: C:\Users\ajeannin\.config\eodag\eodag.yml
    2022-10-07 16:47:56,923 eodag.core                       [INFO    ] (core             ) usgs: provider needing auth for search has been pruned because no crendentials could be found
    2022-10-07 16:47:56,923 eodag.core                       [INFO    ] (core             ) aws_eos: provider needing auth for search has been pruned because no crendentials could be found
    2022-10-07 16:47:57,010 eodag.core                       [DEBUG   ] (core             ) Opening product types index in C:\Users\ajeannin\.config\eodag\.index
    2022-10-07 16:47:57,020 eodag.core                       [INFO    ] (core             ) Locations configuration loaded from C:\Users\ajeannin\.config\eodag\locations.yml
    2022-10-07 16:47:57,022 eodag.config                     [INFO    ] (config           ) capella: unknown provider found in user conf, trying to use provided configuration
    2022-10-07 16:47:57,110 eodag.core                       [INFO    ] (core             ) No product type could be guessed with provided arguments
    2022-10-07 16:47:57,686 eodag.core                       [INFO    ] (core             ) Searching product type 'None' on provider: capella
    2022-10-07 16:47:57,686 eodag.core                       [DEBUG   ] (core             ) Using plugin class for search: StaticStacSearch
    2022-10-07 16:47:58,682 eodag.utils.stac_reader          [DEBUG   ] (stac_reader      ) Fetching 125 items
    2022-10-07 16:47:58,683 eodag.utils.stac_reader          [DEBUG   ] (stac_reader      ) read_local_json is not the right STAC opener
    2022-10-07 16:48:00,570 eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Getting genric provider product type definition parameters for None
    2022-10-07 16:48:00,571 eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Building the query string that will be used for search
    2022-10-07 16:48:00,571 eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Retrieving queryable metadata from metadata_mapping
    2022-10-07 16:48:00,609 eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Next page URL could not be collected
    2022-10-07 16:48:00,626 eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Next page Query-object could not be collected
    2022-10-07 16:48:00,643 eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Next page merge could not be collected
    2022-10-07 16:48:00,643 eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Adapting 125 plugin results to eodag product representation
    2022-10-07 16:48:02,877 eodag.plugins.crunch.filter_date [DEBUG   ] (filter_date      ) Start filtering by date
    2022-10-07 16:48:02,893 eodag.plugins.crunch.filter_date [INFO    ] (filter_date      ) Finished filtering products. 4 resulting products
    2022-10-07 16:48:02,893 eodag.plugins.crunch.filter_overlap [DEBUG   ] (filter_overlap   ) Start filtering for overlapping products
    2022-10-07 16:48:02,893 eodag.plugins.crunch.filter_overlap [WARNING ] (filter_overlap   ) geometry not found in cruncher arguments, filtering disabled.
    2022-10-07 16:48:02,893 eodag.plugins.crunch.filter_property [DEBUG   ] (filter_property  ) Start filtering for products matching operator.eq(product.properties['sar:product_type'], SLC)
    2022-10-07 16:48:02,893 eodag.plugins.crunch.filter_property [INFO    ] (filter_property  ) Finished filtering products. 4 resulting products
    2022-10-07 16:48:02,893 eodag.plugins.crunch.filter_property [DEBUG   ] (filter_property  ) Start filtering for products matching operator.eq(product.properties['sensorMode'], spotlight)
    2022-10-07 16:48:02,893 eodag.plugins.crunch.filter_property [INFO    ] (filter_property  ) Finished filtering products. 2 resulting products
    2022-10-07 16:48:02,898 eodag.core                       [INFO    ] (core             ) Found 2 result(s) on provider 'capella'
    2022-10-07 16:48:02,898 eodag.core                       [INFO    ] (core             ) Downloading 2 products
    Downloaded products:   0%|          | 0/2 [00:00<?, ?product/s]
    0.00B [00:00, ?B/s]
    CAPELLA_C03_SP_SLC_HH_20211214232643_20211214232646: 0.00B [00:00, ?B/s]2022-10-07 16:48:02,900 eodag.plugins.download.base      [INFO    ] (base             ) Download url: https://capella-open-data.s3.us-west-2.amazonaws.com/collections/capella-open-data-spotlight/items/CAPELLA_C03_SP_SLC_HH_20211214232643_20211214232646
    2022-10-07 16:48:03,847 eodag.plugins.download.aws       [WARNING ] (aws              ) Unexpected error: An error occurred (NoSuchKey) when calling the ListObjects operation: The specified key does not exist.
    2022-10-07 16:48:03,847 eodag.plugins.download.aws       [WARNING ] (aws              ) Skipping data//2021/12/14/CAPELLA_C03_SP_SLC_HH_20211214232643_20211214232646/CAPELLA_C03_SP_SLC_HH_20211214232643_20211214232646.tif
    2022-10-07 16:48:04,678 eodag.plugins.download.aws       [WARNING ] (aws              ) Unexpected error: An error occurred (NoSuchKey) when calling the ListObjects operation: The specified key does not exist.
    2022-10-07 16:48:04,678 eodag.plugins.download.aws       [WARNING ] (aws              ) Skipping data//2021/12/14/CAPELLA_C03_SP_SLC_HH_20211214232643_20211214232646/CAPELLA_C03_SP_SLC_HH_20211214232643_20211214232646_extended.json
    2022-10-07 16:48:05,496 eodag.plugins.download.aws       [WARNING ] (aws              ) Unexpected error: An error occurred (NoSuchKey) when calling the ListObjects operation: The specified key does not exist.
    2022-10-07 16:48:05,496 eodag.plugins.download.aws       [WARNING ] (aws              ) Skipping data//2021/12/14/CAPELLA_C03_SP_GEO_HH_20211214232633_20211214232656/CAPELLA_C03_SP_GEO_HH_20211214232633_20211214232656_preview.tif
    2022-10-07 16:48:06,301 eodag.plugins.download.aws       [WARNING ] (aws              ) Unexpected error: An error occurred (NoSuchKey) when calling the ListObjects operation: The specified key does not exist.
    2022-10-07 16:48:06,301 eodag.plugins.download.aws       [WARNING ] (aws              ) Skipping data//2021/12/14/CAPELLA_C03_SP_GEO_HH_20211214232633_20211214232656/CAPELLA_C03_SP_GEO_HH_20211214232633_20211214232656_thumb.png
    2022-10-07 16:48:06,302 eodag.plugins.download.base      [ERROR   ] (base             ) Stopped because of credentials problems with provider capella
    Traceback (most recent call last):
      File "C:\Users\ajeannin\Anaconda3\envs\radar_dev\lib\site-packages\eodag\plugins\download\base.py", line 456, in download_all
        product.download(
      File "C:\Users\ajeannin\Anaconda3\envs\radar_dev\lib\site-packages\eodag\api\product\_product.py", line 288, in download
        fs_path = self.downloader.download(
      File "C:\Users\ajeannin\Anaconda3\envs\radar_dev\lib\site-packages\eodag\plugins\download\aws.py", line 339, in download
        raise AuthenticationError(", ".join(auth_error_messages))
    eodag.utils.exceptions.AuthenticationError: An error occurred (NoSuchKey) when calling the ListObjects operation: The specified key does not exist.
    Downloaded products:   0%|          | 0/2 [00:03<?, ?product/s]
    Traceback (most recent call last):
      File "D:\PythonCode\radar_dev\radar_dev\utils.py", line 48, in <module>
        paths = dag.download_all(capella_prod)
      File "C:\Users\ajeannin\Anaconda3\envs\radar_dev\lib\site-packages\eodag\api\core.py", line 1288, in download_all
        paths = download_plugin.download_all(
      File "C:\Users\ajeannin\Anaconda3\envs\radar_dev\lib\site-packages\eodag\plugins\download\aws.py", line 920, in download_all
        return super(AwsDownload, self).download_all(
      File "C:\Users\ajeannin\Anaconda3\envs\radar_dev\lib\site-packages\eodag\plugins\download\base.py", line 456, in download_all
        product.download(
      File "C:\Users\ajeannin\Anaconda3\envs\radar_dev\lib\site-packages\eodag\api\product\_product.py", line 288, in download
        fs_path = self.downloader.download(
      File "C:\Users\ajeannin\Anaconda3\envs\radar_dev\lib\site-packages\eodag\plugins\download\aws.py", line 339, in download
        raise AuthenticationError(", ".join(auth_error_messages))
    eodag.utils.exceptions.AuthenticationError: An error occurred (NoSuchKey) when calling the ListObjects operation: The specified key does not exist.
    CAPELLA_C03_SP_SLC_HH_20211214232643_20211214232646: 0.00B [00:03, ?B/s]
    
    Process finished with exit code 1
    

    Environment:

    • Conda version: 22.9.0
    • Python version: 3.7
    • EODAG version: eodag version tells me
    (radar_dev) λ eodag version                                                                                                                                                                                                                                                                          
    eodag (Earth Observation Data Access Gateway): version 0.0.0
    

    But I think I got the 2.5.0 version as it is the latest on Conda env (should I open a new issue ?)

    bug 
    opened by AriJeannin 12
  • onda: error with the geometry parameter

    onda: error with the geometry parameter

    The geometry passed to onda doesn't seem to work anymore, try for instance the following request (cuont request) made to onda in its end to end test:

    https://catalogue.onda-dias.eu/dias-catalogue/Products?productType=S2MSI1C&$format=json&$search=%22footprint:%22Intersects%28POLYGON%20%28%280.2564%2043.1956%2C%200.2564%2043.9078%2C%202.3798%2043.9078%2C%202.3798%2043.1956%2C%200.2564%2043.1956%29%29%29%22%20AND%20productType:S2MSI1C%20AND%20beginPosition:[2021-03-12T00:00:00.000Z%20TO%20%2A]%20AND%20endPosition:[%2A%20TO%202021-03-19T00:00:00.000Z]%22

    The returned error is:

    "Error from server : Unable to parse shape given formats \"lat,lon\", \"x y\" or as WKT because java.text.ParseException: java.lang.UnsupportedOperationException: Unsupported shape of this SpatialContext. Try JTS or Geo3D. input: POLYGON ((0.2564 43.1956, 0.2564 43.9078, 2.3798 43.9078, 2.3798 43.1956, 0.2564 43.1956))"
    
    bug priority::1 
    opened by maximlt 12
  • Fail to run the basic tutorial

    Fail to run the basic tutorial

    Original report by Mickaël Savinaud (Bitbucket: savmickael, GitHub: savmickael).


    When I run the following code (close to the tutorial one) I get the following error :

    
    /home/msavinaud/dev/eodag/venv/lib/python3.5/site-packages/eodag/config.py:42: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
      self.source = yaml.load(fh)
    /home/msavinaud/dev/eodag/venv/lib/python3.5/site-packages/eodag/config.py:185: YAMLLoadWarning: calling yaml.load_all() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
      for provider_config in providers_configs:
    Traceback (most recent call last):
      File "generate_s2_tile_list_info.py", line 67, in <module>
        sys.exit(main())
      File "generate_s2_tile_list_info.py", line 38, in main
        dag = EODataAccessGateway(user_conf_file_path=conf_path)
      File "/home/msavinaud/dev/eodag/venv/lib/python3.5/site-packages/eodag/api/core.py", line 51, in __init__
        self.providers_config = load_default_config()
      File "/home/msavinaud/dev/eodag/venv/lib/python3.5/site-packages/eodag/config.py", line 185, in load_default_config
        for provider_config in providers_configs:
      File "/home/msavinaud/dev/eodag/venv/lib/python3.5/site-packages/yaml/__init__.py", line 130, in load_all
        yield loader.get_data()
      File "/home/msavinaud/dev/eodag/venv/lib/python3.5/site-packages/yaml/constructor.py", line 37, in get_data
        return self.construct_document(self.get_node())
      File "/home/msavinaud/dev/eodag/venv/lib/python3.5/site-packages/yaml/constructor.py", line 47, in construct_document
        data = self.construct_object(node)
      File "/home/msavinaud/dev/eodag/venv/lib/python3.5/site-packages/yaml/constructor.py", line 92, in construct_object
        data = constructor(self, node)
      File "/home/msavinaud/dev/eodag/venv/lib/python3.5/site-packages/yaml/constructor.py", line 420, in construct_undefined
        node.start_mark)
    yaml.constructor.ConstructorError: could not determine a constructor for the tag '!provider'
      in "/home/msavinaud/dev/eodag/venv/lib/python3.5/site-packages/eodag/resources/providers.yml", line 19, column 1
    
    
    

    Here my conf file

    peps:
        download:
            extract: false
        auth:
            credentials:
                username: login
                password: my_pass
    
    bug 
    opened by sbrunato 11
  • Add quicklooks as EOProduct attribute/properties field

    Add quicklooks as EOProduct attribute/properties field

    Original report by Baptiste Meylheuc (Bitbucket: bmeylheuc, ).


    Add quicklooks to properties EOProduct attribute, or directly as a new attribute. A simple look on the data allows to quickly check if the product correspond to what the user want before downloading it.

    In an IDE like Jupyter notebook, it can be a real plus as the images are very well embedded to the structure.

    enhancement 
    opened by sbrunato 10
  • Connection timeout bug in search API

    Connection timeout bug in search API

    Describe the bug

    No result from provider 'creodias' due to an error during the search.

    urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x7f0614fa07f0>, 'Connection to finder.creodias.eu timed out. (connect timeout=5)')

    Timeout error is not properly handled and propagated back. The error (len() of None) appears already inside the eodag code:

    normalize_remaining_count = len(results)
    TypeError: object of type 'NoneType' has no len()
    

    Code To Reproduce

    all_products = session.search_all(**search_criteria) -> with the specific search and creodias provider for example.

    Output

    [2022-11-14 23:38:30,795][eodag.core][INFO] - Searching product type 'S2_MSI_L1C' on provider: creodias
    [2022-11-14 23:38:30,795][eodag.core][INFO] - Iterate search over multiple pages: page #1
    [2022-11-14 23:38:30,795][eodag.plugins.search.qssearch][INFO] - Sending search request: https://finder.creodias.eu/resto/api/collections/Sentinel2/search.json?startDate=2019-05-01T00:00:00&completionDate=2019-10-01T00:00:00&geometry=POLYGON ((9.1010 46.6489, 9.0945 46.6490, 9.0946 46.6534, 9.1011 46.6534, 9.1010 46.6489))&productType=L1C&maxRecords=2000&page=1
    [2022-11-14 23:38:35,883][eodag.plugins.search.qssearch][ERROR] - Skipping error while searching for creodias QueryStringSearch instance:
    Traceback (most recent call last):
      File ".../lib/python3.9/site-packages/urllib3/connection.py", line 174, in _new_conn
        conn = connection.create_connection(
      File ".../lib/python3.9/site-packages/urllib3/util/connection.py", line 95, in create_connection
        raise err
      File ".../lib/python3.9/site-packages/urllib3/util/connection.py", line 85, in create_connection
        sock.connect(sa)
    socket.timeout: timed out
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File ".../lib/python3.9/site-packages/urllib3/connectionpool.py", line 703, in urlopen
        httplib_response = self._make_request(
      File ".../lib/python3.9/site-packages/urllib3/connectionpool.py", line 386, in _make_request
        self._validate_conn(conn)
      File ".../lib/python3.9/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
        conn.connect()
      File ".../lib/python3.9/site-packages/urllib3/connection.py", line 358, in connect
        self.sock = conn = self._new_conn()
      File ".../lib/python3.9/site-packages/urllib3/connection.py", line 179, in _new_conn
        raise ConnectTimeoutError(
    urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x7f0614fa07f0>, 'Connection to finder.creodias.eu timed out. (connect timeout=5)')
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File ".../lib/python3.9/site-packages/requests/adapters.py", line 489, in send
        resp = conn.urlopen(
      File ".../lib/python3.9/site-packages/urllib3/connectionpool.py", line 787, in urlopen
        retries = retries.increment(
      File ".../lib/python3.9/site-packages/urllib3/util/retry.py", line 592, in increment
        raise MaxRetryError(_pool, url, error or ResponseError(cause))
    urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='finder.creodias.eu', port=443): Max retries exceeded with url: /resto/api/collections/Sentinel2/search.json?startDate=2019-05-01T00:00:00&completionDate=2019-10-01T00:00:00&geometry=POLYGON%20((9.1010%2046.6489,%209.0945%2046.6490,%209.0946%2046.6534,%209.1011%2046.6534,%209.1010%2046.6489))&productType=L1C&maxRecords=2000&page=1 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f0614fa07f0>, 'Connection to finder.creodias.eu timed out. (connect timeout=5)'))
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File ".../lib/python3.9/site-packages/eodag/plugins/search/qssearch.py", line 864, in _request
        response = requests.get(url, timeout=HTTP_REQ_TIMEOUT, **kwargs)
      File ".../lib/python3.9/site-packages/requests/api.py", line 73, in get
        return request("get", url, params=params, **kwargs)
      File ".../lib/python3.9/site-packages/requests/api.py", line 59, in request
        return session.request(method=method, url=url, **kwargs)
      File ".../lib/python3.9/site-packages/requests/sessions.py", line 587, in request
        resp = self.send(prep, **send_kwargs)
      File ".../lib/python3.9/site-packages/requests/sessions.py", line 701, in send
        r = adapter.send(request, **kwargs)
      File ".../lib/python3.9/site-packages/requests/adapters.py", line 553, in send
        raise ConnectTimeout(e, request=request)
    requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='finder.creodias.eu', port=443): Max retries exceeded with url: /resto/api/collections/Sentinel2/search.json?startDate=2019-05-01T00:00:00&completionDate=2019-10-01T00:00:00&geometry=POLYGON%20((9.1010%2046.6489,%209.0945%2046.6490,%209.0946%2046.6534,%209.1011%2046.6534,%209.1010%2046.6489))&productType=L1C&maxRecords=2000&page=1 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f0614fa07f0>, 'Connection to finder.creodias.eu timed out. (connect timeout=5)'))
    [2022-11-14 23:38:35,888][eodag.core][INFO] - No result from provider 'creodias' due to an error during search. Raise verbosity of log messages for details
    
    File ".../sentinel_api.py", line 49, in search_products
        all_products = session.search_all(**search_criteria)
      File ".../lib/python3.9/site-packages/eodag/api/core.py", line 1099, in search_all
        for page_results in self.search_iter_page(
      File ".../lib/python3.9/site-packages/eodag/api/core.py", line 953, in search_iter_page
        products, _ = self._do_search(
      File ".../lib/python3.9/site-packages/eodag/api/core.py", line 1339, in _do_search
        res, nb_res = search_plugin.query(count=count, **kwargs)
      File ".../lib/python3.9/site-packages/eodag/plugins/search/qssearch.py", line 341, in query
        eo_products = self.normalize_results(provider_results, **kwargs)
      File ".../lib/python3.9/site-packages/eodag/plugins/search/qssearch.py", line 677, in normalize_results
        normalize_remaining_count = len(results)
    TypeError: object of type 'NoneType' has no len()
    

    Environment:

    • Python version: 3.9
    • EODAG version: 2.6.0

    Additional context

    The above error appeared in a large-scale search for 50k different searches. The error appeared on a specific search, after 10s of thousands of successful searches.

    bug 
    opened by ds2268 7
  • run Ship detection notebook. Error found

    run Ship detection notebook. Error found

    graph_process = os.path.join(workspace, 'ShipDetection.xml')
    with open(graph_process, 'w') as g_2:
        g_2.write(
    """
    <graph id="Graph">
      <version>1.0</version>
        <node id="Read">
          <operator>Read</operator>
          <sources/>
          <parameters>
            <file>${inputproduct}</file>
          </parameters>
        </node>
        <node id="Land-Sea-Mask">
          <operator>Land-Sea-Mask</operator>
          <sources>
            <sourceProduct refid="Read"/>
          </sources>
          <parameters>
            <geometry>Gulf_of_Trieste_seamask_UTM33</geometry>
            <landMask>false</landMask>
            <useSRTM>false</useSRTM>
            <shorelineExtension>10</shorelineExtension>
          </parameters>
        </node>
        <node id="Calibration">
    

    GPT complain on this line <geometry>Gulf_of_Trieste_seamask_UTM33</geometry>:

    expression: Undefined symbol 'Gulf_of_Trieste_seamask_UTM33'. due to Undefined symbol 'Gulf_of_Trieste_seamask_UTM33'. done.

    I actually checked the file of the first step: S1A_IW_GRDH_1SDV_20161009T165807_4550_Orb.dim And did found there is vector data with the same name.

    question 
    opened by huayue21 7
  • Small errors in serializing/deserializing

    Small errors in serializing/deserializing

    Hello,

    I noticed that there are small errors in serializing and deserializing EoProduct.
    They sadly prevent EOProduct == deserialize(serialize(EOProduct)).

    I've traced those errors to these functions in _product.py:

    • as_dict: Some fields are forgotten, such as downloader, downloader_auth, search_args, search_kwargs. They could be managed the same way as provider for example.
    • from_geojson: The fields beginning by eodag_ should be popped in order to make them disappear from EOProduct's properties.

    Best Regards,

    Rémi

    opened by remi-braun 7
  • Name collision of downloaded products

    Name collision of downloaded products

    Original report by Stéphan AIME (Bitbucket: Fifan31, GitHub: Fifan31).


    So far, title of product is used to name a downloaded product. But when products share almost the same extend (stereo products for instance), they may have the same title that lead to a name collision.

    The 1st product is downloaded (and unzipped), the second is downloaded overriding the first one and it is not unzipped as the name already exists.

    Proposed solution: add a unique suffix (uuid?) to product title

    bug 
    opened by sbrunato 7
  • Add pagination to the search interface

    Add pagination to the search interface

    Original report by Oyono (Bitbucket: aoyono, GitHub: aoyono).


    The current search interface does not allow the user to control the number of products he wants during a search. This issue aims at solving this problem

    enhancement 
    opened by sbrunato 7
  • sobloo end of service?

    sobloo end of service?

    https://sobloo.eu/index.html image

    • For search: In the documentation it seams that products can still be searched using extenSO API on https://sobloo.eu, but all the requests I tried using postman configuration return
    <Error>
      <Code>NoSuchKey</Code>
      <Message>The specified key does not exist.</Message>
    </Error>
    
    • For download, data has to be moved to a personal bucket (10 GB free) using the API

    • [ ] If this can be used to update sobloo provider settings, use it. And if not, remove it from available providers

    • [ ] Check if a new provider should be created using extenSO API

    bug 
    opened by sbrunato 0
  • LandSat C2L2 download issue

    LandSat C2L2 download issue

    Thanks for this very useful and easy to use package !

    I am trying to download some LandSat Collection 2 Level 2 data but I keep having the same download issue. The search returns the tiles I am looking for, but the download doesn't go through.

    Here is the code am running:

    from eodag import setup_logging  # module that downloads S2 data
    from eodag import EODataAccessGateway  # module that downloads S2 data
    import geopandas as gpd
    
    setup_logging(2)
    dag = EODataAccessGateway()
    
    shapefile = '/home/me/Notebooks/Shapefiles/my_shapefile.shp'
    
    # Open shapefile containing geometry
    geopandas_shape = gpd.read_file(shapefile)
    geopandas_shape = geopandas_shape.to_crs(epsg='4326')  # Force WGS84 projection
    bounds = geopandas_shape.geometry.total_bounds  # In WGS84 projection
    
    
    search_criteria = {
        'productType': 'LANDSAT_C2L2', 
        'start': '2020-05-01',
        'end': '2020-09-01',
        'geom': list(bounds)
    }
    
    all_products = dag.search_all(**search_criteria)
    
    products_to_download = all_products.filter_property(cloudCover=80, operator='lt')
    product_paths = dag.download_all(products_to_download, extract=False)  # No archive extraction
    

    And here is the output:

    • For the search part:
    2022-12-14 10:12:04,721 eodag.config                     [INFO    ] Loading user configuration from: /home/auclairj/.config/eodag/eodag.yml
    2022-12-14 10:12:04,821 eodag.core                       [INFO    ] aws_eos: provider needing auth for search has been pruned because no crendentials could be found
    2022-12-14 10:12:04,911 eodag.core                       [INFO    ] Locations configuration loaded from /home/auclairj/.config/eodag/locations.yml
    2022-12-14 10:12:05,478 eodag.core                       [WARNING ] Product type 'LANDSAT_C2L2' is not available with provider 'scihub'. Searching it on provider 'usgs' instead.
    2022-12-14 10:12:05,478 eodag.core                       [INFO    ] Iterate search over multiple pages: page #1
    2022-12-14 10:12:06,844 eodag.plugins.apis.usgs          [INFO    ] Sending search request for landsat_ot_c2_l2 with {'start_date': '2020-05-01', 'end_date': '2020-09-01', 'll': {'longitude': 0.58786257937628, 'latitude': 41.705398845326464}, 'ur': {'longitude': 0.7960598497347214, 'latitude': 41.81649364004583}, 'max_results': 5000, 'starting_number': 1}
    2022-12-14 10:12:13,346 eodag.core                       [INFO    ] Found 16 result(s) on provider 'usgs'
    
    • For the download part (I get this for all the demanded products over and over, here is the output for one product):
    2022-12-14 10:40:52,326 eodag.plugins.download.base      [INFO    ] Download url: https://earthexplorer.usgs.gov/download/external/options/landsat_ot_c2_l2/LC81990312020236LGN00/M2M/
    2022-12-14 10:40:54,145 eodag.plugins.download.base      [INFO    ] No usgs request url was found for LC08_L2SP_199031_20200823_20200905_02_T1
    LC08_L2SP_199031_20200823_20200905_02_T1: 0.00B [00:01, ?B/s]
    

    When I copy paste the given url in my navigator I can manually download the product.

    I have no issue using this module for Sentinel 2 products (scihub and theia). I have the correct id and password written in the eodag config file.

    Environment:

    • Python version: Python 3.8.13
    • Eodag version: eodag (Earth Observation Data Access Gateway): version 2.6.0
    conda list
    $ conda list
    
    # Name                    Version                   Build  Channel
    _libgcc_mutex             0.1                        main
    _openmp_mutex             5.1                       1_gnu
    affine                    2.3.1                    pypi_0    pypi
    alabaster                 0.7.12             pyhd3eb1b0_0
    arrow                     1.2.3            py38h06a4308_0
    astroid                   2.11.7           py38h06a4308_0
    asttokens                 2.0.8                    pypi_0    pypi
    atomicwrites              1.4.0                      py_0
    attrs                     22.1.0           py38h06a4308_0
    autopep8                  1.6.0              pyhd3eb1b0_1
    babel                     2.9.1              pyhd3eb1b0_0
    backcall                  0.2.0              pyhd3eb1b0_0
    beautifulsoup4            4.11.1           py38h06a4308_0
    binaryornot               0.4.4              pyhd3eb1b0_1
    black                     22.6.0           py38h06a4308_0
    blas                      1.0                         mkl
    bleach                    4.1.0              pyhd3eb1b0_0
    boto3                     1.24.91                  pypi_0    pypi
    botocore                  1.27.91                  pypi_0    pypi
    brotlipy                  0.7.0           py38h27cfd23_1003
    bzip2                     1.0.8                h7b6447c_0
    c-ares                    1.18.1               h7f8727e_0
    ca-certificates           2022.10.11           h06a4308_0
    cairo                     1.16.0               h19f5f5c_2
    cdsapi                    0.5.1                    pypi_0    pypi
    certifi                   2022.9.24        py38h06a4308_0
    cffi                      1.15.1           py38h74dc2b5_0
    cfitsio                   3.470                h5893167_7
    cftime                    1.6.2                    pypi_0    pypi
    chardet                   4.0.0           py38h06a4308_1003
    charset-normalizer        2.1.1                    pypi_0    pypi
    click                     8.1.3                    pypi_0    pypi
    click-plugins             1.1.1                    pypi_0    pypi
    cligj                     0.7.2                    pypi_0    pypi
    cloudpickle               2.0.0              pyhd3eb1b0_0
    colorama                  0.4.5            py38h06a4308_0
    contourpy                 1.0.5                    pypi_0    pypi
    cookiecutter              1.7.3              pyhd3eb1b0_0
    cryptography              38.0.1           py38h9ce1e76_0
    curl                      7.84.0               h5eee18b_0
    cycler                    0.11.0                   pypi_0    pypi
    dbus                      1.13.18              hb2f20db_0
    debugpy                   1.6.3                    pypi_0    pypi
    decorator                 5.1.1              pyhd3eb1b0_0
    defusedxml                0.7.1              pyhd3eb1b0_0
    diff-match-patch          20200713           pyhd3eb1b0_0
    dill                      0.3.6            py38h06a4308_0
    docutils                  0.18.1           py38h06a4308_3
    ecmwf-api-client          1.6.3                    pypi_0    pypi
    entrypoints               0.4              py38h06a4308_0
    eodag                     2.6.0                    pypi_0    pypi
    eodag-sentinelsat         0.4.1                    pypi_0    pypi
    eto                       1.1.0                    pypi_0    pypi
    executing                 1.1.1                    pypi_0    pypi
    expat                     2.4.9                h6a678d5_0
    fiona                     1.8.21                   pypi_0    pypi
    flake8                    4.0.1              pyhd3eb1b0_1
    flasgger                  0.9.5                    pypi_0    pypi
    flask                     2.2.2                    pypi_0    pypi
    fontconfig                2.13.1               h6c09931_0
    fonttools                 4.37.4                   pypi_0    pypi
    freetype                  2.11.0               h70c0345_0
    freexl                    1.0.6                h27cfd23_0
    gdal                      3.4.1            py38h2c27f0e_0
    geojson                   2.5.0                    pypi_0    pypi
    geomet                    0.3.0                    pypi_0    pypi
    geopandas                 0.11.1                   pypi_0    pypi
    geos                      3.8.0                he6710b0_0
    geotiff                   1.7.0                hd69d5b1_0
    giflib                    5.2.1                h7b6447c_0
    glib                      2.69.1               h4ff587b_1
    gst-plugins-base          1.14.0               h8213a91_2
    gstreamer                 1.14.0               h28cd5cc_2
    h5netcdf                  1.1.0                    pypi_0    pypi
    h5py                      3.7.0                    pypi_0    pypi
    hdf4                      4.2.13               h3ca952b_2
    hdf5                      1.10.6               h3ffc7dd_1
    html2text                 2020.1.16                pypi_0    pypi
    icu                       58.2                 he6710b0_3
    idna                      3.4              py38h06a4308_0
    imagesize                 1.4.1            py38h06a4308_0
    importlib-metadata        5.0.0                    pypi_0    pypi
    importlib-resources       5.10.0                   pypi_0    pypi
    importlib_metadata        4.11.3               hd3eb1b0_0
    inflection                0.5.1            py38h06a4308_0
    intel-openmp              2021.4.0          h06a4308_3561
    intervaltree              3.1.0              pyhd3eb1b0_0
    ipykernel                 6.16.0                   pypi_0    pypi
    ipython                   8.5.0                    pypi_0    pypi
    ipython_genutils          0.2.0              pyhd3eb1b0_1
    isort                     5.9.3              pyhd3eb1b0_0
    itsdangerous              2.1.2                    pypi_0    pypi
    jedi                      0.18.1           py38h06a4308_1
    jeepney                   0.7.1              pyhd3eb1b0_0
    jellyfish                 0.9.0            py38h7f8727e_0
    jinja2                    3.1.2            py38h06a4308_0
    jinja2-time               0.2.0              pyhd3eb1b0_3
    jmespath                  1.0.1                    pypi_0    pypi
    jpeg                      9e                   h7f8727e_0
    json-c                    0.13.1               h1bed415_0
    jsonpath-ng               1.5.3                    pypi_0    pypi
    jsonschema                4.16.0           py38h06a4308_0
    jupyter-client            7.4.2                    pypi_0    pypi
    jupyter-core              4.11.1                   pypi_0    pypi
    jupyterlab_pygments       0.1.2                      py_0
    kealib                    1.4.14               hb50703a_1
    keyring                   23.4.0           py38h06a4308_0
    kiwisolver                1.4.4                    pypi_0    pypi
    krb5                      1.19.2               hac12032_0
    landsatxplore             0.13.0                   pypi_0    pypi
    lazy-object-proxy         1.6.0            py38h27cfd23_0
    ld_impl_linux-64          2.38                 h1181459_1
    lerc                      3.0                  h295c915_0
    libboost                  1.73.0              h28710b8_12
    libclang                  10.0.1          default_hb85057a_2
    libcurl                   7.84.0               h91b91d3_0
    libdap4                   3.19.1               h6ec2957_0
    libdeflate                1.8                  h7f8727e_5
    libedit                   3.1.20210910         h7f8727e_0
    libev                     4.33                 h7f8727e_1
    libevent                  2.1.12               h8f2d780_0
    libffi                    3.3                  he6710b0_2
    libgcc-ng                 11.2.0               h1234567_1
    libgdal                   3.4.1                h05199a0_1
    libgfortran-ng            11.2.0               h00389a5_1
    libgfortran5              11.2.0               h1234567_1
    libgomp                   11.2.0               h1234567_1
    libkml                    1.3.0                h7ecb851_5
    libllvm10                 10.0.1               hbcb73fb_5
    libnetcdf                 4.8.1                h42ceab0_1
    libnghttp2                1.46.0               hce63b2e_0
    libpng                    1.6.37               hbc83047_0
    libpq                     12.9                 h16c4e8d_3
    libsodium                 1.0.18               h7b6447c_0
    libspatialindex           1.9.3                h2531618_0
    libspatialite             4.3.0a              hbedb2dc_20
    libssh2                   1.10.0               h8f2d780_0
    libstdcxx-ng              11.2.0               h1234567_1
    libtiff                   4.4.0                hecacb30_0
    libuuid                   1.0.3                h7f8727e_2
    libwebp                   1.2.4                h11a3e52_0
    libwebp-base              1.2.4                h5eee18b_0
    libxcb                    1.15                 h7f8727e_0
    libxkbcommon              1.0.1                hfa300c1_0
    libxml2                   2.9.14               h74e7548_0
    libxslt                   1.1.35               h4e12654_0
    libzip                    1.8.0                h5cef20c_0
    lxml                      4.9.1            py38h1edc446_0
    lz4-c                     1.9.3                h295c915_1
    markdown                  3.4.1                    pypi_0    pypi
    markupsafe                2.1.1            py38h7f8727e_0
    matplotlib                3.6.1                    pypi_0    pypi
    matplotlib-inline         0.1.6            py38h06a4308_0
    mccabe                    0.7.0              pyhd3eb1b0_0
    memory-profiler           0.61.0                   pypi_0    pypi
    mistune                   2.0.4                    pypi_0    pypi
    mkl                       2021.4.0           h06a4308_640
    mkl-service               2.4.0            py38h7f8727e_0
    mkl_fft                   1.3.1            py38hd3c417c_0
    mkl_random                1.2.2            py38h51133e4_0
    multiprocess              0.70.14                  pypi_0    pypi
    munch                     2.5.0                    pypi_0    pypi
    mypy_extensions           0.4.3            py38h06a4308_1
    nbclient                  0.5.13           py38h06a4308_0
    nbconvert                 6.5.4            py38h06a4308_0
    nbformat                  5.5.0            py38h06a4308_0
    ncurses                   6.3                  h5eee18b_3
    nest-asyncio              1.5.6                    pypi_0    pypi
    netcdf4                   1.6.2                    pypi_0    pypi
    nspr                      4.33                 h295c915_0
    nss                       3.74                 h0370c37_0
    numpy                     1.23.4                   pypi_0    pypi
    numpydoc                  1.5.0            py38h06a4308_0
    opencv-python             4.6.0.66                 pypi_0    pypi
    openjpeg                  2.4.0                h3ad879b_0
    openssl                   1.1.1s               h7f8727e_0
    owslib                    0.27.2                   pypi_0    pypi
    p-tqdm                    1.4.0                    pypi_0    pypi
    packaging                 21.3               pyhd3eb1b0_0
    pandas                    1.5.0                    pypi_0    pypi
    pandocfilters             1.5.0              pyhd3eb1b0_0
    parso                     0.8.3              pyhd3eb1b0_0
    pathos                    0.3.0                    pypi_0    pypi
    pathspec                  0.9.0            py38h06a4308_0
    pcre                      8.45                 h295c915_0
    pexpect                   4.8.0              pyhd3eb1b0_3
    pickleshare               0.7.5           pyhd3eb1b0_1003
    pillow                    9.2.0                    pypi_0    pypi
    pip                       22.3.1                   pypi_0    pypi
    pixman                    0.40.0               h7f8727e_1
    pkgutil-resolve-name      1.3.10           py38h06a4308_0
    platformdirs              2.5.2            py38h06a4308_0
    pluggy                    1.0.0            py38h06a4308_1
    ply                       3.11                     py38_0
    poppler                   0.81.0               h01f5e8b_2
    poppler-data              0.4.11               h06a4308_0
    pox                       0.3.2                    pypi_0    pypi
    poyo                      0.5.0              pyhd3eb1b0_0
    ppft                      1.7.6.6                  pypi_0    pypi
    proj                      6.2.1                h05a3930_0
    prompt-toolkit            3.0.31                   pypi_0    pypi
    psutil                    5.9.2                    pypi_0    pypi
    ptyprocess                0.7.0              pyhd3eb1b0_2
    pure-eval                 0.2.2                    pypi_0    pypi
    pycodestyle               2.8.0              pyhd3eb1b0_0
    pycparser                 2.21               pyhd3eb1b0_0
    pydocstyle                6.1.1              pyhd3eb1b0_0
    pyflakes                  2.4.0              pyhd3eb1b0_0
    pygments                  2.13.0                   pypi_0    pypi
    pylint                    2.14.5           py38h06a4308_0
    pyls-spyder               0.4.0              pyhd3eb1b0_0
    pyopenssl                 22.0.0             pyhd3eb1b0_0
    pyparsing                 3.0.9            py38h06a4308_0
    pyproj                    3.4.0                    pypi_0    pypi
    pyqt                      5.15.7           py38h6a678d5_1
    pyqt5-sip                 12.11.0          py38h6a678d5_1
    pyqtwebengine             5.15.7           py38h6a678d5_1
    pyrsistent                0.18.1                   pypi_0    pypi
    pyshp                     2.3.1                    pypi_0    pypi
    pysocks                   1.7.1            py38h06a4308_0
    pystac                    1.6.1                    pypi_0    pypi
    python                    3.8.13               h12debd9_0
    python-dateutil           2.8.2              pyhd3eb1b0_0
    python-fastjsonschema     2.16.2           py38h06a4308_0
    python-lsp-black          1.2.1            py38h06a4308_0
    python-lsp-jsonrpc        1.0.0              pyhd3eb1b0_0
    python-lsp-server         1.5.0            py38h06a4308_0
    python-slugify            5.0.2              pyhd3eb1b0_0
    pytz                      2022.4                   pypi_0    pypi
    pyxdg                     0.27               pyhd3eb1b0_0
    pyyaml                    6.0              py38h7f8727e_1
    pyzmq                     24.0.1                   pypi_0    pypi
    qdarkstyle                3.0.2              pyhd3eb1b0_0
    qstylizer                 0.1.10             pyhd3eb1b0_0
    qt-main                   5.15.2               h327a75a_7
    qt-webengine              5.15.9               hd2b0992_4
    qtawesome                 1.0.3              pyhd3eb1b0_0
    qtconsole                 5.3.2            py38h06a4308_0
    qtpy                      2.2.0            py38h06a4308_0
    qtwebkit                  5.212                h4eab89a_4
    rasterio                  1.3.2                    pypi_0    pypi
    readline                  8.1.2                h7f8727e_1
    requests                  2.28.1           py38h06a4308_0
    requests-futures          1.0.0                    pypi_0    pypi
    rope                      0.22.0             pyhd3eb1b0_0
    rtree                     0.9.7            py38h06a4308_1
    s3transfer                0.6.0                    pypi_0    pypi
    scipy                     1.9.2                    pypi_0    pypi
    secretstorage             3.3.1            py38h06a4308_0
    sentinelsat               1.1.1                    pypi_0    pypi
    setuptools                65.5.0           py38h06a4308_0
    shapely                   1.8.5.post1              pypi_0    pypi
    sip                       6.6.2            py38h6a678d5_0
    six                       1.16.0             pyhd3eb1b0_1
    snowballstemmer           2.2.0              pyhd3eb1b0_0
    snuggs                    1.4.7                    pypi_0    pypi
    sortedcontainers          2.4.0              pyhd3eb1b0_0
    soupsieve                 2.3.2.post1      py38h06a4308_0
    sphinx                    5.0.2            py38h06a4308_0
    sphinxcontrib-applehelp   1.0.2              pyhd3eb1b0_0
    sphinxcontrib-devhelp     1.0.2              pyhd3eb1b0_0
    sphinxcontrib-htmlhelp    2.0.0              pyhd3eb1b0_0
    sphinxcontrib-jsmath      1.0.1              pyhd3eb1b0_0
    sphinxcontrib-qthelp      1.0.3              pyhd3eb1b0_0
    sphinxcontrib-serializinghtml 1.1.5              pyhd3eb1b0_0
    spyder-kernels            2.3.3            py38h06a4308_0
    sqlite                    3.39.3               h5082296_0
    stack-data                0.5.1                    pypi_0    pypi
    text-unidecode            1.3                pyhd3eb1b0_0
    textdistance              4.2.1              pyhd3eb1b0_0
    three-merge               0.1.1              pyhd3eb1b0_0
    tiledb                    2.3.3                h1132f93_2
    tinycss                   0.4             pyhd3eb1b0_1002
    tinycss2                  1.2.1            py38h06a4308_0
    tk                        8.6.12               h1ccaba5_0
    toml                      0.10.2             pyhd3eb1b0_0
    tomli                     2.0.1            py38h06a4308_0
    tomlkit                   0.11.1           py38h06a4308_0
    tornado                   6.2              py38h5eee18b_0
    tqdm                      4.64.1                   pypi_0    pypi
    traitlets                 5.4.0                    pypi_0    pypi
    typing-extensions         4.3.0            py38h06a4308_0
    typing_extensions         4.3.0            py38h06a4308_0
    ujson                     5.4.0            py38h6a678d5_0
    unidecode                 1.2.0              pyhd3eb1b0_0
    urllib3                   1.26.12          py38h06a4308_0
    usgs                      0.3.4                    pypi_0    pypi
    watchdog                  2.1.6            py38h06a4308_0
    wcwidth                   0.2.5              pyhd3eb1b0_0
    webencodings              0.5.1                    py38_1
    werkzeug                  2.2.2                    pypi_0    pypi
    whatthepatch              1.0.2            py38h06a4308_0
    wheel                     0.37.1             pyhd3eb1b0_0
    whoosh                    2.7.4                    pypi_0    pypi
    wrapt                     1.14.1           py38h5eee18b_0
    wurlitzer                 3.0.2            py38h06a4308_0
    xarray                    2022.11.0                pypi_0    pypi
    xerces-c                  3.2.3                h780794e_0
    xlrd                      2.0.1                    pypi_0    pypi
    xz                        5.2.6                h5eee18b_0
    yaml                      0.2.5                h7b6447c_0
    yapf                      0.31.0             pyhd3eb1b0_0
    zeromq                    4.3.4                h2531618_0
    zipp                      3.9.0                    pypi_0    pypi
    zlib                      1.2.12               h5eee18b_3
    zstd                      1.5.2                ha4553b6_0
    
    pip list
    $ pip list
    
    Package                       Version
    ----------------------------- -----------
    affine                        2.3.1
    alabaster                     0.7.12
    arrow                         1.2.3
    astroid                       2.11.7
    asttokens                     2.0.8
    atomicwrites                  1.4.0
    attrs                         22.1.0
    autopep8                      1.6.0
    Babel                         2.9.1
    backcall                      0.2.0
    beautifulsoup4                4.11.1
    binaryornot                   0.4.4
    black                         22.6.0
    bleach                        4.1.0
    boto3                         1.24.91
    botocore                      1.27.91
    brotlipy                      0.7.0
    cdsapi                        0.5.1
    certifi                       2022.9.24
    cffi                          1.15.1
    cftime                        1.6.2
    chardet                       4.0.0
    charset-normalizer            2.1.1
    click                         8.1.3
    click-plugins                 1.1.1
    cligj                         0.7.2
    cloudpickle                   2.0.0
    colorama                      0.4.5
    contourpy                     1.0.5
    cookiecutter                  1.7.3
    cryptography                  38.0.1
    cycler                        0.11.0
    debugpy                       1.6.3
    decorator                     5.1.1
    defusedxml                    0.7.1
    diff-match-patch              20200713
    dill                          0.3.6
    docutils                      0.18.1
    ecmwf-api-client              1.6.3
    entrypoints                   0.4
    eodag                         2.6.0
    eodag-sentinelsat             0.4.1
    ETo                           1.1.0
    executing                     1.1.1
    fastjsonschema                2.16.2
    Fiona                         1.8.21
    flake8                        4.0.1
    flasgger                      0.9.5
    Flask                         2.2.2
    fonttools                     4.37.4
    GDAL                          3.4.1
    geojson                       2.5.0
    geomet                        0.3.0
    geopandas                     0.11.1
    h5netcdf                      1.1.0
    h5py                          3.7.0
    html2text                     2020.1.16
    idna                          3.4
    imagesize                     1.4.1
    importlib-metadata            5.0.0
    importlib-resources           5.10.0
    inflection                    0.5.1
    intervaltree                  3.1.0
    ipykernel                     6.16.0
    ipython                       8.5.0
    ipython-genutils              0.2.0
    isort                         5.9.3
    itsdangerous                  2.1.2
    jedi                          0.18.1
    jeepney                       0.7.1
    jellyfish                     0.9.0
    Jinja2                        3.1.2
    jinja2-time                   0.2.0
    jmespath                      1.0.1
    jsonpath-ng                   1.5.3
    jsonschema                    4.16.0
    jupyter_client                7.4.2
    jupyter-core                  4.11.1
    jupyterlab-pygments           0.1.2
    keyring                       23.4.0
    kiwisolver                    1.4.4
    landsatxplore                 0.13.0
    lazy-object-proxy             1.6.0
    lxml                          4.9.1
    Markdown                      3.4.1
    MarkupSafe                    2.1.1
    matplotlib                    3.6.1
    matplotlib-inline             0.1.6
    mccabe                        0.7.0
    memory-profiler               0.61.0
    mistune                       2.0.4
    mkl-fft                       1.3.1
    mkl-random                    1.2.2
    mkl-service                   2.4.0
    multiprocess                  0.70.14
    munch                         2.5.0
    mypy-extensions               0.4.3
    nbclient                      0.5.13
    nbconvert                     6.5.4
    nbformat                      5.5.0
    nest-asyncio                  1.5.6
    netCDF4                       1.6.2
    numpy                         1.23.4
    numpydoc                      1.5.0
    opencv-python                 4.6.0.66
    OWSLib                        0.27.2
    p-tqdm                        1.4.0
    packaging                     21.3
    pandas                        1.5.0
    pandocfilters                 1.5.0
    parso                         0.8.3
    pathos                        0.3.0
    pathspec                      0.9.0
    pexpect                       4.8.0
    pickleshare                   0.7.5
    Pillow                        9.2.0
    pip                           22.3.1
    pkgutil_resolve_name          1.3.10
    platformdirs                  2.5.2
    pluggy                        1.0.0
    ply                           3.11
    pox                           0.3.2
    poyo                          0.5.0
    ppft                          1.7.6.6
    prompt-toolkit                3.0.31
    psutil                        5.9.2
    ptyprocess                    0.7.0
    pure-eval                     0.2.2
    pycodestyle                   2.8.0
    pycparser                     2.21
    pydocstyle                    6.1.1
    pyflakes                      2.4.0
    Pygments                      2.13.0
    pylint                        2.14.5
    pyls-spyder                   0.4.0
    pyOpenSSL                     22.0.0
    pyparsing                     3.0.9
    pyproj                        3.4.0
    PyQt5-sip                     12.11.0
    pyrsistent                    0.18.1
    pyshp                         2.3.1
    PySocks                       1.7.1
    pystac                        1.6.1
    python-dateutil               2.8.2
    python-lsp-black              1.2.1
    python-lsp-jsonrpc            1.0.0
    python-lsp-server             1.5.0
    python-slugify                5.0.2
    pytz                          2022.4
    pyxdg                         0.27
    PyYAML                        6.0
    pyzmq                         24.0.1
    QDarkStyle                    3.0.2
    qstylizer                     0.1.10
    QtAwesome                     1.0.3
    qtconsole                     5.3.2
    QtPy                          2.2.0
    rasterio                      1.3.2
    requests                      2.28.1
    requests-futures              1.0.0
    rope                          0.22.0
    Rtree                         0.9.7
    s3transfer                    0.6.0
    scipy                         1.9.2
    SecretStorage                 3.3.1
    sentinelsat                   1.1.1
    setuptools                    65.5.0
    Shapely                       1.8.5.post1
    sip                           6.6.2
    six                           1.16.0
    snowballstemmer               2.2.0
    snuggs                        1.4.7
    sortedcontainers              2.4.0
    soupsieve                     2.3.2.post1
    Sphinx                        5.0.2
    sphinxcontrib-applehelp       1.0.2
    sphinxcontrib-devhelp         1.0.2
    sphinxcontrib-htmlhelp        2.0.0
    sphinxcontrib-jsmath          1.0.1
    sphinxcontrib-qthelp          1.0.3
    sphinxcontrib-serializinghtml 1.1.5
    spyder-kernels                2.3.3
    stack-data                    0.5.1
    text-unidecode                1.3
    textdistance                  4.2.1
    three-merge                   0.1.1
    tinycss                       0.4
    tinycss2                      1.2.1
    toml                          0.10.2
    tomli                         2.0.1
    tomlkit                       0.11.1
    tornado                       6.2
    tqdm                          4.64.1
    traitlets                     5.4.0
    typing_extensions             4.3.0
    ujson                         5.4.0
    Unidecode                     1.2.0
    urllib3                       1.26.12
    usgs                          0.3.4
    watchdog                      2.1.6
    wcwidth                       0.2.5
    webencodings                  0.5.1
    Werkzeug                      2.2.2
    whatthepatch                  1.0.2
    wheel                         0.37.1
    Whoosh                        2.7.4
    wrapt                         1.14.1
    wurlitzer                     3.0.2
    xarray                        2022.11.0
    xlrd                          2.0.1
    yapf                          0.31.0
    zipp                          3.9.0
    
    bug 
    opened by jeremy-cesbio 1
  • Setting Custom provider as preferred doesn't work

    Setting Custom provider as preferred doesn't work

    Describe the bug Setting custom provider as preferred doesn't seem to work.

    Code To Reproduce

    from eodag import setup_logging
    from eodag.api.core import EODataAccessGateway
    
    if __name__ == "__main__":
        # Create an EODAG custom STAC provider
        setup_logging(verbose=3)
        dag = EODataAccessGateway()
    
        # Add the custom STAC provider + output + login an password
        outputs_prefix = r"D:\capella_eodag"
    
        dag.update_providers_config("""
        capella_provider:
            search:
                type: StaticStacSearch
                api_endpoint: https://capella-open-data.s3.us-west-2.amazonaws.com/stac/capella-open-data-by-product-type/capella-open-data-slc/collection.json
            products:
                GENERIC_PRODUCT_TYPE:
                    productType: '{productType}'
            download:
                type: AwsDownload
                flatten_top_dirs: True
                outputs_prefix: %s
        """ % (outputs_prefix))
    
        # Set the custom STAC provider as preferred
        dag.set_preferred_provider("capella_provider")
        print(dag.get_preferred_provider())
    

    Output

    2022-12-07 10:40:35,074 eodag.config                     [INFO    ] (config           ) Loading user configuration from: C:\Users\rbraun\.config\eodag\eodag.yml
    2022-12-07 10:40:35,236 eodag.core                       [INFO    ] (core             ) usgs: provider needing auth for search has been pruned because no crendentials could be found
    2022-12-07 10:40:35,236 eodag.core                       [INFO    ] (core             ) aws_eos: provider needing auth for search has been pruned because no crendentials could be found
    2022-12-07 10:40:35,360 eodag.core                       [DEBUG   ] (core             ) Opening product types index in C:\Users\rbraun\.config\eodag\.index
    2022-12-07 10:40:35,370 eodag.core                       [INFO    ] (core             ) Locations configuration loaded from C:\Users\rbraun\.config\eodag\locations.yml
    2022-12-07 10:40:35,370 eodag.config                     [INFO    ] (config           ) capella_provider: unknown provider found in user conf, trying to use provided configuration
    ('peps', 1)
    

    Environment:

    • Python version: Python 3.9.13
    • EODAG version: eodag (Earth Observation Data Access Gateway): version 2.7.0

    Additional context

    The custom provider comes from #519

    bug 
    opened by remi-braun 5
  • Query by tile: unfound Sentinel-2 product

    Query by tile: unfound Sentinel-2 product

    Describe the bug When querying a Sentinel-2 product by tile (with the provided shapefile), the result may be empty for partial products (edge of swath) if the tile centroid is outside the S2 footprint.

    Code To Reproduce

    import os
    import tempfile
    
    from eodag import EODataAccessGateway, setup_logging
    from eodag.config import override_config_from_file
    
    from eodownload.dir_utils import get_data_directory
    
    OUTPUT = r"D:\_EODAG\OUTPUT\"
    
    os.environ["EODAG__EARTH_SEARCH__DOWNLOAD__EXTRACT"] = "false"
    os.environ["EODAG__EARTH_SEARCH__DOWNLOAD__OUTPUTS_PREFIX"] = OUTPUT
    os.environ[
        "EODAG__EARTH_SEARCH__AUTH__CREDENTIALS__AWS_ACCESS_KEY_ID"
    ] = "xxx"
    os.environ[
        "EODAG__EARTH_SEARCH__AUTH__CREDENTIALS__AWS_SECRET_ACCESS_KEY"
    ] = "xxx"
    os.environ["EODAG__EARTH_SEARCH__AUTH__CREDENTIALS__AWS_PROFILE"] = "image-download"
    
    if __name__ == "__main__":
        setup_logging(3)  # DEBUG level
    
        with tempfile.TemporaryDirectory() as tmp:
    
            locations_yaml_content = """
                shapefiles:
                  - name: s2_tiles_centroids
                    path: {}
                    attr: index
                """.format(
                get_data_directory() / "s2_tiles_centroids.shp"
            )
    
            custom_loc = os.path.join(tmp, "custom_locations.yml")
            with open(custom_loc, "w") as f_yml:
                f_yml.write(locations_yaml_content.strip())
    
            # Create gateway
            dag = EODataAccessGateway(locations_conf_path=custom_loc)
            dag.set_preferred_provider("earth_search")
    
            # Query all
            products = dag.search_all(
                start="2022-05-07",
                end="2022-05-08",
                **{
                    "productType": "S2_MSI_L2A",
                    "locations": dict(s2_tiles_centroids="36NUG"),
                }
            )
    
            # Download all
            dag.download_all(products)
    
    

    Output

    2022-11-25 11:46:40,486 eodag.config                     [INFO    ] (config           ) Loading user configuration from: C:\Users\rbraun\.config\eodag\eodag.yml
    2022-11-25 11:46:40,631 eodag.core                       [INFO    ] (core             ) usgs: provider needing auth for search has been pruned because no crendentials could be found
    2022-11-25 11:46:40,631 eodag.core                       [INFO    ] (core             ) aws_eos: provider needing auth for search has been pruned because no crendentials could be found
    2022-11-25 11:46:40,755 eodag.core                       [DEBUG   ] (core             ) Opening product types index in C:\Users\rbraun\.config\eodag\.index
    2022-11-25 11:46:40,760 eodag.core                       [INFO    ] (core             ) Locations configuration loaded from C:\Users\rbraun\AppData\Local\Temp\tmpygpootyf\custom_locations.yml
    2022-11-25 11:46:40,760 eodag.config                     [INFO    ] (config           ) Loading user configuration from: D:\_EODOWNLOAD\eodownload\eodownload\data\sertit_conf_template.yml
    2022-11-25 11:46:40,776 eodag.config                     [INFO    ] (config           ) usgs: unknown provider found in user conf, trying to use provided configuration
    2022-11-25 11:46:40,776 eodag.config                     [INFO    ] (config           ) aws_eos: unknown provider found in user conf, trying to use provided configuration
    2022-11-25 11:46:41,335 eodag.core                       [DEBUG   ] (core             ) Searching for all the products with provider earth_search and a maximum of 500 items per page.
    2022-11-25 11:46:42,121 eodag.core                       [INFO    ] (core             ) Searching product type 'S2_MSI_L2A' on provider: earth_search
    2022-11-25 11:46:42,122 eodag.core                       [DEBUG   ] (core             ) Using plugin class for search: StacSearch
    2022-11-25 11:46:42,122 eodag.core                       [INFO    ] (core             ) Iterate search over multiple pages: page #1
    2022-11-25 11:46:42,122 eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Mapping eodag product type to provider product type
    2022-11-25 11:46:42,122 eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Getting provider product type definition parameters for S2_MSI_L2A
    2022-11-25 11:46:42,203 eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Building the query string that will be used for search
    2022-11-25 11:46:42,203 eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Retrieving queryable metadata from metadata_mapping
    2022-11-25 11:46:42,203 eodag.plugins.search.qssearch    [INFO    ] (qssearch         ) Sending search request: https://earth-search.aws.element84.com/v0/search
    2022-11-25 11:46:42,203 eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Query parameters: {'collections': ['sentinel-s2-l2a'], 'datetime': '2022-05-07T00:00:00.000Z/2022-05-08T00:00:00.000Z', 'intersects': {'type': 'Point', 'coordinates': [31.695828581223143, 1.3127651504842646]}, 'limit': 500, 'page': 1}
    2022-11-25 11:46:43,107 eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Next page Query-object could not be collected
    2022-11-25 11:46:43,122 eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Next page merge could not be collected
    2022-11-25 11:46:43,122 eodag.plugins.search.qssearch    [DEBUG   ] (qssearch         ) Adapting 0 plugin results to eodag product representation
    2022-11-25 11:46:43,122 eodag.core                       [DEBUG   ] (core             ) Iterate over pages: last products found on page 0
    2022-11-25 11:46:43,122 eodag.core                       [INFO    ] (core             ) Found 0 result(s) on provider 'earth_search'
    2022-11-25 11:46:43,122 eodag.core                       [INFO    ] (core             ) Empty search result, nothing to be downloaded !
    

    However a product does exist here: 2022-11-25_12h04_40

    Environment:

    • Python version: 3.9.13
    • EODAG version: version 2.6.2.post1
    bug 
    opened by remi-braun 1
  • whoosh opened file

    whoosh opened file

    Whoosh index files to not seem to be properly closed. Using lsof +D ~/.config/eodag/.index we can see that a *.seg file stays opened after using guess_product_type().

    This can cause an error when trying to remove the directory on windows:

      self = <TemporaryDirectory 'D:\\a\\eodag\\eodag\\.tox\\py37\\tmp\\tmp3vn9c2q2'>
      
          def cleanup(self):
              if self._finalizer.detach():
      >           _shutil.rmtree(self.name)
    
    PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'D:\\a\\eodag\\eodag\\.tox\\py37\\tmp\\tmp3vn9c2q2\\.config\\eodag\\.index\\MAIN_3urmr60f1bed1akw.seg'
    

    Error occurred while attempting to test guess_product_type through cli.

    bug 
    opened by sbrunato 0
  • Definition of timeout/wait option from configuration file

    Definition of timeout/wait option from configuration file

    It would be interesting to be able to configure timeout/wait option for download*() methods from the configuration file.

    In case it's already possible, I haven't seen it documented on https://eodag.readthedocs.io/en/stable/getting_started_guide/configure.html

    enhancement 
    opened by LucHermitte 2
Releases(v2.7.0)
  • v2.7.0(Nov 29, 2022)

  • v2.6.2(Nov 15, 2022)

  • v2.6.1(Oct 19, 2022)

  • v2.6.0(Oct 7, 2022)

    Source code(tar.gz)
    Source code(zip)
  • v2.5.2(Jul 5, 2022)

  • v2.5.1(Jun 27, 2022)

  • v2.5.0(Jun 7, 2022)

    Source code(tar.gz)
    Source code(zip)
  • v2.4.0(Mar 9, 2022)

    Source code(tar.gz)
    Source code(zip)
  • v2.3.4(Oct 8, 2021)

    • Link to the new eodag Jupyterlab extension: eodag-labextension (#352)
    • STAC client and server update to STAC 1.0.0 (#347)
    • Fixes get_quicklook() for onda provider (#344, thanks @drnextgis)
    • Fixed issue when downloading S2_MSI_L2A products from mundi (#350)
    • Various minor fixes and improvements (#340)(#341)(#345)
    Source code(tar.gz)
    Source code(zip)
  • v2.3.3(Aug 11, 2021)

  • v2.3.2(Jul 29, 2021)

    • Enable additional arguments like productType when searching by id (#329)
    • Prevent EOL auto changes on windows causing docker crashes (#324)
    • Configurable eodag logging in docker stac-server (#323)
    • Fixes missing productType in product properties when searching by id (#320)
    • Fixes duplicate logging in search_all() (#330)
    • Various minor fixes and improvements (#319)(#321)
    Source code(tar.gz)
    Source code(zip)
  • v2.3.1(Jul 9, 2021)

    • Dockerfile update to be compatible with stac-browser v2.0 (#314)
    • Adds new notebook extra dependency (#317)
    • EOProduct drivers definition update (#316)
    Source code(tar.gz)
    Source code(zip)
  • v2.3.0(Jun 24, 2021)

    New since v2.2.0:

    • Re-structured and more complete documentation (#233, and also #224, #254, #282, #287, #301)
    • Homogenized inconsistent paths returned by download() and download_all() methods (#244)(#292)
    • Rewritten progress callback mechanism (#276)(#285)
    • Sentinel products SAFE-format build for STAC AWS providers (#218)
    • New CLI optional --quicklooks flag in eodag download command (#279, thanks @ahuarte47)
    • New product types for Sentinel non-SAFE products (#228)
    • Creodias metadata mapping update (#294)
    • setup_logging() is now easier to import (#221)
    • get_logging_verbose() function added (#283)
    • Documentation on how to request USGS M2M API access (#269)
    • User friendly parameters mapping documentation (#299)
    • Auto extract if extract is not set (#249)
    • Fixed how download_all() updates the passed list of products (#253)
    • Fixed user config file loading with settings of providers from ext plugin (#235, thanks @ahuarte47)
    • Improved and less strict handling of misconfigured user settings (#293)(#296)
    • ISO 8601 formatted datetimes accepted by all providers (#257)
    • GENERIC_PRODUCT_TYPE not returned any more by list_product_types() (#261)
    • Warning displayed when searching with non preferred provider (#260)
    • Search kwargs used for guessing a product type not propagated any more (#248)
    • Deprecated load_stac_items(), StaticStacSearch search plugin should be used instead (#225)
    • ipywidgets no more needed in NotebookWidgets (#223)
    • Various minor fixes and improvements (#219)(#246)(#247)(#258)(#233)(#273)(#274)(#280)(#284)(#288)(#290)(#295)

    New since v2.3.0b1:

    • Removed Sentinel-3 products not available on peps any more (#304, thanks @tpfd)
    • Prevent display_html() in ipython shell (#307)
    • Fixed plugins reload after having updated providers settings from user configuration (#306)
    Source code(tar.gz)
    Source code(zip)
  • v2.3.0b1(Jun 11, 2021)

    • Re-structured and more complete documentation (#233, and also #224, #254, #282, #287, #301)
    • Homogenized inconsistent paths returned by download() and download_all() methods (#244)(#292)
    • Rewritten progress callback mechanism (#276)(#285)
    • Sentinel products SAFE-format build for STAC AWS providers (#218)
    • New CLI optional --quicklooks flag in eodag download command (#279, thanks @ahuarte47)
    • New product types for Sentinel non-SAFE products (#228)
    • Creodias metadata mapping update (#294)
    • setup_logging() is now easier to import (#221)
    • get_logging_verbose() function added (#283)
    • Documentation on how to request USGS M2M API access (#269)
    • User friendly parameters mapping documentation (#299)
    • Auto extract if extract is not set (#249)
    • Fixed how download_all() updates the passed list of products (#253)
    • Fixed user config file loading with settings of providers from ext plugin (#235, thanks @ahuarte47)
    • Improved and less strict handling of misconfigured user settings (#293)(#296)
    • ISO 8601 formatted datetimes accepted by all providers (#257)
    • GENERIC_PRODUCT_TYPE not returned any more by list_product_types() (#261)
    • Warning displayed when searching with non preferred provider (#260)
    • Search kwargs used for guessing a product type not propagated any more (#248)
    • Deprecated load_stac_items(), StaticStacSearch search plugin should be used instead (#225)
    • ipywidgets no more needed in NotebookWidgets (#223)
    • Various minor fixes and improvements (#219)(#246)(#247)(#258)(#233)(#273)(#274)(#280)(#284)(#288)(#290)(#295)
    Source code(tar.gz)
    Source code(zip)
  • v2.2.0(Mar 26, 2021)

    • New search_all and search_iter_page methods to simplify pagination handling (GH-190)
    • Docker-compose files for STAC API server with STAC-browser (GH-183)
    • Fixed USGS plugin which now uses Machine-to-Machine (M2M) API (GH-209)
    • Windows support added in Continuous Integration (GH-192)
    • Fixes issue with automatically load configution from EODAG external plugins, fixes GH-184
    • More explicit signature for setup_logging, fixes GH-197
    • Various minor fixes
    Source code(tar.gz)
    Source code(zip)
  • v2.1.1(Mar 18, 2021)

  • v2.1.0(Mar 10, 2021)

    • earth_search and usgs_satapi_aws (Landsat Collection 2) as new providers;
    • Updated HTTPDownload plugin, handling products with multiple assets;
    • New plugin AwsAuth, enables AWS authentication using no-sign-request, profile, ~/.aws/*;
    • New search plugin StaticStacSearch and updated STAC client tutorial;
    • New tutorial for Copernicus DEM;
    • UTC timezone set for dates by default;
    • Locations must now be passed to search() method as a dictionnary;
    • Metadata mapping update and uniformization, fixes GH-154;
    • Various minor fixes and tests update;
    • Drop support of Python 3.5.
    Source code(tar.gz)
    Source code(zip)
  • v2.0(Jan 28, 2021)

    New since v2.0b2:

    • User can add a new provider using Python API
    • More download options using Python API, fixes GH-145 and GH-112
    • New tutorials for STAC and search by geometry, fixes GH-139
    • New crunches FilterDate, FilterProperty and updated FilterOverlap, fixes GH-137
    • Simplified requirements. No more dependencies with GDAL (fiona), and jq
    • Better credentials error handling
    • Documentation and tutorials update
    • Various minor fixes, code refactorization, and tests update

    New since v1.6.0:

    • STAC API compliant REST server
    • Common configuration for STAC providers
    • astraea_eod as new STAC provider
    • Search by geometry / bbox / location name, fixes GH-49
    • New method deserialize_and_register, fixes GH-140
    • Load static stac catalogs as SearchResult
    • Search on unknown product types using GENERIC_PRODUCT_TYPE
    • get_data, drivers and rpc server moved to eodag-cube
    • Removed fixed dependencies, fixes GH-82
    • Use template as default locations configuration file
    • removed Python 2.7 support
    Source code(tar.gz)
    Source code(zip)
  • v2.0b2(Dec 18, 2020)

    • New method deserialize_and_register, fixes GH-140
    • Load static stac catalogs as SearchResult
    • Search on unknown product types using GENERIC_PRODUCT_TYPE
    • get_data, drivers and rpc server moved to eodag-cube
    • Removed fixed dependencies, fixes GH-82
    • Use locations conf template by default
    Source code(tar.gz)
    Source code(zip)
  • v2.0b1(Dec 2, 2020)

    • STAC API compliant REST server
    • Common configuration for STAC providers
    • astraea_eod as new STAC provider
    • Search by geometry / bbox / location name, fixes #49
    • removed Python 2.7 support
    Source code(tar.gz)
    Source code(zip)
pwy - A simple weather tool.

A simple weather tool. I made this as a way for me to learn Python, API, and PyPi packaging. Name changed from wwy to pwy.

Clint 105 Dec 31, 2022
Automaton - python script to execute bash command based on changes in size of a file.

automaton python script to execute given command = everytime size of a given file changes,hence everytime a file is modified.(almost) download automa

asrar bhat 1 Jan 03, 2022
A simple file transfer tools, similar to rz / sz but compatible with tmux (control mode), which works with iTerm2 and has a nice progress bar

trzsz A simple file transfer tools, similar to rz/sz but compatible with tmux (control mode), which works with iTerm2 and has a nice progress bar. Why

561 Jan 05, 2023
Colab-xterm allows you to open a terminal in a cell

colab-xterm Colab-xterm allows you to open a terminal in a cell. Usage Install package and load the extension !pip install git+https://github.com/popc

InfuseAI 194 Dec 29, 2022
A command-line based, minimal torrent streaming client made using Python and Webtorrent-cli.

ABOUT A command-line based, minimal torrent streaming client made using Python and Webtorrent-cli. Installation pip install -r requirements.txt It use

Janardon Hazarika 17 Dec 11, 2022
jenkins-tui is a terminal based user interface for Jenkins.

jenkins-tui 📦 jenkins-tui is a terminal based user interface for Jenkins. 🚧 ⚠️ This app is a prototype and in very early stages of development. Ther

Craig Gumbley 22 Oct 24, 2022
CLI tool to fix linked references for dates.

Fix Logseq dates This is a CLI tool to fix the date references following a change in date format since the current version (0.4.4) of Logseq does not

Isaac Dadzie 5 May 18, 2022
Wordle-cli - Command-line clone of Josh Wardle's WORDLE

Command-line clone of Josh Wardle's WORDLE, inspired by Paul Battley's Ruby vers

Klipspringer 32 Jan 03, 2023
Projeto Reverse Shell For Python

Use com sabedoria!!! Modo de uso: Linux (inclui Android e Mac): - apt-get update - apt install python3 (ou "python" apenas) - git clone https://github

1 Jan 03, 2022
CLTools provides various tools and command to use in the terminal.

CLTools provides various tools and command to use in the terminal. As of date, CLTools is only able to generate temporary email addresses and receive emails. There are plans to integrate more tools a

Ashwin Chugh 2 Feb 14, 2022
A command-line utility that creates projects from cookiecutters (project templates), e.g. Python package projects, VueJS projects.

Cookiecutter A command-line utility that creates projects from cookiecutters (project templates), e.g. creating a Python package project from a Python

18.6k Dec 30, 2022
Openstack bucket retention cli

Openstack bucket retention cli

Fatih Sarhan 3 Apr 03, 2022
Doro is a CLI based pomodoro app and countdown timer application built using python.

Doro - CLI based pomodoro app Doro is a CLI based pomodoro app and countdown timer application built using python. Install $ pip install doro Usage Po

Suresh Kumar 14 May 23, 2022
Run an FFmpeg command and see the percentage progress and ETA.

Run an FFmpeg command and see the percentage progress and ETA.

25 Dec 22, 2022
Tools hacking termux in the name ant-attack

Hello friends, I am ama.player0000. Web developer, software, Android command line (termux). (1)=Well, ant-attack tool is a tool to attack sites and disable them. (2)=You can use those CCTV servers, s

༺AMA.PLAYER༻ 1 Dec 17, 2021
A simple CLI tool for tracking Pikud Ha'oref alarms.

Pikud Ha'oref Alarm Tracking A simple CLI tool for tracking Pikud Ha'oref alarms. Polls the unofficial API endpoint every second for incoming alarms.

Yuval Adam 24 Oct 10, 2022
Get Air Quality Index for your city/country 😷

Air Quality Index CLI Get Air Quality index for your City. Installation $ pip install air-quality-cli Contents Air Quality Index CLI Installation Cont

Yankee 40 Oct 21, 2022
Simple CLI for managing Postgres databases in Flask.

Overview Simple CLI that provides the following commands: flask psql create flask psql init flask psql drop flask psql setup: create → init flask psql

Daniel Reeves 21 Oct 03, 2022
Yts-cli-streamer - A CLI movie streaming client which works on yts.mx API written in python

YTSP It is a CLI movie streaming client which works on yts.mx API written in pyt

1 Feb 05, 2022
DShell, a tool that combines with discord.py and Jishaku to present to you, shell channels.

Discord shell or dshell for short is a Python package that combines with discord.py and Jishaku to transform an ordinary Discord channel into one capable of running bash commands using a Discord bot.

11 Nov 15, 2022