A utility to search, download and process Landsat 8 satellite imagery

Overview

Landsat-util

https://travis-ci.org/developmentseed/landsat-util.svg?branch=master Downloads License

Landsat-util is a command line utility that makes it easy to search, download, and process Landsat imagery.

Docs

For full documentation visit: https://pythonhosted.org/landsat-util/

To run the documentation locally:

$ pip install -r requirements/dev.txt
$ cd docs
$ make html

Recently Added Features

  • Improved pansharpening
  • Use BQA bands for cloud/snow coverage and use in color correction
  • Add support for different NDVI color maps (three included)
  • Add support for image clipping using the new --clip flag

Change Log

See CHANGES.txt.

Comments
  • Searching returns no result

    Searching returns no result

    Hi people!

    landsat is not returning any results. For example:

    $ landsat search -p 228,81 --latest 5
    {
        "limit": 25, 
        "results": [], 
        "status": "SUCCESS", 
        "total": 0, 
        "total_returned": 0
    }
    ['Search completed!']
    
    $ landsat search -p 23,35 --latest 5
    {
        "limit": 25, 
        "results": [], 
        "status": "SUCCESS", 
        "total": 0, 
        "total_returned": 0
    }
    ['Search completed!']
    
    $ landsat search -p 226,84 --latest 5
    {
        "limit": 25, 
        "results": [], 
        "status": "SUCCESS", 
        "total": 0, 
        "total_returned": 0
    }
    

    Any suggestion/pointer to solve this problem? Thanks in advance!

    Matias

    opened by matiaslee 27
  • doesn't work

    doesn't work

    Its probably a simple fix but I'm not an IT guy. I keep getting the following error

    landsat search --cloud 4 --start "january 1 2014" --end "january 10 2014" -p 009,045 Traceback (most recent call last): File "/Users/schull/anaconda/bin/landsat", line 5, in import landsat.landsat File "/Users/schull/anaconda/lib/python2.7/site-packages/landsat/landsat.py", line 21, in from image import Process, FileDoesNotExist File "/Users/schull/anaconda/lib/python2.7/site-packages/landsat/image.py", line 13, in import rasterio File "/Users/schull/anaconda/lib/python2.7/site-packages/rasterio/init.py", line 8, in from rasterio._base import eval_window, window_shape, window_index ImportError: dlopen(/Users/schull/anaconda/lib/python2.7/site-packages/rasterio/_base.so, 2): Library not loaded: libgdal.1.dylib Referenced from: /Users/schull/anaconda/lib/python2.7/site-packages/rasterio/_base.so Reason: image not found

    Any ideas how to fix it?

    opened by bucricket 19
  • Landsat8 download problem from USGS-EROS for L1C product not available on AWS S3 or Google Storage

    Landsat8 download problem from USGS-EROS for L1C product not available on AWS S3 or Google Storage

    I am not having success trying to load a set of specific Landsat-8 scenes from USGS EROS site -- for scenes not available on AWS or Google-- trying this from an OSX machine.

    I get the scene names using '> landsat search' just fine... With '> landsat download', I get "product not available on AWS S3 or Google Storage"... one of these scenes, as an example, is LC81290582016007LGN02.

    When I supply my USGS EROS credentials --username --password (I do have m2m / special inventory access enabled on this account), I get the error message below. Any help greatly appreciated...

    #---- Time spent : 0.75 seconds Traceback (most recent call last): File "/usr/local/bin/landsat", line 11, in sys.exit(main()) File "/usr/local/lib/python2.7/site-packages/landsat/landsat.py", line 491, in main exit(*main(args)) File "/usr/local/lib/python2.7/site-packages/landsat/landsat.py", line 408, in main files = d.download(args.scenes, bands) File "/usr/local/lib/python2.7/site-packages/landsat/downloader.py", line 80, in download files.append(self.usgs_eros(scene, self.download_dir)) File "/usr/local/lib/python2.7/site-packages/landsat/downloader.py", line 95, in usgs_eros error_tree = ElementTree.fromstring(str(e.message)) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1311, in XML parser.feed(text) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1653, in feed self._raiseerror(v) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1517, in _raiseerror raise err xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 171, column 22

    opened by gybely 12
  • returned non-zero exit status 4

    returned non-zero exit status 4

    Hi. Congratulations on this very usefull tool. I'm having trouble processing the images after dowloading them. Here's the console text:

    "C:\Programs\landsat-util-master>python C:\Programs\landsat-util-master\landsat\l andsat.py search --download --imageprocess --cloud 20 --start "january 1 2014" - -end "april 10 2014" pr 204 032 1 items were found { "limit": 100, "results": [ { "cloud": 4.16, "date": "2014-03-19", "path": "204", "row": "032", "sat_type": "L8", "sceneID": "LC82040322014078LGN00", "thumbnail": "http://earthexplorer.usgs.gov/browse/landsat_8/2014/20 4/032/LC82040322014078LGN00.jpg" } ], "status": "SUCCESS", "total": 1, "total_returned": 1 } Starting the download: Skipping existing item: file://C:\Users\Administrator/landsat/zip/LC820403220140 78LGN00.tar.bz 1 images were downloaded c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00 folder cr eated c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/warp fold er created c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/scaled fo lder created c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/final fol der created Unzipping LC82040322014078LGN00 - It might take some time Creating output file that is 7828P x 7932L. Processing input file c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC8204032 2014078LGN00/LC82040322014078LGN00_B4.TIF. 0...10...20...30...40...50...60...70...80...90...100 - done. Creating output file that is 7828P x 7932L. Processing input file c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC8204032 2014078LGN00/LC82040322014078LGN00_B3.TIF. 0...10...20...30...40...50...60...70...80...90...100 - done. Creating output file that is 7828P x 7932L. Processing input file c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC8204032 2014078LGN00/LC82040322014078LGN00_B2.TIF. 0...10...20...30...40...50...60...70...80...90...100 - done. Creating output file that is 15655P x 15862L. Processing input file c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC8204032 2014078LGN00/LC82040322014078LGN00_B8.TIF. 0...10...20...30...40...50...60...70...80...90...100 - done. Starting the Min/Max process with designated -percent cut- for band 4 of LC82040 322014078LGN00 ... Finished processing band 4 of LC82040322014078LGN00 Starting the Min/Max process with designated -percent cut- for band 3 of LC82040 322014078LGN00 ... Finished processing band 3 of LC82040322014078LGN00 Starting the Min/Max process with designated -percent cut- for band 2 of LC82040 322014078LGN00 ... Finished processing band 2 of LC82040322014078LGN00 scaling pan to min max with 2% cut for band 4 Input file size is 7828, 7932 0...10...20...30...40...50...60...70...80...90...100 - done. scaling pan to min max with 2% cut for band 3 Input file size is 7828, 7932 0...10...20...30...40...50...60...70...80...90...100 - done. scaling pan to min max with 2% cut for band 2 Input file size is 7828, 7932 0...10...20...30...40...50...60...70...80...90...100 - done. Invalid Parameter - -combine Traceback (most recent call last): File "C:\Programs\landsat-util-master\landsat\landsat.py", line 285, in main() File "C:\Programs\landsat-util-master\landsat\landsat.py", line 282, in __main __ main(args) File "C:\Programs\landsat-util-master\landsat\landsat.py", line 236, in main p.full() File "C:\Programs\landsat-util-master\landsat\image_helper.py", line 108, in f ull self._combine() File "C:\Programs\landsat-util-master\landsat\image_helper.py", line 340, in _ combine subprocess.check_call(argv) File "C:\OSGeo4W64\apps\Python27\lib\subprocess.py", line 542, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['convert', '-identify', '-combine', u'c :\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/war p/LC82040322014078LGN00_B4.TIF', u'c:\users\admini~1\appdata\local\temp\1
    \tmpybepa2/LC82040322014078LGN00/warp/LC82040322014078LGN00_B3.TIF', u'c:\users \admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/warp/LC8204 0322014078LGN00_B2.TIF', u'c:\users\admini~1\appdata\local\temp\1\tmpybep a2/LC82040322014078LGN00/final/rgb-null.TIF']' returned non-zero exit status 4 "

    Installation Issues v0.2 
    opened by vascobnunes 12
  • landsat collection 1

    landsat collection 1

    Hi,

    I'm wondering if you are planning on implementing the new collection 1 information? The ESPA ordering system no longer stores the pre-collection data and so the sceneID information doesn't work anymore. We need the Landsat-product ID now.

    Mitch

    opened by bucricket 11
  • Loosen requirement versions

    Loosen requirement versions

    I essentially widened the allowed versions for all dependencies, taking care of specifying rasterio<1. The current setup has been tested in all Python >= 3.4 versions (see commit logs). I also updated the Dockerfile, hopefully fixing another issue.

    opened by astrojuanlu 10
  • Support for Copernicus Sentinel-2 imaginery

    Support for Copernicus Sentinel-2 imaginery

    This is just a far fetched open question: would be interesting to support the download / processing of imaginery from the Sentinel-2 satellites of the Copernicus mission? A few points of interest:

    The mission

    Multi-spectral data with 13 bands in the visible, near infrared, and short wave infrared part of the spectrum Systematic global coverage of land surfaces from 56° S to 84° N, coastal waters, and all of the Mediterranean Sea Revisiting every 5 days at the Equator under the same viewing conditions Spatial resolution of 10 m, 20 m and 60 m 290 km field of view Free and open data policy

    Imaginery

    Among others, there will be 4 bands at 10m resolution (50% more than Landsat-8): red, green, blue and VNIR. Nicely, there is no need to pansharpen in order to get images at max. resolution.

    bands

    Data access

    Data will be free to access. The good news are that there is an API to perform searches (including geographical bounding, cloud cover, date, etc.) and download quicklooks and products. This API is well documented here. The bad news are that the API requires user registration:

    curl -u username:password "https://scihub.esa.int/dhus/search?q=S2A*&format=json"
    

    Also, the images are distributed in a specific SAFE format which I'm not sure GDAL supports yet (it seems that GDAL supports Sentinel-1 SAFE, but not Sentinel-2 SAFE).


    As you can see the tools are not so ready yet, but I think it could be interesting to have this issue in mind in order to someday extend the functionality of landsat-util, no? The possibility of 50% higher resolution imaginary with 1/3 of the revisit time without the need of pansharpening (which highly increases the resources needed) is very tempting!

    What do you think? =)

    opened by JorgeGT 10
  • update join method for download URL

    update join method for download URL

    replaced join() with "/".join() for building google and Amazon S3 URLs that are compatible across platforms. Remove trailing "/" from S3 and Google base URL in settings.py

    opened by RoboDonut 9
  • Add an option for parallel downloading

    Add an option for parallel downloading

    As I've proposed in Issue #102, these are the modifications in order to allow the use of new command-line flag --use-aria2c true, in order to download the scenes using aria2c's parallel download. This is my first contribution so, in case something looks odd, just let me know, ok? Cheers.

    opened by vbenso 9
  • download band 8 in continuous download/process/pansharpen command

    download band 8 in continuous download/process/pansharpen command

    This command landsat download LC81410412015088LGN00 --bands 432 -p --pansharpen fails because band 8 is not downloaded from aws.

    Have to make sure that band 8 is downloaded if pansharpen flag is used for 2015 and newer imagery.

    opened by scisco 8
  • magick error

    magick error

    After updating Xcode to 4.6.3 and running brew doctor, still i get this error after running the install for landsat-util--has anyone seen this before?

    1 error generated. make[1]: *** [magick/magick_libMagickCore_6_Q16_la-annotate.lo] Error 1 make[1]: *** Waiting for unfinished jobs.... make: *** [install] Error 2

    opened by ebarry 8
  • Coordinates to clip the raster

    Coordinates to clip the raster

    Hi there, I am working on a raster file and want to clip an area from the raster using latitude and longitude. But Landsat API requires coordinates in WGS84 datum. How can I convert these lat and long coordinates into the desired format?. Thanks.

    opened by EhsanAlahi 0
  • Update dependencies for new rasterio version

    Update dependencies for new rasterio version

    Hi,

    Recently rasterio released version 1.0 and is currently at version 1.0.23. That has some important updates and features, and at the moment I cannot install landsat-util because of its rasterio version requirements (currently rasterio~=0.36)

    Would it be possible to update that?

    opened by alessioarena 1
  • Docker search SSL error

    Docker search SSL error

    $ sudo docker run -it -v ~/landsat:/root/landsat developmentseed/landsat-util:latest landsat search --lat 33.9436973 --lon -118.4073388       
    
    /usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
      InsecurePlatformWarning
    Traceback (most recent call last):
      File "/usr/local/bin/landsat", line 9, in <module>
        load_entry_point('landsat-util==0.13.1', 'console_scripts', 'landsat')()
      File "/landsat/landsat/landsat.py", line 488, in __main__
        print(main(args))
      File "/landsat/landsat/landsat.py", line 357, in main
        geojson=args.geojson)
      File "/landsat/landsat/search.py", line 96, in search
        r = requests.get('%s?search=%s&limit=%s' % (self.api_url, search_string, limit))
      File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 69, in get
        return request('get', url, params=params, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 50, in request
        response = session.request(method=method, url=url, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 465, in request
        resp = self.send(prep, **send_kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 573, in send
        r = adapter.send(request, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 431, in send
        raise SSLError(e, request=request)
    requests.exceptions.SSLError: [Errno 1] _ssl.c:510: error:14077438:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert internal error
    
    opened by palewire 1
  • Problem with using landsat-util downloads lansat8 2018 year data

    Problem with using landsat-util downloads lansat8 2018 year data

    When I tried to download Landsat 8 data with year of 2018 , then there was an error LC81210382018100LGN00 is not available on AWS S3 or Google Storage image

    opened by 120534 0
  • Would like to use landsat-util on windows

    Would like to use landsat-util on windows

    I'm using a windows 10 system. Should I use a docker and run it on Hyper-V? Is this the only way? Anyone familiar with the start-up could help me out? Thanks!

    opened by gracie2017 3
Releases(v0.13.1)
  • v0.13.1(Jan 18, 2017)

  • v0.13.0(Mar 25, 2016)

  • v0.12.2(Mar 25, 2016)

  • v0.12.1(Feb 18, 2016)

  • v0.12.0(Feb 18, 2016)

  • v0.10.0(Jan 5, 2016)

    • add support for bare json output
    • faster travis tests
    • add street search
    • apply ndvigrey to process and download
    • fixes #127
    • update commands help file
    • apply pansharpen and ndvi only if -p is used
    • download zip if bands are not specified
    • better handle url joins to a fix error on Windows
    • other small bug fixes
    Source code(tar.gz)
    Source code(zip)
  • v0.9.1(Jan 5, 2016)

  • v0.8.0(Oct 23, 2015)

    0.8.0 (2015-09-22)

    • Improved docs
    • Add --ndvi flag
    • Handle downloading new bands (10, 11, QA)
    • Improved color correction
    • Remove noise in pansharpened image processing
    Source code(tar.gz)
    Source code(zip)
  • v0.7.0(May 29, 2015)

    • New documentation
    • Deployed to readthedocs
    • Automate deployment to pypi
    • Adds docker support
    • skip unzipping if images already unzipped
    • add force-unzip flag
    • fix a bug where multiple downloads was not followed by multiple process #81
    • fix a bug where if scenes was downloaded from google instead of aws, process failed #84
    • download band 8 when pansharpen fixes #73
    Source code(tar.gz)
    Source code(zip)
  • v0.6.3(Apr 29, 2015)

    includes:

    • adjust lower rescaling bound, closes #66 for now
    • better pixel math for #71
    • update destination transform calculation for reprojection in polar regions
    Source code(tar.gz)
    Source code(zip)
  • v0.6.2(Apr 25, 2015)

  • v0.6.1(Apr 23, 2015)

    Includes:

    • Updated tests that run faster
    • New upload command for uploading final image to S3
    • New command for continuous upload, process and upload
    • Updated image processing that produces better color correction
    • Other bug fixes
    Source code(tar.gz)
    Source code(zip)
  • v0.5.1(Apr 8, 2015)

  • v0.5.0(Mar 10, 2015)

    Version 0.5.0 includes:

    • Fewer dependencies
    • Uses wheel on OSX
    • New image processing algorithm which is faster and require less storage
    • Updated API
    • Includes longitude latitude search
    • Improved console output
    • Accepts bands combinations for processing images
    • Includes a new faster downloader
    • More comprehensive tests (96% coverage)
    • Added to pypi pip install landsat-util
    • Brew formula removed
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Aug 28, 2014)

Owner
Development Seed
Development Seed
framework for large-scale SAR satellite data processing

pyroSAR A Python Framework for Large-Scale SAR Satellite Data Processing The pyroSAR package aims at providing a complete solution for the scalable or

John Truckenbrodt 389 Dec 21, 2022
Deal with Bing Maps Tiles and Pixels / WGS 84 coordinates conversions, and generate grid Shapefiles

PyBingTiles This is a small toolkit in order to deal with Bing Tiles, used i.e. by Facebook for their Data for Good datasets. Install Clone this repos

Shoichi 1 Dec 08, 2021
A toolbox for processing earth observation data with Python.

eo-box eobox is a Python package with a small collection of tools for working with Remote Sensing / Earth Observation data. Package Overview So far, t

13 Jan 06, 2022
glTF to 3d Tiles Converter. Convert glTF model to Glb, b3dm or 3d tiles format.

gltf-to-3d-tiles glTF to 3d Tiles Converter. Convert glTF model to Glb, b3dm or 3d tiles format. Usage λ python main.py --help Usage: main.py [OPTION

58 Dec 27, 2022
An API built to format given addresses using Python and Flask.

An API built to format given addresses using Python and Flask. About The API returns properly formatted data, i.e. removing duplicate fields, distingu

1 Feb 27, 2022
List of Land Cover datasets in the GEE Catalog

List of Land Cover datasets in the GEE Catalog A list of all the Land Cover (or discrete) datasets in Google Earth Engine. Values, Colors and Descript

David Montero Loaiza 5 Aug 24, 2022
Python renderer for OpenStreetMap with custom icons intended to display as many map features as possible

Map Machine project consists of Python OpenStreetMap renderer: SVG map generation, SVG and PNG tile generation, Röntgen icon set: unique CC-BY 4.0 map

Sergey Vartanov 0 Dec 18, 2022
geobeam - adds GIS capabilities to your Apache Beam and Dataflow pipelines.

geobeam adds GIS capabilities to your Apache Beam pipelines. What does geobeam do? geobeam enables you to ingest and analyze massive amounts of geospa

Google Cloud Platform 61 Nov 08, 2022
Location field and widget for Django. It supports Google Maps, OpenStreetMap and Mapbox

django-location-field Let users pick locations using a map widget and store its latitude and longitude. Stable version: django-location-field==2.1.0 D

Caio Ariede 481 Dec 29, 2022
Satellite imagery for dummies.

felicette Satellite imagery for dummies. What can you do with this tool? TL;DR: Generate JPEG earth imagery from coordinates/location name with public

Shivashis Padhi 1.8k Jan 03, 2023
Calculate & view the trajectory and live position of any earth-orbiting satellite

satellite-visualization A cross-platform application to calculate & view the trajectory and live position of any earth-orbiting satellite in 3D. This

Space Technology and Astronomy Cell - Open Source Society 3 Jan 08, 2022
Water Detect Algorithm

WaterDetect Synopsis WaterDetect is an end-to-end algorithm to generate open water cover mask, specially conceived for L2A Sentinel 2 imagery from MAJ

142 Dec 30, 2022
Software for Advanced Spatial Econometrics

GeoDaSpace Software for Advanced Spatial Econometrics GeoDaSpace current version 1.0 (32-bit) Development environment: Mac OSX 10.5.x (32-bit) wxPytho

GeoDa Center 38 Jan 03, 2023
A Python package for delineating nested surface depressions from digital elevation data.

Welcome to the lidar package lidar is Python package for delineating the nested hierarchy of surface depressions in digital elevation models (DEMs). I

Qiusheng Wu 166 Jan 03, 2023
Manage your XYZ Hub or HERE Data Hub spaces from Python.

XYZ Spaces for Python Manage your XYZ Hub or HERE Data Hub spaces and Interactive Map Layer from Python. FEATURED IN: Online Python Machine Learning C

HERE Technologies 30 Oct 18, 2022
PySAL: Python Spatial Analysis Library Meta-Package

Python Spatial Analysis Library PySAL, the Python spatial analysis library, is an open source cross-platform library for geospatial data science with

Python Spatial Analysis Library 1.1k Dec 18, 2022
Script that allows to download data with satellite's orbit height and create CSV with their change in time.

Satellite orbit height ◾ Requirements Python = 3.8 Packages listen in reuirements.txt (run pip install -r requirements.txt) Account on Space Track ◾

Alicja Musiał 2 Jan 17, 2022
Python library to decrypt Airtag reports, as well as a InfluxDB/Grafana self-hosted dashboard example

Openhaystack-python This python daemon will allow you to gather your Openhaystack-based airtag reports and display them on a Grafana dashboard. You ca

Bezmenov Denys 19 Jan 03, 2023
Python tools for geographic data

GeoPandas Python tools for geographic data Introduction GeoPandas is a project to add support for geographic data to pandas objects. It currently impl

GeoPandas 3.5k Jan 03, 2023
Google maps for Jupyter notebooks

gmaps gmaps is a plugin for including interactive Google maps in the IPython Notebook. Let's plot a heatmap of taxi pickups in San Francisco: import g

Pascal Bugnion 747 Dec 19, 2022