From Planet Gallery: Mining in Peru 2016 © Planet

Order Up: Using and Building with Planet ’s new Orders v2 API

Tim Berners-Lee wrote

“It’s difficult to imagine the power that you’re going to have when so many different sorts of data are available.”

Tell me a story about why we need better tools and I will tell you while some good research starts with good data, almost all research must start with getting the data first. It doesn’t matter if you use remote sensing to solve puzzles of the earth system, or simply likes a good time-lapse or even stitches endless pieces of datasets together, “all data must start with gathering”. Imagine if satellite data could be structured, if it could be delivered to you, already clipped and reprojected and trimmed and made analysis ready for you. Imagine your order could speak to you, tell you in an email when it has completed as you work on something else. I think combining these and many more functionalities went into orders v2 API and as an early user, I got to play around with it and see it evolve over a period of time. As per me, their message was simple from gathering we went to

All data must start with ordering

This tutorial is in no way an overall synopsis of the ordersv2 API for that you can head here. But this was my attempt at using the API and also introducing the new command line tool I built around it to build toolchains. If you are confused about tools and toolchains, don’t worry I will introduce them later or you can head over here to read about it here

Setting the Stage

No, the Data API is not going away, it is still one of the best ways to search for and query data, and you can still use it to activate and download data. But if you are doing this for hundreds of datasets, if you would like to have datasets in bundles (a combination of all assets belonging to an item) you can now use the new Orders v2 API to run this.

For those who have used the Clips API from Planet and enjoyed it might also be aware that Clips API was deprecated in April 2018. So this includes your favorite tools to clip and ship as well along with some premium tools and features. In the next couple of paragraphs, I will highlight some of the tools and toolchains that I found useful for my own work but then again you can read the reference for me.

Building with the ordersv2 API

Now to work with this atleast in a programmatic interface you need to make yourself aware of an Application Programming Interfaces or API. In general you can get information about API request types here and read a detailed introduction to an API from restful here.

From Planet’s Orders v2 API Examples © Planet

At this point let me tell you where we are headed

  • You need to have python installed on your system so you can pull and use the command line interface I made and for debugging if you need it.
  • Read through API and get a general idea about what it is if you want to understand how the tool works.
  • That being said I have already built the tool so you don’t really need to build much yourself but you still need python and it should be accessible in Command prompt in your windows environment. Linux and Mac come prebuilt with python.
  • I have built the tool using Python 2.7.15 and I had no major issues upto Python 3.6.6.
  • Throughout the tutorial I will introduce some terms about the ordersv2 API and talk more about the Command Line Interface (CLI) I built to use it.

Setup and prerequsities

Things you will need to get setup includes a few things

  • Make sure that Python is installed and accessible in your command prompt or terminal (Windows users note we are using command prompt and not the powershell if in doubt use windows key+r/Run, followed by cmd and hit return)
  • Planet’s own CLI which allows you to interact with the data API
  • This is important Initialize Planet CLI, type planet init in your command prompt or terminal and enter your email and password to initialize.
Initialize the planet client (Windows Command Prompt or use terminal)
  • Shapely is notoriously difficult as a library to install on windows machines so follow the steps mentioned from Shapely’s PyPI package page
  • You can download and install from the Unofficial Wheel files from here download depending on the python version you have. You will get a wheel file or a file ending with .whl. You can now simply browse to the folder or migrate to it in your command prompt , for example in my case I was running Python 2.7.15 and win32 version so the command was
    pip install Shapely-1.6.4.post1-cp27-cp27m-win32.whl
  • Or you can use anaconda to install. Again, both of these options are mentioned on Shapely’s Official PyPI page

Product Bundles

Product bundles are a combination of items and asset types together, designed to make the experience of downloading, images and associates files seamless. To give you an example when you download a PSScene4Band analytic image, you often also download the metadata file which is an xml file and the associated unusable data mask (udm) file and the product bundle configures this for you and allow you to order them all together.

For example for Item type PSScene4Band and asset type analytic the product bundle includes
"PSScene4Band": ["analytic","analytic_xml","udm"]
For PSScene4Band and asset type analytic_sr (surface reflectance) the product bundle includes
"PSScene4Band": ["analytic_sr","udm","analytic_xml"]

To find out about all product bundles you can head over and find them here

Toolbox and Toolchains

The toolbox comes with a few tools that everyone has access to, apart from being able to simply order the datasets, you can choose to make it analysis ready or preprocess it to some degree. One of the most sought after tool by many and one that is included by default for all users is the Clip tool which would effectively replace the deprecated Clips API and performs this on all your images before you download them saving you on your quota. There is an entire toolbox that is included with Ordersv2 API, apart from the default tools that are beyond the Education and Research License users. Some of the additional tools include

  • Top of Atmosphere reflectance( TOAR) from the radiance imagery
  • Composite as the name suggested is to composite multiple images into one
  • Reproject is another useful one to do just the same reproject before downloading them and one of my favorite and powerful ones is
  • Band Math where you can perform simple band math operations on all images in the stack of images you want to download. So you may be able to download indices such as NDVI directly instead of raw imagery.

The choice of tools are important but more important is the order in which these tools should be tied together. Read the instructions here about creating a valid tool chain.

porder: Simple Command Line Interface(CLI) for Ordersv2 API

The idea of building something around ordersv2 API came with the potential to explore, tie and work with complex orders using a simple command line interface (CLI). You can explore the simple setup using the payloads in the readme and go through the API and place post calls and get requests using something like Postman. This tool was created to tie multiple operations together within the framework of what tools might be available to Education and Research users (at the time of writing this story you should have access to clip,zip and email notification) as well as users with additional tools based on their license type. You can read about and find the tool on GitHub here.

You can install using two methods

pip install porder

on Ubuntu I found it helps to specify the pip type and use sudo

sudo pip2 install porder or sudo pip3 install porder

or you can also try the following (It might require sudo privileges for Linux users)

git clone https://github.com/samapriya/porder.git
cd porder
python setup.py install

Once the setup is completed you should be able to access porder in your terminal by typing porder -h

porder Command Line Interface in Windows command prompt

Get your quota straight

To make sure you have enough quota left before placing an order, use this to estimate the area left. The API will run a check against your quota as well before order is complete. The tool will print the following information by using porder quota

Allocation Name: 
Allocation active from:
Quota Enabled:
Total Quota in SqKm:
Total Quota used:
Remaining Quota in SqKm:

Next up: Get your ID list or Difference ID List

The next step is to interact with Data API and get an idlist to be submitted in the order, there are two tools that I have included here. The first one is idlist which as the name suggests is a tool to generate the IDlist for a given geometry in combination with date and cloud filters. This is the first step and constitutes the core of the order you want to place.

porder ID list get item ids

A simple setup would be

porder idlist --input "C:\johndoe\geometry.geojson" --start "2017-01-01" --end "2018-12-31" --item "PSScene4Band" --asset "analytic_sr" --number 800 --outfile "C:\johndoe\orderlist.csv"

But maybe you have already downloaded some images to your computer or a local system, and though the downloader can skip re-downloading these images, it is better to skip these while placing the order all together. The difflist tool allows you to check with files you have already downloaded and creates a difference ID list with only those ID(s) that you do not already have. This tool has been designed to work with complete images only and not clips for now.

porder Diff ID list get item ids you have not yet downloaded

A simple setup would be

porder difflist --folder "F:\johndoe\ps4b_xml" --typ "metadata" --input "F:\johndoe\grid.geojson" --item "PSScene4Band" --asset "analytic_sr" --start "2018-06-01" --end "2018-08-01" --cmin 0 --cmax 0.9 --outfile "F:\johndoe\diff.txt"

Split the List: Use if Needed

Sometimes it is possible that your list is too dry and there are limits on the maximum number of items IDs that could be included in a list. I am hoping these limits will be included in the documentation somewhere, for now you can choose to create smaller orders or split the IDlist into equal parts.

porder idsplit used to split your idlist or difflist

A simple setup would be

porder idsplit --idlist "C:\johndone\orderlist.csv" --lines "100" --local "C:\johndoe\split"

Ordering Now

This tool is designed to take in multiple arguments and chain together tools. The --op argument allows you to take operations, delivery and notifications in a sequence for example --op clip email performs clip and sends you an email notification once the order has completed, failed or had any any change of status. For now clip, Top of Atmospheric Reflectance (toar), composite, zip and email tools are included from the tool list. Which tool is available to you, depends on your license type

porder order tool chain your tools and order up

Once you place the order the order url is generated and copied to your clipboard so you can paste it in your browser if you would like. A simple setup to clip and ship with email notification for example would be

porder order --name "testorder" --idlist "idlistfile.txt" --item "PSScene4Band" --asset "analytic" --boundary "clip_boundary.geojon" --op clip email 

The link would look something like this(not a real link) and use your API key as the username and leave the password blank
https://api.planet.com/compute/ops/orders/v2/22015422-8608-48e7-9456-b84a439da91e

Save it since you need it to download the files using the download or multipart download tools. Incase you placed your order and lost your download url, you can search based on your download name. All your orders are listed at the API endpoint.

https://api.planet.com/compute/ops/orders/v2

Download and Multipart Download

These two tools allows you to download the files to your local drive, the first tool which is download only does this sequentially and uses chunks to download your order, the multipart tool uses a library to split each file into parts for faster download.

Download using the order url created earlier

The multipart download setup is the same as download setup

Multipart download tool uses a different download library

There you have it, you just placed and order, and downloaded it. Depending on the size of your order the time required to download can vary, but you can also write your own version of a downloader.

Next iterations of the tool, I am hoping to include a downloader that employs multiprocessing to download quicker and also create an overlap tool that checks the minimum percentage overlap an image has with your geometry before creating the orderlist.

Next Iteration to calculate percentage overlap of each image with your geometry

Please cite the tool, or star it on GitHub if you find it useful and use it as it allows me to understand usage and develop better tools.

Samapriya Roy. (2018, November 26). samapriya/porder: porder: Simple CLI for Planet ordersV2 API (Version 0.0.1). Zenodo. http://doi.org/10.5281/zenodo.1534692

If you find this guide useful please click the clap 👏 button a couple of times below to show your support (You can click more than once :D ) ! and if you have used it to recreate something or to create better workflow, let me know.

--

--

--

Remote sensing applications, large scale data processing and management, API applications along with network analysis and geostatistical methods

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Winners of the MLOps for Kubeflow Badge Raffle for KubeCon + CloudNativeCon Europe 2022

Competing Frameworks: Alibaba Puts SDK Flutter to the Test

Connecting to a git repository using libgit2sharp

Throw it over the wall: A lego game which can shift waterfall mindsets

PHCR Airdrop Winners

Easy Validations In Android’s Modular App Architectures

Real-Time Bitcoin Price Streaming using Oracle Cloud Services

Best Note-Taking Apps of 2019

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Samapriya Roy

Samapriya Roy

Remote sensing applications, large scale data processing and management, API applications along with network analysis and geostatistical methods

More from Medium

GeoServer + Postgis environment build

Branch Network Visualization with geocoding

PostgreSQL and PostGIS installation guide…

Getting Started with Python Language — 3(WomenInTech)