Updated Data Pipelines: pip install ppipe

Samapriya Roy
3 min readJul 11, 2018

--

I am writing this as a snippet to comment on the new changes and updates to the Planet Google Earth Engine Pipeline (or ppipe) as the tool is named on GitHub and since then has now been released on PyPI. Since the last time I wrote that article a lot of time has gone by and improvements have occurred at both at the way in Google Earth Engine (GEE) allows us to ingest data as well as for interacting with the Planet and Earth Engine’s own CLI(s). I have updated the GitHub Readme to include all these changes. First things first

  1. Easier Install- You can now install tool directly from PyPI directly so go ahead and try pip install ppipe . The setup file already takes care of the requirements so as long as you have authenticated Planet’s CLI and earthengine-cli this tool will integrate directly. This tool has been tested on both windows and linux environments and including Google Colab
  2. Now do a quiet authentication for environments such as jupyter lab or google colab environments
    ppipe planetkey --type “quiet" --key "you api key"
    However don’t try this on a shared environment since it is possible to simply do a bash history and retrieve keys. For shared machines simply use
    ppipe planetkey which uses the planet init method to automatically initialize and save your API key.
  3. Now download from saved searches- It is possible to now download from saved searches from your Planet Explorer searches. This allows you to use the Planet Explorer’s GUI to select dates and filters and then use this tool and the saved search name to download your searches. This method skips the need to setup and activate assets in separate steps
ppipe savedsearch --name "name of your saved search" --asset "analytic/analytic_dn/analytic_sr,analytic_xml" --path "where do you want to download this" --limit "optional if you want to download limited number of images"
Saved Search Downloads

4. Improvements to Earth Engine Side of things

  • Earth Engine Quota- This is a minor tool but prints out the amount of total space and space used in your earth engine quota. simple type
    ppipe quota
  • Upload now uses manifests- Google introduced the idea of using a manifest to better assign metadata and properties to images being upload in its CLI and we utilized this to integrated a manifest for every single Planet item and asset type combinations.
Updated Upload tool for ppipe now inclues manifest

So you can now upload for example PlanetScope 4 Band analytic using

ppipe upload --source "folder with images" --dest "users/johndoe/ps4band" -m "metadata csv file you created" -mf "Manifest type for PSScene4Band analytic use PS4B" -u "johndoe@gmail.com" --nodata "0"

I am now at release 0.3.7 for PyPI and as need arises and as I find time I will keep updating this tool based on input from users. As always you can read about updates from Planet and Google Earth Engine at their developer pages. If you feel this tool and updates have been helpful, show your support with claps and guess what you can clap more than once 👏 😀

--

--

Samapriya Roy
Samapriya Roy

Written by Samapriya Roy

Remote sensing applications, large scale data processing and management, API applications along with network analysis and geostatistical methods

Responses (2)