HyP3: ASF's On Demand Processing Platform¶
The Alaska Satellite Facility's Hybrid Pluggable Processing Pipeline, or HyP3 (pronounced "hype"), is a cloud-native processing platform developed to efficiently process Synthetic Aperture Radar (SAR) imagery.
Through HyP3, ASF offers a family of cloud-native, scalable product generation services that make its data holdings more accessible. HyP3 primarily runs compute-heavy workflows, which take tens of minutes to hours of compute time, to create products that are easier to use, GIS-ready, and customized to a user’s needs.
How it Works¶
HyP3 was designed to address many common issues for users of SAR data:
- Most SAR datasets require at least some processing to remove distortions before they are analysis-ready
- SAR processing is computing-intensive
- Software for SAR processing is complicated to use and/or prohibitively expensive
- Producing analysis-ready SAR data has a steep learning curve that acts as a barrier to entry
HyP3 solves these problems by providing an interface where users can request SAR processing on demand. These processing requests are picked up by automated systems, which handle the complexity of SAR processing on behalf of the user. Refer to our Architecture page to learn more about the structure of the HyP3 platform.
HyP3 doesn't require users to have a lot of knowledge of SAR processing before getting started; they only need to submit the input data and set a few optional parameters if desired. With HyP3, analysis-ready products are just a few clicks away.
The HyP3 Service Family¶
HyP3+ is now available!
For users who need more processing than the 10,000 credit limit allows, ASF now hosts HyP3+, a user-supported version of HyP3 that allows you to purchase as many credits as you need.
Explore HyP3 at no cost using HyP3 Basic, our NASA-supported On Demand processing service, or purchase credits for use with HyP3+, our user-supported service that lets you take your application global. Need help scaling your own, in-house workflows? Contact us!
|
HyP3 Basic Supported by NASA Earthdata |
HyP3+ User-supported Deployment |
|---|---|
|
hyp3-api.asf.alaska.edu Products expire after 14 days Round-robin processing queue Products distributed via CloudFront |
hyp3-plus.asf.alaska.edu Products expire after 30 days Smaller user-queue with higher throughput. Get your products faster! Products provided in a public AWS S3 Bucket |
| 10,000 Credits per month free* | 1 Credit = $0.05 |
*More available by request, as our budget allows. Send requests to ASF User Services: uso@asf.alaska.edu
Getting started¶
On Demand products processed by HyP3 can be requested quickly and easily, either by using the Vertex web interface or programmatically. These services are currently only available for Sentinel-1 datasets.
Different HyP3 job types consume different credit amounts. In HyP3 Basic, users are allotted 10,000 credits per month for free. Refer to the Credits page for more information.
If you do not have enough credits to generate all the products you need for your project, you can purchase additional credits in HyP3+.
Authentication¶
HyP3 users must authenticate with Earthdata Login (EDL) credentials before they can submit jobs to HyP3 (either HyP3 Basic or HyP3+) for processing or access information about the resulting On Demand products. Refer to our Authentication page for guidance.
Web Access¶
ASF's Data Search Vertex portal provides a rich interface to explore Sentinel-1 acquisitions and find images to submit for On Demand processing. It also provides tools for selecting pairs and stacks for InSAR analysis.
Programmatic Access¶
Requesting and downloading On Demand products can also be done programmatically:
HyP3 API REST Endpoints¶
The HyP3 API REST Endpoints are different for HyP3 Basic and HyP3+. Even though the same EDL username can be used to process On Demand products in HyP3 Basic and HyP3+ deployments, you will not be able to search for products across both APIs.
If you generate products for a single project using both the HyP3 Basic and HyP3+ deployments, you will need to use two separate searches to access all of your products, even if the project names are the same, when using Vertex or the HyP3 API.
When using the HyP3 SDK for Python, you can combine your results into one list using the following approach:
import hyp3_sdk as sdk
hyp3 = sdk.HyP3()
hyp3_plus = sdk.HyP3('https://hyp3-plus.asf.alaska.edu')
jobs = hyp3.find_jobs(...)
jobs += hyp3_plus.find_jobs(...)
jobs.download_files()
Public Visibility of Jobs¶
Warning
All jobs submitted to HyP3, whether via web access or programmatic access, are publicly visible. Anyone with access to HyP3 can potentially:
- View your jobs and associated metadata, including job name and user ID.
- Download any products generated by your jobs.
In particular, do not include any sensitive information in your job names.
What's New¶
Follow @ASFHyP3 on Twitter, or check our What's New page to keep up to date on all things HyP3!
Contact Us¶
Want to talk about HyP3? We would love to hear from you!
Found a bug? Want to request a feature? Open an issue
General questions? Suggestions? Or just want to talk to the team? Chat with us on Gitter
You can also reach us by email through ASF User Services: uso@asf.alaska.edu