Finnish Data Hub System User Guide
Overview

The Finnish Data Hub provides complete, free and open access to Sentinel-1 and Sentinel-2 user products.

Sentinel-1 Data Offer

The Sentinel-1 data offer for the Finnish Data Hub consists of:

The Sentinel-1 acquisitions zones with the related mode and polarisation are defined on a cyclic (12 days) basis in the observation scenarios.

The Sentinel-1 ground segment production baseline is described in the production scenario.

Sentinel-2 Data Offer
The Sentinel-2 data offer for the Finnish Data Hub will consist of Level-1C user products.
The Data Hub Archive
The Finnish Data Hub maintains an archive of all the products for download via HTTP.
Free and Open data access
Anyone can register online via registration form. The self-registration process is automatic and immediate. Registration grants access rights for searching and downloading Sentinels products. Sentinels products are available at no cost for anybody. The data available through the Data Hub is governed by the Terms and Conditions of the use and distribution of Sentinel data, which the User is deemed to have accepted by using the Sentinel data.
Full-text Search and Saved Search Queries
Search query on the products stored on the archive and filtering of results via a full-text search bar. Predefined search filters for the different acquisition modes, product types, product levels and geographical areas. "Make your own" search via specific semantics (boolean operators and wildcards). Text search query can be saved and used for activating the e-mail notification service and for batch scripting.
Products Online Inspection
Online inspection of the searched products by browsing and pre-viewing the product metadata and measurements without downloading it. A preview panel displays information on the product contents and structure.
Products Download and quota
Sentinel products are provided for download via HTTP in the .ZIP archive file format. Click and download, shopping cart, batch download. A maximum of 2 concurrent downloads per user is allowed in order to ensure a download capacity for all users.
Batch scripting
The Data Hub exposes the Open Data Protocol ( OData) interface for accessing the EO data stored on the archive. This protocol is based on top of the well-supported HTTPS/ REST transfer protocol that can be handled by a large set of client tools as simple as common Web browsers, download-managers or computer programs such as cURL or wget. The Odata protocol provides easy access to the Data Hub and can be used for building URI for performing search queries and product downloads offering to the users the capability to remotely run scripts in batch mode.


© ESA
How To Register
The email-based registration can be performed by the user following these steps:
  1. Fill out the form from the Registration page
  2. The Finnish Data Hub Support Team will send an e-mail to the provided e-mail address containing a link for registration confirmation.
  3. Following the confirmation by the user, the new account is automatically active with "Search" and "Download" rights.
Login

Registered users can login through the authentication module in the Finnish Data Hub home page.

If credentials are correctly inserted a user logo appears:

If credentials are not correctly inserted (i.e. wrong username or password) an error message appears:

For resetting the password send e-mail to the the Finnish Data Hub Support Team.

© ESA
Register a New Account

Sentinel data access is free and open to all.

On completion of the registration form below you will receive an e-mail with a link to validate your e-mail address. Following this you can start to download the data.


Please, explain what other Domain do you belong to:


Please, explain what other Usage do you belong to:


If you have a problem with the registration, please, contact with our Support Team finhub-support@nsdc.fmi.fi


Graphical User Interface
Search Panel

The search panel provides a "full text search bar", "advanced search options" and a "map tool".

Map Tool

The map tool provides a geographical search feature. To define a region of interest click on the icon to set "draw region of interest" modality and then click and drag on the map for drawing a selection box. By clicking on the "Search" button the Data Hub will search for any product whose footprint intersects the selected region.

The map tool can be combined with the full text search and the advanced search to further restrict the results.

Search results

The Search results list provides all the products matching the submitted search query. Each result consists of:

  • 64x64 thumbnail when available
  • product name
  • product URI
  • a Date corresponding to the Sensing Start date
  • the instrument name
  • the instrument imaging mode
  • the satellite name
  • the size of the product

The footprints of the products matching the search query are displayed on the map. Each footprint gets highlighted when hovering on the corresponding product in the search results list. NOTE: the map tool shows only the footprints of the products in the current search list page. To widen results you can change the page size (25, 50 or 100).

Profile Panel

The profile panel contains a "My Information" subsection, with all the account details and the possibility to change the password.

The "My saved searches" subsection allows to manage the searches saved by the user.

Cart Panel

The cart panel shows the products added to the cart. It's possible to download the single product or all the products in the cart as a bulk download.


back to top © ESA
Full Text Search

Search queries on the rolling archive can be performed by entering a text query in the full-text search bar.

Simple search query

The table below shows some simple text queries that can be typed in the full-text search bar:

Text QueryDescription
blank field or *returns all products in the archive
L0returns all Level-0 products
L1returns all Level-1 products
SLCreturns all SLC products
GRDreturns all GRD products
IWreturns all products in IW mode
EWreturns all products in EW mode
SMreturns all products in SM mode
DESCENDINGreturns all products of the descending passes
ASCENDINGreturns all products of the ascending passes
HHreturns all products containing a measurement dataset in HH polarisation
VVreturns all products containing a measurement dataset in VV polarisation
HVreturns all products containing a measurement dataset in HV polarisation
VHreturns all products containing a measurement dataset in VH polarisation
HH HV or HH+HVreturns all products in dual polarisation HH/HV
VV VH or VV+VHreturns all products in dual polarisation VV/VH
Wildcards and Operators

Wildcards and Operators can be used for restricting search queries. Wildcards, in particular, are useful when performing a query on the product filename. Operators for combining different search criteria. Admitted wildcards and operators are shown in the tables below

WildcardDescription
*any sequence of zero or more characters
?any one character


OperatorDescription
ANDNarrow search and retrieve records containing all of the words it separates.
ORBroaden search and retrieve records containing any of the words it separates.
NOTNarrow search and retrieve records that do not contain the term following it.
( )Group words or phrases when combining Boolean phrases and to show the order in which relationships should be considered.

Examples of queries on products filename using wildcards:

ExampleDescription
*_S1_*returns Stripmap products with elevation beam 1
*_1SD?_*returns Level-1 products in dual polarisation
*_?SD?_*returns Level-0 and Level-1 products in dual polarisation
*_0SS?_*returns Level-0 products in single polarisation

Examples of queries with Operators:

ExampleDescription
IW AND L0returns Level-0 products in IW mode
EW OR SMreturns all products in EW mode and all products in SM mode
L1 NOT IWreturns Level-1 products in EW mode and in SM mode
L0 NOT (SLC AND VV)returns all Level-0 GRD products, all Level-0 SLC products in dual polarization and all Level-0 SLC products in single polarization HH
Search Keywords

It is also possible to make full-text queries using search keywords.
The syntax is the following:
<keyword>:<values>
Depending on the keyword, the value(s) can be specified as a single value or range of values.

Search keywords can be combined with each other using Operators.
The following table contains the list of the keywords and their corresponding usage.

Search KeywordSyntax and Examples

platformname:
Search based on the Satellite Platform name regardless of the serial identifier (e.g. A, B, C ...)

platformname:<platform name>
Examples:
platformname:Sentinel-1
platformname:Sentinel-2

beginposition:
A time interval search based on the Sensing Start Time of the products.

endposition:
A time interval search based on the Sensing Stop Time of the products.

ingestiondate:
A time interval search based on the time of publication of the product on the Data Hub.

These three keywords are used with a time range expressed with the following syntax:

<keyword>:[<timestamp> TO <timestamp>]

The <timestamp< value can be expressed in one of the the following formats:

  • (ISO8601) yyyy-MM-ddThh:mm:ss.SSSZ
  • NOW
  • NOW-MINUTE(S)
  • NOW-HOUR(S)
  • NOW-DAY(S)
  • NOW-MONTH(S)

being n = 1,2,...,10,...,100,...

Examples:

beginposition:[2014-01-01T00:00:00.000Z TO NOW]
beginposition:[2014-01-01T00:00:00.000Z TO 2014-02-01T00:00:00.000Z]
beginposition:[NOW-2DAYS TO NOW]

endposition:[2014-01-01T00:00:00.000Z TO NOW]
endposition:[2014-01-01T00:00:00.000Z TO 2014-02-01T00:00:00.000Z]
endposition:[NOW-1HOUR TO NOW]

ingestiondate:[2014-01-01T00:00:00.000Z TO NOW]
ingestiondate:[2014-01-01T00:00:00.000Z TO 2014-02-01T00:00:00.000Z]
ingestiondate:[NOW-30MINUTES TO NOW]

collection:
The name of a predefined collection of products.

collection:<collection name>
(the list of the available collection names will be advertised when available)

Example:
collection:mycollectionname

filename:
Search based on the product filename

filename:<filename>

Possible <filename> values are:

  • The full product file name expressed according to the product naming convention:
    MMM_BB_TTR_LFPP_YYYMMDDTHHMMSS_YYYYMMDDTHHMMSS_OOOOO_DDDDD_CCCC
  • Part of the filename using wildcards.

Examples:

filename:S1A_EW*
filename:S1A_EW_GRDH_1SDH_20141003T003840_20141003T003920_002658_002F54_4DD1
filename:*1SD?_20141003T003840*

footprint:
Geographical search of the products whose footprint intersects or is included in a specific geographic type.

Syntax is the following:

footprint:"intersects(<geographic type>)"

The <geographic type> value can be expressed as a polygon or as a point according to the syntax described below.

POLYGON:
<geographic type>=POLYGON((P1Lon P1Lat, P2Lon P2Lat, , PnLon PnLat, P1Lon P1Lat))

where P1Lon and P1Lat are the Longitude and Latitude coordinates of the first point of the polygon in decimal degrees (DDD) format (e.g. 2.17403, 41.40338) and so on. The coordinates of the last point of the polygon must coincide with the coordinates of the first point of the polygon. The polygon describing the geographical area can have a maximum of 200 points that must be within an area described by 10 degrees of latitude and 10 degrees of longitude.

Example:

The polygon of the example is a bounding box around the Mediterranean Sea:
footprint:"Intersects(POLYGON((-4.53 29.85, 26.75 29.85, 26.75 46.80,-4.53 46.80,-4.53 29.85)))"

POINT:
<geographic type>= Lat, Lon

where the Latitude (Lat) and Longitude (Lon) values are expressed in decimal degrees (DDD) format (e.g. 41.40338, 2.17403 ).

Examples:
footprint:"intersects(Lat, Lon)"

  • Rome city centre - footprint:"intersects(41.9000, 12.5000)"
  • Etna Volcano - footprint:"intersects(37.7550, 14.9950)"
  • Bárðarbunga Volcano - footprint:"intersects(64.6300, -17.5300)"
  • Istanbul city centre - footprint:"intersects(41.0136, 28.9550)"
  • Paris city centre - footprint:"intersects(48.8567, 2.3508)"
  • Mexico City city centre - footprint:"intersects(19.4333, -99.1333)"
  • Cagliari city centre - footprint:"intersects(39.2500, 9.0500)"
  • Villa San Faustino city centre - footprint:"intersects(42.7340, 12.5324)"
  • London city centre: footprint:"intersects(51.5072, 0.1275)"
  • Berlin city centre: footprint:"intersects(52.5167, 13.3833)"

orbitnumber:
Absolute orbit number of the oldest line within the image data (the start of the product).

lastorbitnumber: Absolute orbit number of the most recent line within the image data (the end of the product)

These keywords can be used with a single value or with a range of values.

The syntax is the following:

  • orbitnumber:<orbitnumber>
  • lastorbitnumber:<lastorbitnumber>
  • orbitnumber:[<orbitnumber> TO <orbitnumber>]
  • lastorbitnumber:[<lastorbitnumber> TO <lastorbitnumber>]

Possible values for and go from 000001 to 999999.

Examples:

orbitnumber:000020 (or orbitnumber:20)
lastorbitnumber:000510 (or lastorbitnumber:510)
orbitnumber:[001020 TO 001021] (or orbitnumber:[1020 TO 1021])

orbitdirection:
Direction of the orbit (ascending, descending) for the oldest image data in the product (the start of the product).

Possible values are:
Ascending, Descending

Example:
orbitdirection:Ascending

polarisationmode:
valid polarisations for the S1 SAR instrument

Possible values are:
HH, VV, HV, VH, HH HV, VV VH

Examples:
polarisationmode:HH
polarisationmode:VV VH

producttype:
output product type

Possible values are:

SLC, GRD, OCN, S2MSI1C

relativeorbitnumber:
Relative orbit number of the oldest line within the image data (the start of the product).

lastrelativeorbitnumber:
Relative orbit number of the most recent line within the image data (the end of the product)

These keywords can be used with a single value or with a range of values.

The syntax is the following:

  • relativeorbitnumber:<relativeorbitnumber>
  • lastrelativeorbitnumber:<lastrelativeorbitnumber>
  • relativeorbitnumber:[<relativeorbitnumber> TO <relativeorbitnumber>]
  • lastrelativeorbitnumber:[<lastrelativeorbitnumber> TO <lastrelativeorbitnumber>]

Possible values for <relativeorbitnumber> and <lastrelativeorbitnumber> go from 1 to 175.

Examples:
relativeorbitnumber:5
relativeorbitnumber:10
relativeorbitnumber:[25 TO 100]

sensoroperationalmode:
The SAR instrument imaging modes

Possible values are:
SM, IW, EW

Example:
sensoroperationamode:SM

swathidentifier:
Search all valid swath identifiers for the Sentinel-1 SAR instrument. The S1-S6 swaths apply to SM products, the IW and IW1-3 swaths apply to IW products (IW is used for detected IW products where the 3 swaths are merged into one image), the EW and EW1-5 swaths apply to EW products (EW is used for detected EW products where the 5 swaths are merged into one image).

Possible values are:
S1, S2, S3, S4, S5, S6, IW, IW1, IW2, IW3, EW, EW1, EW2, EW3, EW4, EW5

Example:
swathidentifier:S1


back to top © ESA
Advanced Search

The advanced search panel is shown when clicking on "advanced search".
It consists of six search fields. Each search field allows to perform specific queries on the associated parameter. Each search field can be used singularly or in combination with: the others search fields, the full-text search bar and the geographic map tool.

Search Field Name Usage
Sensing Period

This search field is composed of two date entries.
Dates have to be inserted for both entries. "Pick up date" calendars allow date selection.
The query returns all the products whose sensing dates and times are included in the defined period.
In particular it returns all the products that respond to both of the following criteria:

  • Sensing start time equal or greater then 00:00:00 (hh:mm:ss) of the first selected date
  • Sensing stop time equal or less then 23:59:59 (hh:mm:ss) of the second selected date

The Sensing Time corresponds to the time of the satellite on-board acquisition and it is stamped for
each line of the acquired image scene.
The sensing start and stop times of a product correspond to the time of the satellite on-board acquisition of respectively the first and last line of the image in the product.

Click on "Clear date" for removing the search entry. Click on "Today" for selecting current date.

Ingestion Period This search field is composed of two date entries as for "Sensing Period".

The query returns all the products whose publication dates and times on the Data Hub are included
in the defined period.
In particular it returns all the products that respond to both of the following criteria:

  • Publication time equal or greater then 00:00:00 (hh:mm:ss) of the first selected date
  • Publication time equal or less then 23:59:59 (hh:mm:ss) of the second selected date
Please note that the ingestion date does not correpond to the generation date of the product processed at the ground segment. The ingestion date is the date of publication of the product on the Data Hub rolling archive.
Product Type

Accepted entires are:

  • SLC
  • GRD
  • OCN
  • S2MSI1C
Sensor Mode

Accepted entries are:

  • SM
  • IW
  • EW
Polarisation

Accepted entries are:

HH, VV, HV, VH, HH HV, VV VH

Swath

Accepted entries are:

S1, S2, S3, S4, S5, S6, IW, IW1, IW2, IW3, EW, EW1, EW2, EW3, EW4, EW5


back to top © ESA
APIs And Batch Scripting

The Data Hub exposes two dedicated Application Program Interfaces (API) for browsing and accessing the EO data stored in the rolling archive. The APIs are:

The OData interface is a data access protocol built on core protocols like HTTP and commonly accepted methodologies like REST that can be handled by a large set of client tools as simple as common web browsers, download-managers or computer programs such as cURL or Wget.

OpenSearch is a set of technologies that allow publishing of search results in a standard and accessible format. OpenSearch is RESTful technology and complementary to the OData. In fact, OpenSearch can be used to complementary serve as the query aspect of OData, which provides a way to access identified or located results and download them.

Contents:

Open Data Protocol (OData)

The Open Data Protocol (OData) enables the creation of REST-based data services, which allow resources, identified using Uniform Resource Identifiers (URIs) and defined in a data model, to be published and consumed by Web clients using simple HTTP messages.

The OData protocol provides easy access to the Data Hub and can be used for building URI for performing search queries and product downloads offering to the users the capability to remotely run scripts in batch mode.

URI Components

A URI used by an OData service has up to three significant parts: the Service Root URI, the Resource Path and the Query Options.

  • the Service Root URI identifies the root of the OData service
  • the Resource Path identifies the resource to be interacted with. The resource path enables any aspect of the data model (Data Hub Products, Data Hub Collections, etc.) exposed by the OData service
  • the system Query Options part refines the results

Example of and OData URI exposed by the Data Hub Service broken down into its component parts:

OData Service Root URI for the Data Hub:

  • https://scihub.copernicus.eu/dhus/odata/v1

OData Service Root URI for the API Hub:

  • https://scihub.copernicus.eu/apihub/odata/v1/

Data Hub Resource Paths:

  • /Products
  • /Collections

Query Options admitted by the Data Hub service:
  • $format Specifies the HTTP response format of the record e.g. XML or JSON
  • $filter Specifies an expression or function that must evaluate to true for a record to be returned in the collection
  • $orderby Determines what values are used to order a collection of records
  • $select Specifies a subset of properties to return
  • $skip Sets the number of records to skip before it retrieves records in a collection
  • $top Determines the maximum number of records to return

Examples of OData URIs for the Data Hub:

Responses

When users query the Data Hub via the URI, the request is sent to the Data Hub OData service. This service provides the responses. The description of the responses is provided here below focusing on the response body and not covering the standard HTTP response envelope.

The default response format is Atom [RFC4287], an XML-based document format that describes Collections of related information known as 'feeds'. The responses containing a single Product entity differ from those containing a collection of Product entities. Generally speaking, the single Product entity is returned as a bare Atom entry element, while for a collection the same Atom entries denoting the Product entities are wrapped into an Atom feed element.

The response format can however be controlled from the requests through the $format query option already introduced above. The formats currently implemented by Data Hub service are Atom (or XML) and JSON which is a non-standard output of OData but is an experimental behaviour useful for EO context.

The following sub-sections describe the response types for Atom/XML and JSON output formats. The very last sub-section deals with the specific responses returned in case of error.

XML Responses

The Atom/XML format is the default response format, though it can be forced by setting the $format query option to 'atom' or 'xml' indifferently as in the following examples:

Service Metadata Document

In order to help clients discover the OData services, the Data Hub OData Service Metadata Document exposes the Entity Data Model of the service including among others, the Entities and Properties that can be queried. This document can be queried as with the following URL:

The following example is extracted from the complete Service Metadata Document exposed by a Data Hub OData service. Some sections or XML Namespace declarations have been removed for brevity:

Using OData to discover products in the Data Hub archive
Querying the content of the products archive: usage of the /Products resource path

The following OData URI addresses the resource 'Products' exposed by the Data Hub Service which includes the records of all the products stored in the Data Hub archive. Each product record includes the Universally Unique Identifier (UUID), the product name, the link for download, the links for navigating through their nodes and attributes and some properties.
The syntax is:

Example:

By default it provides a list of 50 records showing the properties of the latest 50 products ingested by the Data Hub.

Examples of properties are:

IngestionDate
date on which the Product was ingested by the contacted Data Hub instance. The date referential is UTC.
ContentDate (Start and End)
time period associated to the Product payload. This date regards the period of the subject of the content and not the content itself. For example, for an EO product, the ContentDate deals with the observation date and not to the downlink or processing dates.
ContentGeometry
footprint polygon coordinates

The universally unique identifier (UUID) is a standard identifier used in software construction to uniquely identify information without significant central coordination. A UUID is a 16-octet (128-bit) number.In its canonical form, a UUID is represented by 32 lowercase hexadecimal digits, displayed in five groups separated by hyphens, in the form 8-4-4-4-12 for a total of 36 characters (32 alphanumeric characters and four hyphens).

Querying a single Product in the archive

The responses containing a single Product entity differ from those containing a collection of Product entities. Generally speaking, the single Product entity is returned as a bare Atom entry element, while for a collection the same Atom entries denoting the Product entities are wrapped into an Atom feed element.

Syntax:

The following example is an actual response of a Data Hub OData service that returned a single Product entity identified by the UUID: 18f7993d-eae1-4f7f-9d81-d7cf19c18378.
Again, this snippet has been reformatted for readability purpose but nothing has been removed from the actual response:
https://scihub.copernicus.eu/dhus/odata/v1/Products('18f7993d-eae1-4f7f-9d81-d7cf19c18378')
Response:

Querying the products (paging)

The URI to be used for paging the list of products in the archive shall follow the syntax below:

where ?$skip='N' is the number of records to skip before it retrieves records in a collection and is the maximum number of records to return.

Example:

$top accepts as maximum value 100. For higher values the maximum number of records returned will still be 100.

Using OData to filter the products in the archive

It is possible to use the OData properties to perform search query to filter the products.

Filtering the Products on time-based criteria

For the metadata that are expressed in the UTC format such as

FilterDescription
CreationDateFor future implementation
IngestionDateTime of the products archiving in the Data Hub
EvictionDateFor future implementation
ContentDate/Start and ContentDate/EndTBW

the filter can be performed on the day and/or month and/or year.
The symbols are not allowed in the OData protocol but they are substituted with the following syntax:

SymbolRegular ExpressionMeaning
<ltlower than
leLower or equal than
>gtGreater than
geGreater or equal than
=eqEqual

Here below we provide some examples.

Filtering the products by Ingestion Date

To filter the products by Ingestion Date, the following syntax may apply:

Examples:

Filtering the Products using the file name

The products file name can be used for filtering the products. It shall be noticed that this query criteria is not based on the Medatata indexed from the products content but the criteria is search products matching a predefined string on the file name.

In this case, the Entity Data Model (EDM) string are used for metadata like Name which indicates the name of the product. Every metadata containing a string can be filtered using the substringof filter.

Examples:

Filtering the Products by Sensing Time (file name)

To filter the products by the Sensing Time (start and stop) it can be used the S1 product filename, using the filter substringof. Next section will describe how to make query based on indexed metadata.

The syntax is:

Example:

Querying Product Attributes and Metadata Contents

OData can be used for querying the values of products metadata content.

Get product UUID from product name

To retrieve the UUID of a predefined product, the syntax is:

Example:

Download

OData URL can be used for download full products as well as partial products by using the combination of the Nodes and Attributes indexed for the products and adding at the end of the URI.

Download full product from its UUID

To download a full products the syntax is:

Example:

Discover Product Nodes

Products from all processing levels (Level-0, Level-1 and Level-2) are disseminated in SENTINEL-SAFE format. The data delivered is packaged as a file structure containing a manifest file in XML format listing general product metadata and subfolders for measurement data, annotations, previews and support files. The SENTINEL-SAFE format wraps a folder containing image data in a binary data format and product metadata in XML. This flexibility allows the exploration of parts of the products without download the complete product.
In this section we will provide examples to how explore products nodes.
The following URI will return the XML file including the list of the first level nodes:

Generally the first node is the one whose name is [Product_Name.SAFE] and the URI to visualize the XML schema of this node is the following:

Example of the nodes of a level-1 Sentinel Product are:

  • Manifest.safe
  • Annotation
  • Measurement
  • Preview
  • Support
Download manifest file from the UUID

To download manifest file from the UUID the syntax is:

Example:

Download of Sentinel-1 quick-look file knowing the product UUID

To download the quick-look file from the UUID the syntax is:

Example:

Verifying Download integrity using the MD5 checksum

This can help to reveal if the download was incomplete.
Each product published on the Data Hub provides an MD5 checksum of the downloadable ZIP file. The Message Digit of the file can be discovered using the following OData query:

where UUID is the Universally Unique Identifier of the product. Example:

provides the following MD5:

  • 99F8BCE665CEF2587258F581873DFA53

The integrity of the downloaded product can be checked by comparing the MD5 value of the product at the source with the MD5 of the product located on the users's system. If they are identical the download has been succesfull.
Different Operating Systems require different methods for verifying the checksum of the downloaded products. Find below a list of the most common ones:

Linux

Open a terminal and type:

  • md5sum path/filename
OS X

Open a terminal and type:

  • md5 path/filename
Windows

The File Checksum Integrity Verifier (FCIV) utility does not come pre-installed on windows and needs to be downloaded. Instructions can be found here.
Open a command prompt and type:

  • FCIV -md5 -sha1 path ilename
OpenSSL

This method can be done on any system with OpenSSL installed (i.e. Windows, Linux and MAC). Open a command prompt or terminal and type:

  • openssl md5 path/filename
Open Search

OpenSearch (Solr) is a set of technologies that allow publishing of search results in a standard and accessible format. OpenSearch is RESTful technology and complementary to the OData. In fact, OpenSearch can be used to complementary serve as the query aspect of OData, which provides a way to access identified or located results and download them. The Data Hub implementation uses the Apache Solr search engine.

URI components

where:

  • <dhus_hostname>:<port>/<path> is the Service Root
  • search?q=<query> is the Query
Using Open Search to discover products in the Data Hub archive
Discover the list of the products stored in the archive

The syntax is:

Example:

To display just the first 10 results of the previous query, the can be completed with &rows=10&start=0
So the complete search will be :
<dhus_hostname>:<port>/<path>/search?q=*&rows= <N> &start= <N>

Example:

Discover the products over a predefined Area Of Interest (AOI): Geographical Search

It is possible to search products on the basis of a geographical area of interest, e.g. get the list of products over a geographic area delimitated by the polygon having vertices:

The <geographic type> value can be expressed as a polygon or as a point according to the syntax described below.

POLYGON:

where P1Lon and P1Lat are the Longitude and Latitude coordinates of the first point of the polygon in decimal degrees (DDD) format (e.g. 2.17403, 41.40338) and so on.
The coordinates of the last point of the polygon must coincide with the coordinates of the first point of the polygon.
The polygon describing the geographical area can have a maximum of 200 points that must be within an area described by 10 degrees of latitude and 10 degrees of longitude.
Example:

POINT:

where the Latitude ( Lat ) and Longitude ( Lon ) values are expressed in decimal degrees (DDD) format (e.g. 41.40338, 2.17403 ).
Example:

Open Search Queries examples

The [query] in the open search URI will follow the same syntax used in the full text search. Here below we provide some examples.

Example Open Search
Searches every product with SLC product type or products containing the string "SLC" in the metadata https://scihub.copernicus.eu/dhus/search?q=SLC
Searches every product with SLC product type and S1 sensor Mode or products containing the strings "GRD" and "S1" in the metadata https://.copernicus.eu/dhus/search?q=GRD AND S1
Search every products ingested in the last day https://scihub.copernicus.eu/dhus/search?q=ingestionDate:[NOW-1DAY TO NOW]
Search every products ingested in the last month https://scihub.copernicus.eu/dhus/search?q=ingestionDate:[NOW-30DAYS TO NOW]
Search every products ingested in the last hour https://scihub.copernicus.eu/dhus/search?q=ingestionDate:[NOW-1HOUR TO NOW]
Search every products ingested in the last day with GRD product type https://scihub.copernicus.eu/dhus/search?q=productType:GRD AND ingestionDate:[NOW-1DAY TO NOW]
Search every products having sensing in the last three months https://scihub.copernicus.eu/dhus/search?q=beginPosition:[NOW-3MONTHS TO NOW] AND endPosition:[NOW-3MONTHS TO NOW]
Search every products having polarization mode VV covering the geographic area delimitated by the polygon having vertices: -4.53 29.85, 26.75 29.85, 26.75 46.80,-4.53 46.80,-4.53 29.85 https://scihub.copernicus.eu/dhus/search?q=polarisationmode:VV AND footprint:"Intersects(POLYGON((-4.53 29.85, 26.75 29.85, 26.75 46.80,-4.53 46.80,-4.53 29.85)))"

Other details and further examples are illustrated in the Full Text Search section.

Batch Scripting

The above OData and OpenSearch URIs can be combined to create complex queries to be executed in non-interactive scripts using programs like cURL and Wget.

Query via cURL

Using cURL it is possible to create a script to login to the Data Hub via the following command line:

where:

  • -u : option to specify user and password to use when fetching
  • <URI_QUERY> is a valid OData URI or OpenSearch URI.
Query via Wget

It is possible to use the wget command to create batch scripts:

Query via Wget

It is possible to use the wget command to create batch scripts:

where {USERNAME} is the valid account username, {PASSWORD} is the corresponding authentication password value and {FILE} is the name of the file where to print the output of the query. If '-' is used as {FILE}, documents will be printed to standard output.

The following example shows how to make an OpenSearch query using Wget. The query searches for all the products in the Data Hub archive. The first 25 results are printed in a file named query_results.txt:

  • wget --no-check-certificate --user={USERNAME} --password={PASSWORD} --output-document=query_results.txt 'https://scihub.copernicus.eu/dhus/search?q=*&rows=25'

The following example shows how to make an OpenSearch query using Wget for searching products filtered by product type and ingestion date :

  • wget --no-check-certificate --user={USERNAME} --password={PASSWORD} --output-document=query_results.txt 'https://scihub.copernicus.eu/dhus/search?q=ingestiondate:[NOW-1DAY TO NOW] AND producttype:SLC&rows=1000&start=0&format=json'
Download via Wget

It is also possible to download the products from the Data Hub archive using wget .
The following example shows how to download a single product, identified by its own Data Hub universally unique identifier {UUID}, using an OData URI:

  • wget --no-check-certificate --continue -user={USERNAME} --password={PASSWORD} 'https://scihub.copernicus.eu/dhus/odata/v1/Products('{UUID}')/'

The option --continue is very useful when downloads do not complete due to network problems. Wget will automatically try to continue the download from where it left off, and repeat this until the whole file has been retrieved.

The following example shows how to download the manifest file of a Sentinel-1 product using and Odata URI with Wget. onlyidentified by the universally unque identifier {UUID}:

  • wget --no-check-certificate --user={USERNAME} --password={PASSWORD} "https://scihub.copernicus.eu/dhus/odata/v1/Products('{UUID}')/Nodes('{PRODUCT_FILENAME}')/Nodes('manifest.safe')/"

where {UUID} is the value of the the universally unique identifier of the product, and {PRODUCT_FILENAME} is the filename of the product.

Scripts Examples
dhusget script

The example (dhusget) is a simple demo script illustrating how to use the OData and OpenSearch to query and download the products from the DHuS. It allows:

  1. Search products over a pre-defined AOI
  2. Filter the products by products type and/or ingestion time
  3. Download the products
  4. Download the manifest files only

You can download the script here
It requires the installation of wget .

Usage:
# dhusget.sh [-d <!DHuS URL> ] [-u <username> ] [ -p <password>] [-t <time to search (hours)>] [-c <coordinates ie: x1,y1;x2,y2>] [-T <product type>] [-o <option>]
The options are:

  • -u <username> : data hub username provided after registration on <!DHuS URL>
  • -p <password> : data hub password provided after registration on <!DHuS URL> (note: it's read from stdin, if isn't provided by command line)
  • -t <time to search (hours)> : time interval expressed in hours (integer value) from NOW (time of the launch of the dhusget) to backwards (e.g. insert the value '24' if you would like to retrieve product ingested in the last day)
  • -f <file> : a file containing the time of last successfully download
  • -c <coordinates ie: lat1,lon1:lat2,lon2> : coordinates of two opposite vertices of the rectangular area of interest
  • -T <product type> : product type of the product to search (available values are: SLC, GRD, OCN and RAW) ;
  • -o <option> : what to download; possible options are:
    • 'manifest' to download the manifest of all products returned from the search or
    • 'product' to download all products returned from the search
    • 'all' to download both

N.B.: if this parameter is left blank, the dhusget will return the UUID and the names of the products found in the DHuS archive.

odata-demo script

The script odata-demo is a demo script performing the following selective actions:

  1. List the collections
  2. List <n> products from a specified collection
  3. List first 10 products matching part of product name
  4. List first 10 products matching a specific ingestion date
  5. List first 10 products matching a specific aquisition date
  6. List first 10 products since last <n> days, by product type and intersecting an AOI
  7. Get product id from product name
  8. Get polarisation from a product id
  9. Get relative orbit from a product id
  10. Download Manifest file from a product id
  11. Download quick-look from a product id
  12. Download full product from its id

You can download the script here
It requires the installation of xmlstarlet .
Usage:
# odata-demo.sh [OPTIONS]
The options are:

  • -h, --help displays a help message
  • -j, --json use json output format for OData (default is xml)
  • -p, --passwd=PASSWORD use PASSWORD as password for the Data Hub Server
  • -s, --server=SERVER use SERVER as URL of the Data Hub Server
  • -u, --user=NAME use NAME as username for the Data Hub Server
  • -v, --verbose display curl command lines and results
  • -V, --version display the current version of the script

back to top © ESA